Merge pull request #1542 from fatedier/dev

bump version to v0.30.0

.github/ISSUE_TEMPLATE

@@ -1,5 +1,7 @@
 Issue is only used for submiting bug report and documents typo. If there are same issues or answers can be found in documents, we will close it directly.
 (为了节约时间，提高处理问题的效率，不按照格式填写的 issue 将会直接关闭。)
+(请不要在 issue 评论中出现无意义的 **加1**，**我也是** 等内容，将会被直接删除。)
+(由于个人精力有限，和系统环境，网络环境等相关的求助问题请转至其他论坛或社交平台。)
 
 Use the commands below to provide key information from your environment:
 You do NOT have to include this information if this is a FEATURE REQUEST

.travis.yml

@@ -2,7 +2,6 @@ sudo: false
 language: go
 
 go:
-    - 1.11.x
     - 1.12.x
 
 install:

Makefile

@@ -16,7 +16,7 @@ file:
 
 fmt:
 	go fmt ./...
-	
+
 frps:
 	go build -o bin/frps ./cmd/frps
 

README.md

@@ -6,53 +6,55 @@
 
 ## What is frp?
 
-frp is a fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet. As of now, it supports tcp & udp, as well as http and https protocols, where requests can be forwarded to internal services by domain name.
+frp is a fast reverse proxy to help you expose a local server behind a NAT or firewall to the Internet. As of now, it supports **TCP** and **UDP**, as well as **HTTP** and **HTTPS** protocols, where requests can be forwarded to internal services by domain name.
 
-Now it also try to support p2p connect.
+frp also has a P2P connect mode.
 
 ## Table of Contents
 
 <!-- vim-markdown-toc GFM -->
 
-* [Status](#status)
+* [Development Status](#development-status)
 * [Architecture](#architecture)
 * [Example Usage](#example-usage)
     * [Access your computer in LAN by SSH](#access-your-computer-in-lan-by-ssh)
     * [Visit your web service in LAN by custom domains](#visit-your-web-service-in-lan-by-custom-domains)
     * [Forward DNS query request](#forward-dns-query-request)
-    * [Forward unix domain socket](#forward-unix-domain-socket)
-    * [Expose a simple http file server](#expose-a-simple-http-file-server)
+    * [Forward Unix domain socket](#forward-unix-domain-socket)
+    * [Expose a simple HTTP file server](#expose-a-simple-http-file-server)
     * [Enable HTTPS for local HTTP service](#enable-https-for-local-http-service)
-    * [Expose your service in security](#expose-your-service-in-security)
+    * [Expose your service privately](#expose-your-service-privately)
     * [P2P Mode](#p2p-mode)
 * [Features](#features)
-    * [Configuration File](#configuration-file)
-    * [Configuration file template](#configuration-file-template)
+    * [Configuration Files](#configuration-files)
+    * [Using Environment Variables](#using-environment-variables)
     * [Dashboard](#dashboard)
     * [Admin UI](#admin-ui)
-    * [Authentication](#authentication)
+    * [Authenticating the Client](#authenticating-the-client)
     * [Encryption and Compression](#encryption-and-compression)
         * [TLS](#tls)
-    * [Hot-Reload frpc configuration](#hot-reload-frpc-configuration)
+    * [Hot-Reloading frpc configuration](#hot-reloading-frpc-configuration)
     * [Get proxy status from client](#get-proxy-status-from-client)
-    * [Port White List](#port-white-list)
+    * [Only allowing certain ports on the server](#only-allowing-certain-ports-on-the-server)
     * [Port Reuse](#port-reuse)
+    * [Bandwidth Limit](#bandwidth-limit)
+        * [For Each Proxy](#for-each-proxy)
     * [TCP Stream Multiplexing](#tcp-stream-multiplexing)
     * [Support KCP Protocol](#support-kcp-protocol)
-    * [Connection Pool](#connection-pool)
+    * [Connection Pooling](#connection-pooling)
     * [Load balancing](#load-balancing)
-    * [Health Check](#health-check)
-    * [Rewriting the Host Header](#rewriting-the-host-header)
-    * [Set Headers In HTTP Request](#set-headers-in-http-request)
+    * [Service Health Check](#service-health-check)
+    * [Rewriting the HTTP Host Header](#rewriting-the-http-host-header)
+    * [Setting other HTTP Headers](#setting-other-http-headers)
     * [Get Real IP](#get-real-ip)
         * [HTTP X-Forwarded-For](#http-x-forwarded-for)
         * [Proxy Protocol](#proxy-protocol)
-    * [Password protecting your web service](#password-protecting-your-web-service)
+    * [Require HTTP Basic auth (password) for web services](#require-http-basic-auth-password-for-web-services)
     * [Custom subdomain names](#custom-subdomain-names)
     * [URL routing](#url-routing)
-    * [Connect frps by HTTP PROXY](#connect-frps-by-http-proxy)
+    * [Connecting to frps via HTTP PROXY](#connecting-to-frps-via-http-proxy)
     * [Range ports mapping](#range-ports-mapping)
-    * [Plugin](#plugin)
+    * [Plugins](#plugins)
 * [Development Plan](#development-plan)
 * [Contributing](#contributing)
 * [Donation](#donation)
@@ -62,11 +64,11 @@ Now it also try to support p2p connect.
 
 <!-- vim-markdown-toc -->
 
-## Status
+## Development Status
 
-frp is under development and you can try it with latest release version. Master branch for releasing stable version when dev branch for developing.
+frp is under development. Try the latest release version in the `master` branch, or use the `dev` branch for the version in development.
 
-**We may change any protocol and can't promise backward compatible. Please check the release log when upgrading.**
+**The protocol might change at a release and we don't promise backwards compatibility. Please check the release log when upgrading the client and the server.**
 
 ## Architecture
 
@@ -74,15 +76,15 @@ frp is under development and you can try it with latest release version. Master
 
 ## Example Usage
 
-Firstly, download the latest programs from [Release](https://github.com/fatedier/frp/releases) page according to your os and arch.
+Firstly, download the latest programs from [Release](https://github.com/fatedier/frp/releases) page according to your operating system and architecture.
 
-Put **frps** and **frps.ini** to your server with public IP.
+Put `frps` and `frps.ini` onto your server A with public IP.
 
-Put **frpc** and **frpc.ini** to your server in LAN.
+Put `frpc` and `frpc.ini` onto your server B in LAN (that can't be connected from public Internet).
 
 ### Access your computer in LAN by SSH
 
-1. Modify frps.ini:
+1. Modify `frps.ini` on server A:
 
   ```ini
   # frps.ini
@@ -90,11 +92,11 @@ Put **frpc** and **frpc.ini** to your server in LAN.
   bind_port = 7000
   ```
 
-2. Start frps:
+2. Start `frps` on server A:
 
   `./frps -c ./frps.ini`
 
-3. Modify frpc.ini, `server_addr` is your frps's server IP:
+3. On server B, modify `frpc.ini` to put in your `frps` server public IP as `server_addr` field:
 
   ```ini
   # frpc.ini
@@ -109,21 +111,21 @@ Put **frpc** and **frpc.ini** to your server in LAN.
   remote_port = 6000
   ```
 
-4. Start frpc:
+4. Start `frpc` on server B:
 
   `./frpc -c ./frpc.ini`
 
-5. Connect to server in LAN by ssh assuming that username is test:
+5. From another machine, SSH to server B like this (assuming that username is `test`):
 
   `ssh -oPort=6000 test@x.x.x.x`
 
 ### Visit your web service in LAN by custom domains
 
-Sometimes we want to expose a local web service behind a NAT network to others for testing with your own domain name and unfortunately we can't resolve a domain name to a local ip.
+Sometimes we want to expose a local web service behind a NAT network to others for testing with your own domain name and unfortunately we can't resolve a domain name to a local IP.
 
-However, we can expose a http or https service using frp.
+However, we can expose an HTTP(S) service using frp.
 
-1. Modify frps.ini, configure http port 8080:
+1. Modify `frps.ini`, set the vhost HTTP port to 8080:
 
   ```ini
   # frps.ini
@@ -132,11 +134,11 @@ However, we can expose a http or https service using frp.
   vhost_http_port = 8080
   ```
 
-2. Start frps:
+2. Start `frps`:
 
   `./frps -c ./frps.ini`
 
-3. Modify frpc.ini and set remote frps server's IP as x.x.x.x. The `local_port` is the port of your web service:
+3. Modify `frpc.ini` and set `server_addr` to the IP address of the remote frps server. The `local_port` is the port of your web service:
 
   ```ini
   # frpc.ini
@@ -147,20 +149,20 @@ However, we can expose a http or https service using frp.
   [web]
   type = http
   local_port = 80
-  custom_domains = www.yourdomain.com
+  custom_domains = www.example.com
   ```
 
-4. Start frpc:
+4. Start `frpc`:
 
   `./frpc -c ./frpc.ini`
 
-5. Resolve A record of `www.yourdomain.com` to IP `x.x.x.x` or CNAME record to your origin domain.
+5. Resolve A record of `www.example.com` to the public IP of the remote frps server or CNAME record to your origin domain.
 
-6. Now visit your local web service using url `http://www.yourdomain.com:8080`.
+6. Now visit your local web service using url `http://www.example.com:8080`.
 
 ### Forward DNS query request
 
-1. Modify frps.ini:
+1. Modify `frps.ini`:
 
   ```ini
   # frps.ini
@@ -168,11 +170,11 @@ However, we can expose a http or https service using frp.
   bind_port = 7000
   ```
 
-2. Start frps:
+2. Start `frps`:
 
   `./frps -c ./frps.ini`
 
-3. Modify frpc.ini, set remote frps's server IP as x.x.x.x, forward dns query request to google dns server `8.8.8.8:53`:
+3. Modify `frpc.ini` and set `server_addr` to the IP address of the remote frps server, forward DNS query request to Google Public DNS server `8.8.8.8:53`:
 
   ```ini
   # frpc.ini
@@ -191,17 +193,17 @@ However, we can expose a http or https service using frp.
 
   `./frpc -c ./frpc.ini`
 
-5. Send dns query request by dig:
+5. Test DNS resolution using `dig` command:
 
   `dig @x.x.x.x -p 6000 www.google.com`
 
-### Forward unix domain socket
+### Forward Unix domain socket
 
-Using tcp port to connect unix domain socket like docker daemon.
+Expose a Unix domain socket (e.g. the Docker daemon socket) as TCP.
 
-Configure frps same as above.
+Configure `frps` same as above.
 
-1. Start frpc with configurations:
+1. Start `frpc` with configuration:
 
   ```ini
   # frpc.ini
@@ -216,17 +218,17 @@ Configure frps same as above.
   plugin_unix_path = /var/run/docker.sock
   ```
 
-2. Get docker version by curl command:
+2. Test: Get Docker version using `curl`:
 
   `curl http://x.x.x.x:6000/version`
 
-### Expose a simple http file server
+### Expose a simple HTTP file server
 
-A simple way to visit files in the LAN.
+Browser your files stored in the LAN, from public Internet.
 
-Configure frps same as above.
+Configure `frps` same as above.
 
-1. Start frpc with configurations:
+1. Start `frpc` with configuration:
 
   ```ini
   # frpc.ini
@@ -238,17 +240,17 @@ Configure frps same as above.
   type = tcp
   remote_port = 6000
   plugin = static_file
-  plugin_local_path = /tmp/file
+  plugin_local_path = /tmp/files
   plugin_strip_prefix = static
   plugin_http_user = abc
   plugin_http_passwd = abc
   ```
 
-2. Visit `http://x.x.x.x:6000/static/` by your browser, set correct user and password, so you can see files in `/tmp/file`.
+2. Visit `http://x.x.x.x:6000/static/` from your browser and specify correct user and password to view files in `/tmp/files` on the `frpc` machine.
 
 ### Enable HTTPS for local HTTP service
 
-1. Start frpc with configurations:
+1. Start `frpc` with configuration:
 
   ```ini
   # frpc.ini
@@ -256,28 +258,27 @@ Configure frps same as above.
   server_addr = x.x.x.x
   server_port = 7000
 
-  [test_htts2http]
+  [test_https2http]
   type = https
-  custom_domains = test.yourdomain.com
+  custom_domains = test.example.com
 
   plugin = https2http
   plugin_local_addr = 127.0.0.1:80
   plugin_crt_path = ./server.crt
   plugin_key_path = ./server.key
   plugin_host_header_rewrite = 127.0.0.1
+  plugin_header_X-From-Where = frp
   ```
 
-2. Visit `https://test.yourdomain.com`.
+2. Visit `https://test.example.com`.
 
-### Expose your service in security
+### Expose your service privately
 
-For some services, if expose them to the public network directly will be a security risk.
+Some services will be at risk if exposed directly to the public network. With **STCP** (secret TCP) mode, a preshared key is needed to access the service from another client.
 
-**stcp(secret tcp)** helps you create a proxy avoiding any one can access it.
+Configure `frps` same as above.
 
-Configure frps same as above.
-
-1. Start frpc, forward ssh port and `remote_port` is useless:
+1. Start `frpc` on machine B with the following config. This example is for exposing the SSH service (port 22), and note the `sk` field for the preshared key, and that the `remote_port` field is removed here:
 
   ```ini
   # frpc.ini
@@ -292,7 +293,7 @@ Configure frps same as above.
   local_port = 22
   ```
 
-2. Start another frpc in which you want to connect this ssh server:
+2. Start another `frpc` (typically on another machine C) with the following config to access the SSH service with a security key (`sk` field):
 
   ```ini
   # frpc.ini
@@ -309,23 +310,24 @@ Configure frps same as above.
   bind_port = 6000
   ```
 
-3. Connect to server in LAN by ssh assuming that username is test:
+3. On machine C, connect to SSH on machine B, using this command:
 
-  `ssh -oPort=6000 test@127.0.0.1`
+  `ssh -oPort=6000 127.0.0.1`
 
 ### P2P Mode
 
-**xtcp** is designed for transmitting a large amount of data directly between two client.
+**xtcp** is designed for transmitting large amounts of data directly between clients. A frps server is still needed, as P2P here only refers the actual data transmission.
 
-Now it can't penetrate all types of NAT devices. You can try **stcp** if **xtcp** doesn't work.
+Note it can't penetrate all types of NAT devices. You might want to fallback to **stcp** if **xtcp** doesn't work.
 
-1. Configure a udp port for xtcp:
+1. In `frps.ini` configure a UDP port for xtcp:
 
   ```ini
+  # frps.ini
   bind_udp_port = 7001
   ```
 
-2. Start frpc, forward ssh port and `remote_port` is useless:
+2. Start `frpc` on machine B, expose the SSH port. Note that `remote_port` field is removed:
 
   ```ini
   # frpc.ini
@@ -340,7 +342,7 @@ Now it can't penetrate all types of NAT devices. You can try **stcp** if **xtcp*
   local_port = 22
   ```
 
-3. Start another frpc in which you want to connect this ssh server:
+3. Start another `frpc` (typically on another machine C) with the config to connect to SSH using P2P mode:
 
   ```ini
   # frpc.ini
@@ -357,23 +359,23 @@ Now it can't penetrate all types of NAT devices. You can try **stcp** if **xtcp*
   bind_port = 6000
   ```
 
-4. Connect to server in LAN by ssh assuming that username is test:
+4. On machine C, connect to SSH on machine B, using this command:
 
-  `ssh -oPort=6000 test@127.0.0.1`
+  `ssh -oPort=6000 127.0.0.1`
 
 ## Features
 
-### Configuration File
+### Configuration Files
 
-You can find features which this document not metioned from full example configuration files.
+Read the full example configuration files to find out even more features not described here.
 
-[frps full configuration file](./conf/frps_full.ini)
+[Full configuration file for frps (Server)](./conf/frps_full.ini)
 
-[frpc full configuration file](./conf/frpc_full.ini)
+[Full configuration file for frpc (Client)](./conf/frpc_full.ini)
 
-### Configuration file template
+### Using Environment Variables
 
-Configuration file tempalte can be rendered using os environments. Template uses Go's standard format.
+Environment variables can be referenced in the configuration file, using Go's standard format:
 
 ```ini
 # frpc.ini
@@ -388,7 +390,7 @@ local_port = 22
 remote_port = {{ .Envs.FRP_SSH_REMOTE_PORT }}
 ```
 
-Start frpc program:
+With the config above, variables can be passed into `frpc` program like this:
 
 ```
 export FRP_SERVER_ADDR="x.x.x.x"
@@ -396,12 +398,11 @@ export FRP_SSH_REMOTE_PORT="6000"
 ./frpc -c ./frpc.ini
 ```
 
-frpc will auto render configuration file template using os environments.
-All environments has prefix `.Envs`.
+`frpc` will render configuration file template using OS environment variables. Remember to prefix your reference with `.Envs`.
 
 ### Dashboard
 
-Check frp's status and proxies's statistics information by Dashboard.
+Check frp's status and proxies' statistics information by Dashboard.
 
 Configure a port for dashboard to enable this feature:
 
@@ -413,15 +414,15 @@ dashboard_user = admin
 dashboard_pwd = admin
 ```
 
-Then visit `http://[server_addr]:7500` to see dashboard, default username and password are both `admin`.
+Then visit `http://[server_addr]:7500` to see the dashboard, with username and password both being `admin` by default.
 
 ![dashboard](/doc/pic/dashboard.png)
 
 ### Admin UI
 
-Admin UI help you check and manage frpc's configure.
+The Admin UI helps you check and manage frpc's configuration.
 
-Configure a address for admin UI to enable this feature:
+Configure an address for admin UI to enable this feature:
 
 ```ini
 [common]
@@ -431,15 +432,15 @@ admin_user = admin
 admin_pwd = admin
 ```
 
-Then visit `http://127.0.0.1:7400` to see admin UI, default username and password are both `admin`.
+Then visit `http://127.0.0.1:7400` to see admin UI, with username and password both being `admin` by default.
 
-### Authentication
+### Authenticating the Client
 
-`token` in frps.ini and frpc.ini should be same.
+Always use the same `token` in the `[common]` section in `frps.ini` and `frpc.ini`.
 
 ### Encryption and Compression
 
-Defalut value is false, you could decide if the proxy will use encryption or compression:
+The features are off by default. You can turn on encryption and/or compression:
 
 ```ini
 # frpc.ini
@@ -453,15 +454,15 @@ use_compression = true
 
 #### TLS
 
-frp support TLS protocol between frpc and frps since v0.25.0.
+frp supports the TLS protocol between `frpc` and `frps` since v0.25.0.
 
-Config `tls_enable = true` in `common` section to frpc.ini to enable this feature.
+Config `tls_enable = true` in the `[common]` section to `frpc.ini` to enable this feature.
 
-For port multiplexing, frp send a first byte 0x17 to dial a TLS connection.
+For port multiplexing, frp sends a first byte `0x17` to dial a TLS connection.
 
-### Hot-Reload frpc configuration
+### Hot-Reloading frpc configuration
 
-First you need to set admin port in frpc's configure file to let it provide HTTP API for more features.
+The `admin_addr` and `admin_port` fields are required for enabling HTTP API:
 
 ```ini
 # frpc.ini
@@ -470,17 +471,17 @@ admin_addr = 127.0.0.1
 admin_port = 7400
 ```
 
-Then run command `frpc reload -c ./frpc.ini` and wait for about 10 seconds to let frpc create or update or delete proxies.
+Then run command `frpc reload -c ./frpc.ini` and wait for about 10 seconds to let `frpc` create or update or delete proxies.
 
-**Note that parameters in [common] section won't be modified except 'start' now.**
+**Note that parameters in [common] section won't be modified except 'start'.**
 
 ### Get proxy status from client
 
-Use `frpc status -c ./frpc.ini` to get status of all proxies. You need to set admin port in frpc's configure file.
+Use `frpc status -c ./frpc.ini` to get status of all proxies. The `admin_addr` and `admin_port` fields are required for enabling HTTP API.
 
-### Port White List
+### Only allowing certain ports on the server
 
-`allow_ports` in frps.ini is used for preventing abuse of ports:
+`allow_ports` in `frps.ini` is used to avoid abuse of ports:
 
 ```ini
 # frps.ini
@@ -488,19 +489,34 @@ Use `frpc status -c ./frpc.ini` to get status of all proxies. You need to set ad
 allow_ports = 2000-3000,3001,3003,4000-50000
 ```
 
-`allow_ports` consists of a specific port or a range of ports divided by `,`.
+`allow_ports` consists of specific ports or port ranges (lowest port number, dash `-`, highest port number), separated by comma `,`.
 
 ### Port Reuse
 
-Now `vhost_http_port` and `vhost_https_port` in frps can use same port with `bind_port`. frps will detect connection's protocol and handle it correspondingly.
+`vhost_http_port` and `vhost_https_port` in frps can use same port with `bind_port`. frps will detect the connection's protocol and handle it correspondingly.
 
 We would like to try to allow multiple proxies bind a same remote port with different protocols in the future.
 
+### Bandwidth Limit
+
+#### For Each Proxy
+
+```ini
+# frpc.ini
+[ssh]
+type = tcp
+local_port = 22
+remote_port = 6000
+bandwidth_limit = 1MB
+```
+
+Set `bandwidth_limit` in each proxy's configure to enable this feature. Supported units are `MB` and `KB`.
+
 ### TCP Stream Multiplexing
 
-frp support tcp stream multiplexing since v0.10.0 like HTTP2 Multiplexing. All user requests to same frpc can use only one tcp connection.
+frp supports tcp stream multiplexing since v0.10.0 like HTTP2 Multiplexing, in which case all logic connections to the same frpc are multiplexed into the same TCP connection.
 
-You can disable this feature by modify frps.ini and frpc.ini:
+You can disable this feature by modify `frps.ini` and `frpc.ini`:
 
 ```ini
 # frps.ini and frpc.ini, must be same
@@ -512,36 +528,38 @@ tcp_mux = false
 
 KCP is a fast and reliable protocol that can achieve the transmission effect of a reduction of the average latency by 30% to 40% and reduction of the maximum delay by a factor of three, at the cost of 10% to 20% more bandwidth wasted than TCP.
 
-Using kcp in frp:
+KCP mode uses UDP as the underlying transport. Using KCP in frp:
 
-1. Enable kcp protocol in frps:
+1. Enable KCP in frps:
 
   ```ini
   # frps.ini
   [common]
   bind_port = 7000
-  # kcp needs to bind a udp port, it can be same with 'bind_port'
+  # Specify a UDP port for KCP.
   kcp_bind_port = 7000
   ```
 
-2. Configure the protocol used in frpc to connect frps:
+  The `kcp_bind_port` number can be the same number as `bind_port`, since `bind_port` field specifies a TCP port.
+
+2. Configure `frpc.ini` to use KCP to connect to frps:
 
   ```ini
   # frpc.ini
   [common]
   server_addr = x.x.x.x
-  # specify the 'kcp_bind_port' in frps
+  # Same as the 'kcp_bind_port' in frps.ini
   server_port = 7000
   protocol = kcp
   ```
 
-### Connection Pool
+### Connection Pooling
 
-By default, frps send message to frpc for create a new connection to backward service when getting an user request.If a proxy's connection pool is enabled, there will be a specified number of connections pre-established.
+By default, frps creates a new frpc connection to the backend service upon a user request. With connection pooling, frps keeps a certain number of pre-established connections, reducing the time needed to establish a connection.
 
-This feature is fit for a large number of short connections.
+This feature is suitable for a large number of short connections.
 
-1. Configure the limit of pool count each proxy can use in frps.ini:
+1. Configure the limit of pool count each proxy can use in `frps.ini`:
 
   ```ini
   # frps.ini
@@ -560,7 +578,8 @@ This feature is fit for a large number of short connections.
 ### Load balancing
 
 Load balancing is supported by `group`.
-This feature is available only for type `tcp` now.
+
+This feature is only available for types `tcp` and `http` now.
 
 ```ini
 # frpc.ini
@@ -581,19 +600,19 @@ group_key = 123
 
 `group_key` is used for authentication.
 
-Proxies in same group will accept connections from port 80 randomly.
+Connections to port 80 will be dispatched to proxies in the same group randomly.
 
-### Health Check
+For type `tcp`, `remote_port` in the same group should be the same.
+
+For type `http`, `custom_domains`, `subdomain`, `locations` should be the same.
+
+### Service Health Check
 
 Health check feature can help you achieve high availability with load balancing.
 
-Add `health_check_type = {type}` to enable health check.
+Add `health_check_type = tcp` or `health_check_type = http` to enable health check.
 
-**type** can be tcp or http.
-
-Type tcp will dial the service port and type http will send a http rquest to service and require a 200 response.
-
-Type tcp configuration:
+With health check type **tcp**, the service port will be pinged (TCPing):
 
 ```ini
 # frpc.ini
@@ -601,77 +620,81 @@ Type tcp configuration:
 type = tcp
 local_port = 22
 remote_port = 6000
-# enable tcp health check
+# Enable TCP health check
 health_check_type = tcp
-# dial timeout seconds
+# TCPing timeout seconds
 health_check_timeout_s = 3
-# if continuous failed in 3 times, the proxy will be removed from frps
+# If health check failed 3 times in a row, the proxy will be removed from frps
 health_check_max_failed = 3
-# every 10 seconds will do a health check
+# A health check every 10 seconds
 health_check_interval_s = 10
 ```
 
-Type http configuration:
+With health check type **http**, an HTTP request will be sent to the service and an HTTP 2xx OK response is expected:
+
 ```ini
 # frpc.ini
 [web]
 type = http
 local_ip = 127.0.0.1
 local_port = 80
-custom_domains = test.yourdomain.com
-# enable http health check
+custom_domains = test.example.com
+# Enable HTTP health check
 health_check_type = http
-# frpc will send a GET http request '/status' to local http service
-# http service is alive when it return 2xx http response code
+# frpc will send a GET request to '/status'
+# and expect an HTTP 2xx OK response
 health_check_url = /status
-health_check_interval_s = 10
-health_check_max_failed = 3
 health_check_timeout_s = 3
+health_check_max_failed = 3
+health_check_interval_s = 10
 ```
 
-### Rewriting the Host Header
+### Rewriting the HTTP Host Header
 
-When forwarding to a local port, frp does not modify the tunneled HTTP requests at all, they are copied to your server byte-for-byte as they are received. Some application servers use the Host header for determining which development site to display. For this reason, frp can rewrite your requests with a modified host header. Use the `host_header_rewrite` switch to rewrite incoming HTTP requests.
+By default frp does not modify the tunneled HTTP requests at all as it's a byte-for-byte copy.
+
+However, speaking of web servers and HTTP requests, your web server might rely on the `Host` HTTP header to determine the website to be accessed. frp can rewrite the `Host` header when forwarding the HTTP requests, with the `host_header_rewrite` field:
 
 ```ini
 # frpc.ini
 [web]
 type = http
 local_port = 80
-custom_domains = test.yourdomain.com
-host_header_rewrite = dev.yourdomain.com
+custom_domains = test.example.com
+host_header_rewrite = dev.example.com
 ```
 
-The `Host` request header will be rewritten to `Host: dev.yourdomain.com` before it reach your local http server.
+The HTTP request will have the the `Host` header rewritten to `Host: dev.example.com` when it reaches the actual web server, although the request from the browser probably has `Host: test.example.com`.
 
-### Set Headers In HTTP Request
+### Setting other HTTP Headers
 
-You can set headers for proxy which type is `http`.
+Similar to `Host`, You can override other HTTP request headers with proxy type `http`.
 
 ```ini
 # frpc.ini
 [web]
 type = http
 local_port = 80
-custom_domains = test.yourdomain.com
-host_header_rewrite = dev.yourdomain.com
+custom_domains = test.example.com
+host_header_rewrite = dev.example.com
 header_X-From-Where = frp
 ```
 
-Note that params which have prefix `header_` will be added to http request headers.
-In this example, it will set header `X-From-Where: frp` to http request.
+Note that parameter(s) prefixed with `header_` will be added to HTTP request headers.
+
+In this example, it will set header `X-From-Where: frp` in the HTTP request.
 
 ### Get Real IP
 
 #### HTTP X-Forwarded-For
 
-Features for http proxy only.
+This feature is for http proxy only.
 
-You can get user's real IP from HTTP request header `X-Forwarded-For` and `X-Real-IP`.
+You can get user's real IP from HTTP request headers `X-Forwarded-For` and `X-Real-IP`.
 
 #### Proxy Protocol
 
-frp support Proxy Protocol to send user's real IP to local service. It support all types without UDP.
+frp supports Proxy Protocol to send user's real IP to local services. It support all types except UDP.
 
 Here is an example for https service:
 
@@ -680,21 +703,19 @@ Here is an example for https service:
 [web]
 type = https
 local_port = 443
-custom_domains = test.yourdomain.com
+custom_domains = test.example.com
 
-# now v1 and v2 is supported
+# now v1 and v2 are supported
 proxy_protocol_version = v2
 ```
 
-You can enable Proxy Protocol support in nginx to parse user's real IP to http header `X-Real-IP`.
+You can enable Proxy Protocol support in nginx to expose user's real IP in HTTP header `X-Real-IP`, and then read `X-Real-IP` header in your web service for the real IP.
 
-Then you can get it from HTTP request header in your local service.
-
-### Password protecting your web service
+### Require HTTP Basic auth (password) for web services
 
 Anyone who can guess your tunnel URL can access your local web server unless you protect it with a password.
 
-This enforces HTTP Basic Auth on all requests with the username and password you specify in frpc's configure file.
+This enforces HTTP Basic Auth on all requests with the username and password specified in frpc's configure file.
 
 It can only be enabled when proxy type is http.
 
@@ -703,23 +724,23 @@ It can only be enabled when proxy type is http.
 [web]
 type = http
 local_port = 80
-custom_domains = test.yourdomain.com
+custom_domains = test.example.com
 http_user = abc
 http_pwd = abc
 ```
 
-Visit `http://test.yourdomain.com` and now you need to input username and password.
+Visit `http://test.example.com` in the browser and now you are prompted to enter the username and password.
 
 ### Custom subdomain names
 
-It is convenient to use `subdomain` configure for http、https type when many people use one frps server together.
+It is convenient to use `subdomain` configure for http and https types when many people share one frps server.
 
 ```ini
 # frps.ini
 subdomain_host = frps.com
 ```
 
-Resolve `*.frps.com` to the frps server's IP.
+Resolve `*.frps.com` to the frps server's IP. This is usually called a Wildcard DNS record.
 
 ```ini
 # frpc.ini
@@ -729,35 +750,36 @@ local_port = 80
 subdomain = test
 ```
 
-Now you can visit your web service by host `test.frps.com`.
+Now you can visit your web service on `test.frps.com`.
 
 Note that if `subdomain_host` is not empty, `custom_domains` should not be the subdomain of `subdomain_host`.
 
 ### URL routing
 
-frp support forward http requests to different backward web services by url routing.
+frp supports forwarding HTTP requests to different backend web services by url routing.
 
-`locations` specify the prefix of URL used for routing. frps first searches for the most specific prefix location given by literal strings regardless of the listed order.
+`locations` specifies the prefix of URL used for routing. frps first searches for the most specific prefix location given by literal strings regardless of the listed order.
 
 ```ini
 # frpc.ini
 [web01]
 type = http
 local_port = 80
-custom_domains = web.yourdomain.com
+custom_domains = web.example.com
 locations = /
 
 [web02]
 type = http
 local_port = 81
-custom_domains = web.yourdomain.com
+custom_domains = web.example.com
 locations = /news,/about
 ```
-Http requests with url prefix `/news` and `/about` will be forwarded to **web02** and others to **web01**.
 
-### Connect frps by HTTP PROXY
+HTTP requests with URL prefix `/news` or `/about` will be forwarded to **web02** and other requests to **web01**.
 
-frpc can connect frps using HTTP PROXY if you set os environment `HTTP_PROXY` or configure `http_proxy` param in frpc.ini file.
+### Connecting to frps via HTTP PROXY
+
+frpc can connect to frps using HTTP proxy if you set OS environment variable `HTTP_PROXY`, or if `http_proxy` is set in frpc.ini file.
 
 It only works when protocol is tcp.
 
@@ -771,7 +793,7 @@ http_proxy = http://user:pwd@192.168.1.128:8080
 
 ### Range ports mapping
 
-Proxy name has prefix `range:` will support mapping range ports.
+Proxy with names that start with `range:` will support mapping range ports.
 
 ```ini
 # frpc.ini
@@ -782,15 +804,15 @@ local_port = 6000-6006,6007
 remote_port = 6000-6006,6007
 ```
 
-frpc will generate 8 proxies like `test_tcp_0, test_tcp_1 ... test_tcp_7`.
+frpc will generate 8 proxies like `test_tcp_0`, `test_tcp_1`, ..., `test_tcp_7`.
 
-### Plugin
+### Plugins
 
-frpc only forward request to local tcp or udp port by default.
+frpc only forwards requests to local TCP or UDP ports by default.
 
-Plugin is used for providing rich features. There are built-in plugins such as `unix_domain_socket`, `http_proxy`, `socks5`, `static_file` and you can see [example usage](#example-usage).
+Plugins are used for providing rich features. There are built-in plugins such as `unix_domain_socket`, `http_proxy`, `socks5`, `static_file` and you can see [example usage](#example-usage).
 
-Specify which plugin to use by `plugin` parameter. Configuration parameters of plugin should be started with `plugin_`. `local_ip` and `local_port` is useless for plugin.
+Specify which plugin to use with the `plugin` parameter. Configuration parameters of plugin should be started with `plugin_`. `local_ip` and `local_port` are not used for plugin.
 
 Using plugin **http_proxy**:
 
@@ -808,7 +830,7 @@ plugin_http_passwd = abc
 
 ## Development Plan
 
-* Log http request information in frps.
+* Log HTTP request information in frps.
 
 ## Contributing
 
@@ -816,14 +838,14 @@ Interested in getting involved? We would like to help you!
 
 * Take a look at our [issues list](https://github.com/fatedier/frp/issues) and consider sending a Pull Request to **dev branch**.
 * If you want to add a new feature, please create an issue first to describe the new feature, as well as the implementation approach. Once a proposal is accepted, create an implementation of the new features and submit it as a pull request.
-* Sorry for my poor english and improvement for this document is welcome even some typo fix.
-* If you have some wonderful ideas, send email to fatedier@gmail.com.
+* Sorry for my poor English. Improvements for this document are welcome, even some typo fixes.
+* If you have great ideas, send an email to fatedier@gmail.com.
 
-**Note: We prefer you to give your advise in [issues](https://github.com/fatedier/frp/issues), so others with a same question can search it quickly and we don't need to answer them repeatly.**
+**Note: We prefer you to give your advise in [issues](https://github.com/fatedier/frp/issues), so others with a same question can search it quickly and we don't need to answer them repeatedly.**
 
 ## Donation
 
-If frp help you a lot, you can support us by:
+If frp helps you a lot, you can support us by:
 
 frp QQ group: 606194980
 

README_zh.md

@@ -16,7 +16,7 @@ frp 是一个可用于内网穿透的高性能的反向代理应用，支持 tcp
     * [通过 ssh 访问公司内网机器](#通过-ssh-访问公司内网机器)
     * [通过自定义域名访问部署于内网的 web 服务](#通过自定义域名访问部署于内网的-web-服务)
     * [转发 DNS 查询请求](#转发-dns-查询请求)
-    * [转发 Unix域套接字](#转发-unix域套接字)
+    * [转发 Unix 域套接字](#转发-unix-域套接字)
     * [对外提供简单的文件访问服务](#对外提供简单的文件访问服务)
     * [为本地 HTTP 服务启用 HTTPS](#为本地-http-服务启用-https)
     * [安全地暴露内网服务](#安全地暴露内网服务)
@@ -33,6 +33,8 @@ frp 是一个可用于内网穿透的高性能的反向代理应用，支持 tcp
     * [客户端查看代理状态](#客户端查看代理状态)
     * [端口白名单](#端口白名单)
     * [端口复用](#端口复用)
+    * [限速](#限速)
+        * [代理限速](#代理限速)
     * [TCP 多路复用](#tcp-多路复用)
     * [底层通信可选 kcp 协议](#底层通信可选-kcp-协议)
     * [连接池](#连接池)
@@ -129,7 +131,7 @@ master 分支用于发布稳定版本，dev 分支用于开发，您可以尝试
   vhost_http_port = 8080
   ```
 
-2. 启动 frps；
+2. 启动 frps：
 
   `./frps -c ./frps.ini`
 
@@ -194,7 +196,7 @@ DNS 查询请求通常使用 UDP 协议，frp 支持对内网 UDP 服务的穿
 
   `dig @x.x.x.x -p 6000 www.google.com`
 
-### 转发 Unix域套接字
+### 转发 Unix 域套接字
 
 通过 tcp 端口访问内网的 unix域套接字(例如和 docker daemon 通信)。
 
@@ -270,6 +272,7 @@ frps 的部署步骤同上。
   plugin_crt_path = ./server.crt
   plugin_key_path = ./server.key
   plugin_host_header_rewrite = 127.0.0.1
+  plugin_header_X-From-Where = frp
   ```
 
 2. 通过浏览器访问 `https://test.yourdomain.com` 即可。
@@ -530,6 +533,23 @@ allow_ports = 2000-3000,3001,3003,4000-50000
 
 后续会尝试允许多个 proxy 绑定同一个远端端口的不同协议。
 
+### 限速
+
+#### 代理限速
+
+目前支持在客户端的代理配置中设置代理级别的限速，限制单个 proxy 可以占用的带宽。
+
+```ini
+# frpc.ini
+[ssh]
+type = tcp
+local_port = 22
+remote_port = 6000
+bandwith_limit = 1MB
+```
+
+在代理配置中增加 `bandwith_limit` 字段启用此功能，目前仅支持 `MB` 和 `KB` 单位。
+
 ### TCP 多路复用
 
 从 v0.10.0 版本开始，客户端和服务器端之间的连接支持多路复用，不再需要为每一个用户请求创建一个连接，使连接建立的延迟降低，并且避免了大量文件描述符的占用，使 frp 可以承载更高的并发数。
@@ -597,7 +617,7 @@ tcp_mux = false
 
 可以将多个相同类型的 proxy 加入到同一个 group 中，从而实现负载均衡的功能。
 
-目前只支持 tcp 类型的 proxy。
+目前只支持 TCP 和 HTTP 类型的 proxy。
 
 ```ini
 # frpc.ini
@@ -618,7 +638,9 @@ group_key = 123
 
 用户连接 frps 服务器的 80 端口，frps 会将接收到的用户连接随机分发给其中一个存活的 proxy。这样可以在一台 frpc 机器挂掉后仍然有其他节点能够提供服务。
 
-要求 `group_key` 相同，做权限验证，且 `remote_port` 相同。
+TCP 类型代理要求 `group_key` 相同，做权限验证，且 `remote_port` 相同。
+
+HTTP 类型代理要求 `group_key, custom_domains 或 subdomain 和 locations` 相同。
 
 ### 健康检查
 

assets/assets.go

@@ -55,6 +55,7 @@ func ReadFile(file string) (content string, err error) {
 		if err != nil {
 			return content, err
 		}
+		defer file.Close()
 		buf, err := ioutil.ReadAll(file)
 		if err != nil {
 			return content, err
@@ -65,6 +66,7 @@ func ReadFile(file string) (content string, err error) {
 		if err != nil {
 			return content, err
 		}
+		defer file.Close()
 		buf, err := ioutil.ReadAll(file)
 		if err != nil {
 			return content, err

client/admin.go

@@ -21,7 +21,6 @@ import (
 	"time"
 
 	"github.com/fatedier/frp/assets"
-	"github.com/fatedier/frp/g"
 	frpNet "github.com/fatedier/frp/utils/net"
 
 	"github.com/gorilla/mux"
@@ -36,7 +35,7 @@ func (svr *Service) RunAdminServer(addr string, port int) (err error) {
 	// url router
 	router := mux.NewRouter()
 
-	user, passwd := g.GlbClientCfg.AdminUser, g.GlbClientCfg.AdminPwd
+	user, passwd := svr.cfg.AdminUser, svr.cfg.AdminPwd
 	router.Use(frpNet.NewHttpAuthMiddleware(user, passwd).Middleware)
 
 	// api, see dashboard_api.go

client/admin_api.go

@@ -23,7 +23,6 @@ import (
 	"strings"
 
 	"github.com/fatedier/frp/client/proxy"
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/utils/log"
 )
@@ -47,7 +46,7 @@ func (svr *Service) apiReload(w http.ResponseWriter, r *http.Request) {
 		}
 	}()
 
-	content, err := config.GetRenderedConfFromFile(g.GlbClientCfg.CfgFile)
+	content, err := config.GetRenderedConfFromFile(svr.cfgFile)
 	if err != nil {
 		res.Code = 400
 		res.Msg = err.Error()
@@ -55,7 +54,7 @@ func (svr *Service) apiReload(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
-	newCommonCfg, err := config.UnmarshalClientConfFromIni(nil, content)
+	newCommonCfg, err := config.UnmarshalClientConfFromIni(content)
 	if err != nil {
 		res.Code = 400
 		res.Msg = err.Error()
@@ -63,7 +62,7 @@ func (svr *Service) apiReload(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
-	pxyCfgs, visitorCfgs, err := config.LoadAllConfFromIni(g.GlbClientCfg.User, content, newCommonCfg.Start)
+	pxyCfgs, visitorCfgs, err := config.LoadAllConfFromIni(svr.cfg.User, content, newCommonCfg.Start)
 	if err != nil {
 		res.Code = 400
 		res.Msg = err.Error()
@@ -107,7 +106,7 @@ func (a ByProxyStatusResp) Len() int           { return len(a) }
 func (a ByProxyStatusResp) Swap(i, j int)      { a[i], a[j] = a[j], a[i] }
 func (a ByProxyStatusResp) Less(i, j int) bool { return strings.Compare(a[i].Name, a[j].Name) < 0 }
 
-func NewProxyStatusResp(status *proxy.ProxyStatus) ProxyStatusResp {
+func NewProxyStatusResp(status *proxy.ProxyStatus, serverAddr string) ProxyStatusResp {
 	psr := ProxyStatusResp{
 		Name:   status.Name,
 		Type:   status.Type,
@@ -121,18 +120,18 @@ func NewProxyStatusResp(status *proxy.ProxyStatus) ProxyStatusResp {
 		}
 		psr.Plugin = cfg.Plugin
 		if status.Err != "" {
-			psr.RemoteAddr = fmt.Sprintf("%s:%d", g.GlbClientCfg.ServerAddr, cfg.RemotePort)
+			psr.RemoteAddr = fmt.Sprintf("%s:%d", serverAddr, cfg.RemotePort)
 		} else {
-			psr.RemoteAddr = g.GlbClientCfg.ServerAddr + status.RemoteAddr
+			psr.RemoteAddr = serverAddr + status.RemoteAddr
 		}
 	case *config.UdpProxyConf:
 		if cfg.LocalPort != 0 {
 			psr.LocalAddr = fmt.Sprintf("%s:%d", cfg.LocalIp, cfg.LocalPort)
 		}
 		if status.Err != "" {
-			psr.RemoteAddr = fmt.Sprintf("%s:%d", g.GlbClientCfg.ServerAddr, cfg.RemotePort)
+			psr.RemoteAddr = fmt.Sprintf("%s:%d", serverAddr, cfg.RemotePort)
 		} else {
-			psr.RemoteAddr = g.GlbClientCfg.ServerAddr + status.RemoteAddr
+			psr.RemoteAddr = serverAddr + status.RemoteAddr
 		}
 	case *config.HttpProxyConf:
 		if cfg.LocalPort != 0 {
@@ -184,17 +183,17 @@ func (svr *Service) apiStatus(w http.ResponseWriter, r *http.Request) {
 	for _, status := range ps {
 		switch status.Type {
 		case "tcp":
-			res.Tcp = append(res.Tcp, NewProxyStatusResp(status))
+			res.Tcp = append(res.Tcp, NewProxyStatusResp(status, svr.cfg.ServerAddr))
 		case "udp":
-			res.Udp = append(res.Udp, NewProxyStatusResp(status))
+			res.Udp = append(res.Udp, NewProxyStatusResp(status, svr.cfg.ServerAddr))
 		case "http":
-			res.Http = append(res.Http, NewProxyStatusResp(status))
+			res.Http = append(res.Http, NewProxyStatusResp(status, svr.cfg.ServerAddr))
 		case "https":
-			res.Https = append(res.Https, NewProxyStatusResp(status))
+			res.Https = append(res.Https, NewProxyStatusResp(status, svr.cfg.ServerAddr))
 		case "stcp":
-			res.Stcp = append(res.Stcp, NewProxyStatusResp(status))
+			res.Stcp = append(res.Stcp, NewProxyStatusResp(status, svr.cfg.ServerAddr))
 		case "xtcp":
-			res.Xtcp = append(res.Xtcp, NewProxyStatusResp(status))
+			res.Xtcp = append(res.Xtcp, NewProxyStatusResp(status, svr.cfg.ServerAddr))
 		}
 	}
 	sort.Sort(ByProxyStatusResp(res.Tcp))
@@ -219,14 +218,14 @@ func (svr *Service) apiGetConfig(w http.ResponseWriter, r *http.Request) {
 		}
 	}()
 
-	if g.GlbClientCfg.CfgFile == "" {
+	if svr.cfgFile == "" {
 		res.Code = 400
 		res.Msg = "frpc has no config file path"
 		log.Warn("%s", res.Msg)
 		return
 	}
 
-	content, err := config.GetRenderedConfFromFile(g.GlbClientCfg.CfgFile)
+	content, err := config.GetRenderedConfFromFile(svr.cfgFile)
 	if err != nil {
 		res.Code = 400
 		res.Msg = err.Error()
@@ -277,7 +276,7 @@ func (svr *Service) apiPutConfig(w http.ResponseWriter, r *http.Request) {
 
 	// get token from origin content
 	token := ""
-	b, err := ioutil.ReadFile(g.GlbClientCfg.CfgFile)
+	b, err := ioutil.ReadFile(svr.cfgFile)
 	if err != nil {
 		res.Code = 400
 		res.Msg = err.Error()
@@ -316,7 +315,7 @@ func (svr *Service) apiPutConfig(w http.ResponseWriter, r *http.Request) {
 	}
 	content = strings.Join(newRows, "\n")
 
-	err = ioutil.WriteFile(g.GlbClientCfg.CfgFile, []byte(content), 0644)
+	err = ioutil.WriteFile(svr.cfgFile, []byte(content), 0644)
 	if err != nil {
 		res.Code = 500
 		res.Msg = fmt.Sprintf("write content to frpc config file error: %v", err)

client/control.go

@@ -15,19 +15,20 @@
 package client
 
 import (
+	"context"
 	"crypto/tls"
 	"fmt"
 	"io"
+	"net"
 	"runtime/debug"
 	"sync"
 	"time"
 
 	"github.com/fatedier/frp/client/proxy"
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/models/msg"
-	"github.com/fatedier/frp/utils/log"
 	frpNet "github.com/fatedier/frp/utils/net"
+	"github.com/fatedier/frp/utils/xlog"
 
 	"github.com/fatedier/golib/control/shutdown"
 	"github.com/fatedier/golib/crypto"
@@ -46,7 +47,7 @@ type Control struct {
 	vm *VisitorManager
 
 	// control connection
-	conn frpNet.Conn
+	conn net.Conn
 
 	// tcp stream multiplexing, if enabled
 	session *fmux.Session
@@ -65,16 +66,31 @@ type Control struct {
 	// last time got the Pong message
 	lastPong time.Time
 
+	// The client configuration
+	clientCfg config.ClientCommonConf
+
 	readerShutdown     *shutdown.Shutdown
 	writerShutdown     *shutdown.Shutdown
 	msgHandlerShutdown *shutdown.Shutdown
 
+	// The UDP port that the server is listening on
+	serverUDPPort int
+
 	mu sync.RWMutex
 
-	log.Logger
+	xl *xlog.Logger
+
+	// service context
+	ctx context.Context
 }
 
-func NewControl(runId string, conn frpNet.Conn, session *fmux.Session, pxyCfgs map[string]config.ProxyConf, visitorCfgs map[string]config.VisitorConf) *Control {
+func NewControl(ctx context.Context, runId string, conn net.Conn, session *fmux.Session,
+	clientCfg config.ClientCommonConf,
+	pxyCfgs map[string]config.ProxyConf,
+	visitorCfgs map[string]config.VisitorConf,
+	serverUDPPort int) *Control {
+
+	// new xlog instance
 	ctl := &Control{
 		runId:              runId,
 		conn:               conn,
@@ -84,14 +100,17 @@ func NewControl(runId string, conn frpNet.Conn, session *fmux.Session, pxyCfgs m
 		readCh:             make(chan msg.Message, 100),
 		closedCh:           make(chan struct{}),
 		closedDoneCh:       make(chan struct{}),
+		clientCfg:          clientCfg,
 		readerShutdown:     shutdown.New(),
 		writerShutdown:     shutdown.New(),
 		msgHandlerShutdown: shutdown.New(),
-		Logger:             log.NewPrefixLogger(""),
+		serverUDPPort:      serverUDPPort,
+		xl:                 xlog.FromContextSafe(ctx),
+		ctx:                ctx,
 	}
-	ctl.pm = proxy.NewProxyManager(ctl.sendCh, runId)
+	ctl.pm = proxy.NewProxyManager(ctl.ctx, ctl.sendCh, clientCfg, serverUDPPort)
 
-	ctl.vm = NewVisitorManager(ctl)
+	ctl.vm = NewVisitorManager(ctl.ctx, ctl)
 	ctl.vm.Reload(visitorCfgs)
 	return ctl
 }
@@ -108,6 +127,7 @@ func (ctl *Control) Run() {
 }
 
 func (ctl *Control) HandleReqWorkConn(inMsg *msg.ReqWorkConn) {
+	xl := ctl.xl
 	workConn, err := ctl.connectServer()
 	if err != nil {
 		return
@@ -117,31 +137,31 @@ func (ctl *Control) HandleReqWorkConn(inMsg *msg.ReqWorkConn) {
 		RunId: ctl.runId,
 	}
 	if err = msg.WriteMsg(workConn, m); err != nil {
-		ctl.Warn("work connection write to server error: %v", err)
+		xl.Warn("work connection write to server error: %v", err)
 		workConn.Close()
 		return
 	}
 
 	var startMsg msg.StartWorkConn
 	if err = msg.ReadMsgInto(workConn, &startMsg); err != nil {
-		ctl.Error("work connection closed, %v", err)
+		xl.Error("work connection closed before response StartWorkConn message: %v", err)
 		workConn.Close()
 		return
 	}
-	workConn.AddLogPrefix(startMsg.ProxyName)
 
 	// dispatch this work connection to related proxy
 	ctl.pm.HandleWorkConn(startMsg.ProxyName, workConn, &startMsg)
 }
 
 func (ctl *Control) HandleNewProxyResp(inMsg *msg.NewProxyResp) {
+	xl := ctl.xl
 	// Server will return NewProxyResp message to each NewProxy message.
 	// Start a new proxy handler if no error got
 	err := ctl.pm.StartProxy(inMsg.ProxyName, inMsg.RemoteAddr, inMsg.Error)
 	if err != nil {
-		ctl.Warn("[%s] start error: %v", inMsg.ProxyName, err)
+		xl.Warn("[%s] start error: %v", inMsg.ProxyName, err)
 	} else {
-		ctl.Info("[%s] start proxy success", inMsg.ProxyName)
+		xl.Info("[%s] start proxy success", inMsg.ProxyName)
 	}
 }
 
@@ -160,26 +180,27 @@ func (ctl *Control) ClosedDoneCh() <-chan struct{} {
 }
 
 // connectServer return a new connection to frps
-func (ctl *Control) connectServer() (conn frpNet.Conn, err error) {
-	if g.GlbClientCfg.TcpMux {
+func (ctl *Control) connectServer() (conn net.Conn, err error) {
+	xl := ctl.xl
+	if ctl.clientCfg.TcpMux {
 		stream, errRet := ctl.session.OpenStream()
 		if errRet != nil {
 			err = errRet
-			ctl.Warn("start new connection to server error: %v", err)
+			xl.Warn("start new connection to server error: %v", err)
 			return
 		}
-		conn = frpNet.WrapConn(stream)
+		conn = stream
 	} else {
 		var tlsConfig *tls.Config
-		if g.GlbClientCfg.TLSEnable {
+		if ctl.clientCfg.TLSEnable {
 			tlsConfig = &tls.Config{
 				InsecureSkipVerify: true,
 			}
 		}
-		conn, err = frpNet.ConnectServerByProxyWithTLS(g.GlbClientCfg.HttpProxy, g.GlbClientCfg.Protocol,
-			fmt.Sprintf("%s:%d", g.GlbClientCfg.ServerAddr, g.GlbClientCfg.ServerPort), tlsConfig)
+		conn, err = frpNet.ConnectServerByProxyWithTLS(ctl.clientCfg.HttpProxy, ctl.clientCfg.Protocol,
+			fmt.Sprintf("%s:%d", ctl.clientCfg.ServerAddr, ctl.clientCfg.ServerPort), tlsConfig)
 		if err != nil {
-			ctl.Warn("start new connection to server error: %v", err)
+			xl.Warn("start new connection to server error: %v", err)
 			return
 		}
 	}
@@ -188,23 +209,24 @@ func (ctl *Control) connectServer() (conn frpNet.Conn, err error) {
 
 // reader read all messages from frps and send to readCh
 func (ctl *Control) reader() {
+	xl := ctl.xl
 	defer func() {
 		if err := recover(); err != nil {
-			ctl.Error("panic error: %v", err)
-			ctl.Error(string(debug.Stack()))
+			xl.Error("panic error: %v", err)
+			xl.Error(string(debug.Stack()))
 		}
 	}()
 	defer ctl.readerShutdown.Done()
 	defer close(ctl.closedCh)
 
-	encReader := crypto.NewReader(ctl.conn, []byte(g.GlbClientCfg.Token))
+	encReader := crypto.NewReader(ctl.conn, []byte(ctl.clientCfg.Token))
 	for {
 		if m, err := msg.ReadMsg(encReader); err != nil {
 			if err == io.EOF {
-				ctl.Debug("read from control connection EOF")
+				xl.Debug("read from control connection EOF")
 				return
 			} else {
-				ctl.Warn("read error: %v", err)
+				xl.Warn("read error: %v", err)
 				ctl.conn.Close()
 				return
 			}
@@ -216,20 +238,21 @@ func (ctl *Control) reader() {
 
 // writer writes messages got from sendCh to frps
 func (ctl *Control) writer() {
+	xl := ctl.xl
 	defer ctl.writerShutdown.Done()
-	encWriter, err := crypto.NewWriter(ctl.conn, []byte(g.GlbClientCfg.Token))
+	encWriter, err := crypto.NewWriter(ctl.conn, []byte(ctl.clientCfg.Token))
 	if err != nil {
-		ctl.conn.Error("crypto new writer error: %v", err)
+		xl.Error("crypto new writer error: %v", err)
 		ctl.conn.Close()
 		return
 	}
 	for {
 		if m, ok := <-ctl.sendCh; !ok {
-			ctl.Info("control writer is closing")
+			xl.Info("control writer is closing")
 			return
 		} else {
 			if err := msg.WriteMsg(encWriter, m); err != nil {
-				ctl.Warn("write message to control connection error: %v", err)
+				xl.Warn("write message to control connection error: %v", err)
 				return
 			}
 		}
@@ -238,15 +261,16 @@ func (ctl *Control) writer() {
 
 // msgHandler handles all channel events and do corresponding operations.
 func (ctl *Control) msgHandler() {
+	xl := ctl.xl
 	defer func() {
 		if err := recover(); err != nil {
-			ctl.Error("panic error: %v", err)
-			ctl.Error(string(debug.Stack()))
+			xl.Error("panic error: %v", err)
+			xl.Error(string(debug.Stack()))
 		}
 	}()
 	defer ctl.msgHandlerShutdown.Done()
 
-	hbSend := time.NewTicker(time.Duration(g.GlbClientCfg.HeartBeatInterval) * time.Second)
+	hbSend := time.NewTicker(time.Duration(ctl.clientCfg.HeartBeatInterval) * time.Second)
 	defer hbSend.Stop()
 	hbCheck := time.NewTicker(time.Second)
 	defer hbCheck.Stop()
@@ -257,11 +281,11 @@ func (ctl *Control) msgHandler() {
 		select {
 		case <-hbSend.C:
 			// send heartbeat to server
-			ctl.Debug("send heartbeat to server")
+			xl.Debug("send heartbeat to server")
 			ctl.sendCh <- &msg.Ping{}
 		case <-hbCheck.C:
-			if time.Since(ctl.lastPong) > time.Duration(g.GlbClientCfg.HeartBeatTimeout)*time.Second {
-				ctl.Warn("heartbeat timeout")
+			if time.Since(ctl.lastPong) > time.Duration(ctl.clientCfg.HeartBeatTimeout)*time.Second {
+				xl.Warn("heartbeat timeout")
 				// let reader() stop
 				ctl.conn.Close()
 				return
@@ -278,7 +302,7 @@ func (ctl *Control) msgHandler() {
 				ctl.HandleNewProxyResp(m)
 			case *msg.Pong:
 				ctl.lastPong = time.Now()
-				ctl.Debug("receive heartbeat from server")
+				xl.Debug("receive heartbeat from server")
 			}
 		}
 	}

client/health/health.go

@@ -24,7 +24,7 @@ import (
 	"net/http"
 	"time"
 
-	"github.com/fatedier/frp/utils/log"
+	"github.com/fatedier/frp/utils/xlog"
 )
 
 var (
@@ -50,11 +50,11 @@ type HealthCheckMonitor struct {
 
 	ctx    context.Context
 	cancel context.CancelFunc
-
-	l log.Logger
 }
 
-func NewHealthCheckMonitor(checkType string, intervalS int, timeoutS int, maxFailedTimes int, addr string, url string,
+func NewHealthCheckMonitor(ctx context.Context, checkType string,
+	intervalS int, timeoutS int, maxFailedTimes int,
+	addr string, url string,
 	statusNormalFn func(), statusFailedFn func()) *HealthCheckMonitor {
 
 	if intervalS <= 0 {
@@ -66,7 +66,7 @@ func NewHealthCheckMonitor(checkType string, intervalS int, timeoutS int, maxFai
 	if maxFailedTimes <= 0 {
 		maxFailedTimes = 1
 	}
-	ctx, cancel := context.WithCancel(context.Background())
+	newctx, cancel := context.WithCancel(ctx)
 	return &HealthCheckMonitor{
 		checkType:      checkType,
 		interval:       time.Duration(intervalS) * time.Second,
@@ -77,15 +77,11 @@ func NewHealthCheckMonitor(checkType string, intervalS int, timeoutS int, maxFai
 		statusOK:       false,
 		statusNormalFn: statusNormalFn,
 		statusFailedFn: statusFailedFn,
-		ctx:            ctx,
+		ctx:            newctx,
 		cancel:         cancel,
 	}
 }
 
-func (monitor *HealthCheckMonitor) SetLogger(l log.Logger) {
-	monitor.l = l
-}
-
 func (monitor *HealthCheckMonitor) Start() {
 	go monitor.checkWorker()
 }
@@ -95,13 +91,14 @@ func (monitor *HealthCheckMonitor) Stop() {
 }
 
 func (monitor *HealthCheckMonitor) checkWorker() {
+	xl := xlog.FromContextSafe(monitor.ctx)
 	for {
-		ctx, cancel := context.WithDeadline(monitor.ctx, time.Now().Add(monitor.timeout))
-		err := monitor.doCheck(ctx)
+		doCtx, cancel := context.WithDeadline(monitor.ctx, time.Now().Add(monitor.timeout))
+		err := monitor.doCheck(doCtx)
 
 		// check if this monitor has been closed
 		select {
-		case <-ctx.Done():
+		case <-monitor.ctx.Done():
 			cancel()
 			return
 		default:
@@ -109,25 +106,17 @@ func (monitor *HealthCheckMonitor) checkWorker() {
 		}
 
 		if err == nil {
-			if monitor.l != nil {
-				monitor.l.Trace("do one health check success")
-			}
+			xl.Trace("do one health check success")
 			if !monitor.statusOK && monitor.statusNormalFn != nil {
-				if monitor.l != nil {
-					monitor.l.Info("health check status change to success")
-				}
+				xl.Info("health check status change to success")
 				monitor.statusOK = true
 				monitor.statusNormalFn()
 			}
 		} else {
-			if monitor.l != nil {
-				monitor.l.Warn("do one health check failed: %v", err)
-			}
+			xl.Warn("do one health check failed: %v", err)
 			monitor.failedTimes++
 			if monitor.statusOK && int(monitor.failedTimes) >= monitor.maxFailedTimes && monitor.statusFailedFn != nil {
-				if monitor.l != nil {
-					monitor.l.Warn("health check status change to failed")
-				}
+				xl.Warn("health check status change to failed")
 				monitor.statusOK = false
 				monitor.statusFailedFn()
 			}

client/proxy/proxy.go

@@ -16,6 +16,7 @@ package proxy
 
 import (
 	"bytes"
+	"context"
 	"fmt"
 	"io"
 	"io/ioutil"
@@ -25,19 +26,20 @@ import (
 	"sync"
 	"time"
 
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/models/msg"
 	"github.com/fatedier/frp/models/plugin"
 	"github.com/fatedier/frp/models/proto/udp"
-	"github.com/fatedier/frp/utils/log"
+	"github.com/fatedier/frp/utils/limit"
 	frpNet "github.com/fatedier/frp/utils/net"
+	"github.com/fatedier/frp/utils/xlog"
 
 	"github.com/fatedier/golib/errors"
 	frpIo "github.com/fatedier/golib/io"
 	"github.com/fatedier/golib/pool"
 	fmux "github.com/hashicorp/yamux"
 	pp "github.com/pires/go-proxyproto"
+	"golang.org/x/time/rate"
 )
 
 // Proxy defines how to handle work connections for different proxy type.
@@ -45,15 +47,24 @@ type Proxy interface {
 	Run() error
 
 	// InWorkConn accept work connections registered to server.
-	InWorkConn(frpNet.Conn, *msg.StartWorkConn)
+	InWorkConn(net.Conn, *msg.StartWorkConn)
 
 	Close()
-	log.Logger
 }
 
-func NewProxy(pxyConf config.ProxyConf) (pxy Proxy) {
+func NewProxy(ctx context.Context, pxyConf config.ProxyConf, clientCfg config.ClientCommonConf, serverUDPPort int) (pxy Proxy) {
+	var limiter *rate.Limiter
+	limitBytes := pxyConf.GetBaseInfo().BandwidthLimit.Bytes()
+	if limitBytes > 0 {
+		limiter = rate.NewLimiter(rate.Limit(float64(limitBytes)), int(limitBytes))
+	}
+
 	baseProxy := BaseProxy{
-		Logger: log.NewPrefixLogger(pxyConf.GetBaseInfo().ProxyName),
+		clientCfg:     clientCfg,
+		serverUDPPort: serverUDPPort,
+		limiter:       limiter,
+		xl:            xlog.FromContextSafe(ctx),
+		ctx:           ctx,
 	}
 	switch cfg := pxyConf.(type) {
 	case *config.TcpProxyConf:
@@ -91,9 +102,14 @@ func NewProxy(pxyConf config.ProxyConf) (pxy Proxy) {
 }
 
 type BaseProxy struct {
-	closed bool
-	mu     sync.RWMutex
-	log.Logger
+	closed        bool
+	clientCfg     config.ClientCommonConf
+	serverUDPPort int
+	limiter       *rate.Limiter
+
+	mu  sync.RWMutex
+	xl  *xlog.Logger
+	ctx context.Context
 }
 
 // TCP
@@ -120,9 +136,9 @@ func (pxy *TcpProxy) Close() {
 	}
 }
 
-func (pxy *TcpProxy) InWorkConn(conn frpNet.Conn, m *msg.StartWorkConn) {
-	HandleTcpWorkConnection(&pxy.cfg.LocalSvrConf, pxy.proxyPlugin, &pxy.cfg.BaseProxyConf, conn,
-		[]byte(g.GlbClientCfg.Token), m)
+func (pxy *TcpProxy) InWorkConn(conn net.Conn, m *msg.StartWorkConn) {
+	HandleTcpWorkConnection(pxy.ctx, &pxy.cfg.LocalSvrConf, pxy.proxyPlugin, &pxy.cfg.BaseProxyConf, pxy.limiter,
+		conn, []byte(pxy.clientCfg.Token), m)
 }
 
 // HTTP
@@ -149,9 +165,9 @@ func (pxy *HttpProxy) Close() {
 	}
 }
 
-func (pxy *HttpProxy) InWorkConn(conn frpNet.Conn, m *msg.StartWorkConn) {
-	HandleTcpWorkConnection(&pxy.cfg.LocalSvrConf, pxy.proxyPlugin, &pxy.cfg.BaseProxyConf, conn,
-		[]byte(g.GlbClientCfg.Token), m)
+func (pxy *HttpProxy) InWorkConn(conn net.Conn, m *msg.StartWorkConn) {
+	HandleTcpWorkConnection(pxy.ctx, &pxy.cfg.LocalSvrConf, pxy.proxyPlugin, &pxy.cfg.BaseProxyConf, pxy.limiter,
+		conn, []byte(pxy.clientCfg.Token), m)
 }
 
 // HTTPS
@@ -178,9 +194,9 @@ func (pxy *HttpsProxy) Close() {
 	}
 }
 
-func (pxy *HttpsProxy) InWorkConn(conn frpNet.Conn, m *msg.StartWorkConn) {
-	HandleTcpWorkConnection(&pxy.cfg.LocalSvrConf, pxy.proxyPlugin, &pxy.cfg.BaseProxyConf, conn,
-		[]byte(g.GlbClientCfg.Token), m)
+func (pxy *HttpsProxy) InWorkConn(conn net.Conn, m *msg.StartWorkConn) {
+	HandleTcpWorkConnection(pxy.ctx, &pxy.cfg.LocalSvrConf, pxy.proxyPlugin, &pxy.cfg.BaseProxyConf, pxy.limiter,
+		conn, []byte(pxy.clientCfg.Token), m)
 }
 
 // STCP
@@ -207,9 +223,9 @@ func (pxy *StcpProxy) Close() {
 	}
 }
 
-func (pxy *StcpProxy) InWorkConn(conn frpNet.Conn, m *msg.StartWorkConn) {
-	HandleTcpWorkConnection(&pxy.cfg.LocalSvrConf, pxy.proxyPlugin, &pxy.cfg.BaseProxyConf, conn,
-		[]byte(g.GlbClientCfg.Token), m)
+func (pxy *StcpProxy) InWorkConn(conn net.Conn, m *msg.StartWorkConn) {
+	HandleTcpWorkConnection(pxy.ctx, &pxy.cfg.LocalSvrConf, pxy.proxyPlugin, &pxy.cfg.BaseProxyConf, pxy.limiter,
+		conn, []byte(pxy.clientCfg.Token), m)
 }
 
 // XTCP
@@ -236,12 +252,13 @@ func (pxy *XtcpProxy) Close() {
 	}
 }
 
-func (pxy *XtcpProxy) InWorkConn(conn frpNet.Conn, m *msg.StartWorkConn) {
+func (pxy *XtcpProxy) InWorkConn(conn net.Conn, m *msg.StartWorkConn) {
+	xl := pxy.xl
 	defer conn.Close()
 	var natHoleSidMsg msg.NatHoleSid
 	err := msg.ReadMsgInto(conn, &natHoleSidMsg)
 	if err != nil {
-		pxy.Error("xtcp read from workConn error: %v", err)
+		xl.Error("xtcp read from workConn error: %v", err)
 		return
 	}
 
@@ -250,13 +267,13 @@ func (pxy *XtcpProxy) InWorkConn(conn frpNet.Conn, m *msg.StartWorkConn) {
 		Sid:       natHoleSidMsg.Sid,
 	}
 	raddr, _ := net.ResolveUDPAddr("udp",
-		fmt.Sprintf("%s:%d", g.GlbClientCfg.ServerAddr, g.GlbClientCfg.ServerUdpPort))
+		fmt.Sprintf("%s:%d", pxy.clientCfg.ServerAddr, pxy.serverUDPPort))
 	clientConn, err := net.DialUDP("udp", nil, raddr)
 	defer clientConn.Close()
 
 	err = msg.WriteMsg(clientConn, natHoleClientMsg)
 	if err != nil {
-		pxy.Error("send natHoleClientMsg to server error: %v", err)
+		xl.Error("send natHoleClientMsg to server error: %v", err)
 		return
 	}
 
@@ -267,28 +284,28 @@ func (pxy *XtcpProxy) InWorkConn(conn frpNet.Conn, m *msg.StartWorkConn) {
 	buf := pool.GetBuf(1024)
 	n, err := clientConn.Read(buf)
 	if err != nil {
-		pxy.Error("get natHoleRespMsg error: %v", err)
+		xl.Error("get natHoleRespMsg error: %v", err)
 		return
 	}
 	err = msg.ReadMsgInto(bytes.NewReader(buf[:n]), &natHoleRespMsg)
 	if err != nil {
-		pxy.Error("get natHoleRespMsg error: %v", err)
+		xl.Error("get natHoleRespMsg error: %v", err)
 		return
 	}
 	clientConn.SetReadDeadline(time.Time{})
 	clientConn.Close()
 
 	if natHoleRespMsg.Error != "" {
-		pxy.Error("natHoleRespMsg get error info: %s", natHoleRespMsg.Error)
+		xl.Error("natHoleRespMsg get error info: %s", natHoleRespMsg.Error)
 		return
 	}
 
-	pxy.Trace("get natHoleRespMsg, sid [%s], client address [%s] visitor address [%s]", natHoleRespMsg.Sid, natHoleRespMsg.ClientAddr, natHoleRespMsg.VisitorAddr)
+	xl.Trace("get natHoleRespMsg, sid [%s], client address [%s] visitor address [%s]", natHoleRespMsg.Sid, natHoleRespMsg.ClientAddr, natHoleRespMsg.VisitorAddr)
 
 	// Send detect message
 	array := strings.Split(natHoleRespMsg.VisitorAddr, ":")
 	if len(array) <= 1 {
-		pxy.Error("get NatHoleResp visitor address error: %v", natHoleRespMsg.VisitorAddr)
+		xl.Error("get NatHoleResp visitor address error: %v", natHoleRespMsg.VisitorAddr)
 	}
 	laddr, _ := net.ResolveUDPAddr("udp", clientConn.LocalAddr().String())
 	/*
@@ -298,18 +315,18 @@ func (pxy *XtcpProxy) InWorkConn(conn frpNet.Conn, m *msg.StartWorkConn) {
 	*/
 	port, err := strconv.ParseInt(array[1], 10, 64)
 	if err != nil {
-		pxy.Error("get natHoleResp visitor address error: %v", natHoleRespMsg.VisitorAddr)
+		xl.Error("get natHoleResp visitor address error: %v", natHoleRespMsg.VisitorAddr)
 		return
 	}
 	pxy.sendDetectMsg(array[0], int(port), laddr, []byte(natHoleRespMsg.Sid))
-	pxy.Trace("send all detect msg done")
+	xl.Trace("send all detect msg done")
 
 	msg.WriteMsg(conn, &msg.NatHoleClientDetectOK{})
 
 	// Listen for clientConn's address and wait for visitor connection
 	lConn, err := net.ListenUDP("udp", laddr)
 	if err != nil {
-		pxy.Error("listen on visitorConn's local adress error: %v", err)
+		xl.Error("listen on visitorConn's local adress error: %v", err)
 		return
 	}
 	defer lConn.Close()
@@ -319,22 +336,22 @@ func (pxy *XtcpProxy) InWorkConn(conn frpNet.Conn, m *msg.StartWorkConn) {
 	var uAddr *net.UDPAddr
 	n, uAddr, err = lConn.ReadFromUDP(sidBuf)
 	if err != nil {
-		pxy.Warn("get sid from visitor error: %v", err)
+		xl.Warn("get sid from visitor error: %v", err)
 		return
 	}
 	lConn.SetReadDeadline(time.Time{})
 	if string(sidBuf[:n]) != natHoleRespMsg.Sid {
-		pxy.Warn("incorrect sid from visitor")
+		xl.Warn("incorrect sid from visitor")
 		return
 	}
 	pool.PutBuf(sidBuf)
-	pxy.Info("nat hole connection make success, sid [%s]", natHoleRespMsg.Sid)
+	xl.Info("nat hole connection make success, sid [%s]", natHoleRespMsg.Sid)
 
 	lConn.WriteToUDP(sidBuf[:n], uAddr)
 
 	kcpConn, err := frpNet.NewKcpConnFromUdp(lConn, false, natHoleRespMsg.VisitorAddr)
 	if err != nil {
-		pxy.Error("create kcp connection from udp connection error: %v", err)
+		xl.Error("create kcp connection from udp connection error: %v", err)
 		return
 	}
 
@@ -343,18 +360,18 @@ func (pxy *XtcpProxy) InWorkConn(conn frpNet.Conn, m *msg.StartWorkConn) {
 	fmuxCfg.LogOutput = ioutil.Discard
 	sess, err := fmux.Server(kcpConn, fmuxCfg)
 	if err != nil {
-		pxy.Error("create yamux server from kcp connection error: %v", err)
+		xl.Error("create yamux server from kcp connection error: %v", err)
 		return
 	}
 	defer sess.Close()
 	muxConn, err := sess.Accept()
 	if err != nil {
-		pxy.Error("accept for yamux connection error: %v", err)
+		xl.Error("accept for yamux connection error: %v", err)
 		return
 	}
 
-	HandleTcpWorkConnection(&pxy.cfg.LocalSvrConf, pxy.proxyPlugin, &pxy.cfg.BaseProxyConf,
-		frpNet.WrapConn(muxConn), []byte(pxy.cfg.Sk), m)
+	HandleTcpWorkConnection(pxy.ctx, &pxy.cfg.LocalSvrConf, pxy.proxyPlugin, &pxy.cfg.BaseProxyConf, pxy.limiter,
+		muxConn, []byte(pxy.cfg.Sk), m)
 }
 
 func (pxy *XtcpProxy) sendDetectMsg(addr string, port int, laddr *net.UDPAddr, content []byte) (err error) {
@@ -387,7 +404,7 @@ type UdpProxy struct {
 
 	// include msg.UdpPacket and msg.Ping
 	sendCh   chan msg.Message
-	workConn frpNet.Conn
+	workConn net.Conn
 }
 
 func (pxy *UdpProxy) Run() (err error) {
@@ -416,11 +433,19 @@ func (pxy *UdpProxy) Close() {
 	}
 }
 
-func (pxy *UdpProxy) InWorkConn(conn frpNet.Conn, m *msg.StartWorkConn) {
-	pxy.Info("incoming a new work connection for udp proxy, %s", conn.RemoteAddr().String())
+func (pxy *UdpProxy) InWorkConn(conn net.Conn, m *msg.StartWorkConn) {
+	xl := pxy.xl
+	xl.Info("incoming a new work connection for udp proxy, %s", conn.RemoteAddr().String())
 	// close resources releated with old workConn
 	pxy.Close()
 
+	if pxy.limiter != nil {
+		rwc := frpIo.WrapReadWriteCloser(limit.NewReader(conn, pxy.limiter), limit.NewWriter(conn, pxy.limiter), func() error {
+			return conn.Close()
+		})
+		conn = frpNet.WrapReadWriteCloserToConn(rwc, conn)
+	}
+
 	pxy.mu.Lock()
 	pxy.workConn = conn
 	pxy.readCh = make(chan *msg.UdpPacket, 1024)
@@ -432,32 +457,32 @@ func (pxy *UdpProxy) InWorkConn(conn frpNet.Conn, m *msg.StartWorkConn) {
 		for {
 			var udpMsg msg.UdpPacket
 			if errRet := msg.ReadMsgInto(conn, &udpMsg); errRet != nil {
-				pxy.Warn("read from workConn for udp error: %v", errRet)
+				xl.Warn("read from workConn for udp error: %v", errRet)
 				return
 			}
 			if errRet := errors.PanicToError(func() {
-				pxy.Trace("get udp package from workConn: %s", udpMsg.Content)
+				xl.Trace("get udp package from workConn: %s", udpMsg.Content)
 				readCh <- &udpMsg
 			}); errRet != nil {
-				pxy.Info("reader goroutine for udp work connection closed: %v", errRet)
+				xl.Info("reader goroutine for udp work connection closed: %v", errRet)
 				return
 			}
 		}
 	}
 	workConnSenderFn := func(conn net.Conn, sendCh chan msg.Message) {
 		defer func() {
-			pxy.Info("writer goroutine for udp work connection closed")
+			xl.Info("writer goroutine for udp work connection closed")
 		}()
 		var errRet error
 		for rawMsg := range sendCh {
 			switch m := rawMsg.(type) {
 			case *msg.UdpPacket:
-				pxy.Trace("send udp package to workConn: %s", m.Content)
+				xl.Trace("send udp package to workConn: %s", m.Content)
 			case *msg.Ping:
-				pxy.Trace("send ping message to udp workConn")
+				xl.Trace("send ping message to udp workConn")
 			}
 			if errRet = msg.WriteMsg(conn, rawMsg); errRet != nil {
-				pxy.Error("udp work write error: %v", errRet)
+				xl.Error("udp work write error: %v", errRet)
 				return
 			}
 		}
@@ -469,7 +494,7 @@ func (pxy *UdpProxy) InWorkConn(conn frpNet.Conn, m *msg.StartWorkConn) {
 			if errRet = errors.PanicToError(func() {
 				sendCh <- &msg.Ping{}
 			}); errRet != nil {
-				pxy.Trace("heartbeat goroutine for udp work connection closed")
+				xl.Trace("heartbeat goroutine for udp work connection closed")
 				break
 			}
 		}
@@ -482,20 +507,27 @@ func (pxy *UdpProxy) InWorkConn(conn frpNet.Conn, m *msg.StartWorkConn) {
 }
 
 // Common handler for tcp work connections.
-func HandleTcpWorkConnection(localInfo *config.LocalSvrConf, proxyPlugin plugin.Plugin,
-	baseInfo *config.BaseProxyConf, workConn frpNet.Conn, encKey []byte, m *msg.StartWorkConn) {
-
+func HandleTcpWorkConnection(ctx context.Context, localInfo *config.LocalSvrConf, proxyPlugin plugin.Plugin,
+	baseInfo *config.BaseProxyConf, limiter *rate.Limiter, workConn net.Conn, encKey []byte, m *msg.StartWorkConn) {
+	xl := xlog.FromContextSafe(ctx)
 	var (
 		remote io.ReadWriteCloser
 		err    error
 	)
 	remote = workConn
+	if limiter != nil {
+		remote = frpIo.WrapReadWriteCloser(limit.NewReader(workConn, limiter), limit.NewWriter(workConn, limiter), func() error {
+			return workConn.Close()
+		})
+	}
 
+	xl.Trace("handle tcp work connection, use_encryption: %t, use_compression: %t",
+		baseInfo.UseEncryption, baseInfo.UseCompression)
 	if baseInfo.UseEncryption {
 		remote, err = frpIo.WithEncryption(remote, encKey)
 		if err != nil {
 			workConn.Close()
-			workConn.Error("create encryption stream error: %v", err)
+			xl.Error("create encryption stream error: %v", err)
 			return
 		}
 	}
@@ -518,7 +550,7 @@ func HandleTcpWorkConnection(localInfo *config.LocalSvrConf, proxyPlugin plugin.
 				DestinationPort:    m.DstPort,
 			}
 
-			if h.SourceAddress.To16() == nil {
+			if strings.Contains(m.SrcAddr, ".") {
 				h.TransportProtocol = pp.TCPv4
 			} else {
 				h.TransportProtocol = pp.TCPv6
@@ -538,19 +570,19 @@ func HandleTcpWorkConnection(localInfo *config.LocalSvrConf, proxyPlugin plugin.
 
 	if proxyPlugin != nil {
 		// if plugin is set, let plugin handle connections first
-		workConn.Debug("handle by plugin: %s", proxyPlugin.Name())
+		xl.Debug("handle by plugin: %s", proxyPlugin.Name())
 		proxyPlugin.Handle(remote, workConn, extraInfo)
-		workConn.Debug("handle by plugin finished")
+		xl.Debug("handle by plugin finished")
 		return
 	} else {
 		localConn, err := frpNet.ConnectServer("tcp", fmt.Sprintf("%s:%d", localInfo.LocalIp, localInfo.LocalPort))
 		if err != nil {
 			workConn.Close()
-			workConn.Error("connect to local service [%s:%d] error: %v", localInfo.LocalIp, localInfo.LocalPort, err)
+			xl.Error("connect to local service [%s:%d] error: %v", localInfo.LocalIp, localInfo.LocalPort, err)
 			return
 		}
 
-		workConn.Debug("join connections, localConn(l[%s] r[%s]) workConn(l[%s] r[%s])", localConn.LocalAddr().String(),
+		xl.Debug("join connections, localConn(l[%s] r[%s]) workConn(l[%s] r[%s])", localConn.LocalAddr().String(),
 			localConn.RemoteAddr().String(), workConn.LocalAddr().String(), workConn.RemoteAddr().String())
 
 		if len(extraInfo) > 0 {
@@ -558,6 +590,6 @@ func HandleTcpWorkConnection(localInfo *config.LocalSvrConf, proxyPlugin plugin.
 		}
 
 		frpIo.Join(localConn, remote)
-		workConn.Debug("join connections closed")
+		xl.Debug("join connections closed")
 	}
 }

client/proxy/proxy_manager.go

@@ -1,14 +1,15 @@
 package proxy
 
 import (
+	"context"
 	"fmt"
+	"net"
 	"sync"
 
 	"github.com/fatedier/frp/client/event"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/models/msg"
-	"github.com/fatedier/frp/utils/log"
-	frpNet "github.com/fatedier/frp/utils/net"
+	"github.com/fatedier/frp/utils/xlog"
 
 	"github.com/fatedier/golib/errors"
 )
@@ -20,17 +21,22 @@ type ProxyManager struct {
 	closed bool
 	mu     sync.RWMutex
 
-	logPrefix string
-	log.Logger
+	clientCfg config.ClientCommonConf
+
+	// The UDP port that the server is listening on
+	serverUDPPort int
+
+	ctx context.Context
 }
 
-func NewProxyManager(msgSendCh chan (msg.Message), logPrefix string) *ProxyManager {
+func NewProxyManager(ctx context.Context, msgSendCh chan (msg.Message), clientCfg config.ClientCommonConf, serverUDPPort int) *ProxyManager {
 	return &ProxyManager{
-		proxies:   make(map[string]*ProxyWrapper),
-		sendCh:    msgSendCh,
-		closed:    false,
-		logPrefix: logPrefix,
-		Logger:    log.NewPrefixLogger(logPrefix),
+		sendCh:        msgSendCh,
+		proxies:       make(map[string]*ProxyWrapper),
+		closed:        false,
+		clientCfg:     clientCfg,
+		serverUDPPort: serverUDPPort,
+		ctx:           ctx,
 	}
 }
 
@@ -58,7 +64,7 @@ func (pm *ProxyManager) Close() {
 	pm.proxies = make(map[string]*ProxyWrapper)
 }
 
-func (pm *ProxyManager) HandleWorkConn(name string, workConn frpNet.Conn, m *msg.StartWorkConn) {
+func (pm *ProxyManager) HandleWorkConn(name string, workConn net.Conn, m *msg.StartWorkConn) {
 	pm.mu.RLock()
 	pw, ok := pm.proxies[name]
 	pm.mu.RUnlock()
@@ -97,6 +103,7 @@ func (pm *ProxyManager) GetAllProxyStatus() []*ProxyStatus {
 }
 
 func (pm *ProxyManager) Reload(pxyCfgs map[string]config.ProxyConf) {
+	xl := xlog.FromContextSafe(pm.ctx)
 	pm.mu.Lock()
 	defer pm.mu.Unlock()
 
@@ -120,13 +127,13 @@ func (pm *ProxyManager) Reload(pxyCfgs map[string]config.ProxyConf) {
 		}
 	}
 	if len(delPxyNames) > 0 {
-		pm.Info("proxy removed: %v", delPxyNames)
+		xl.Info("proxy removed: %v", delPxyNames)
 	}
 
 	addPxyNames := make([]string, 0)
 	for name, cfg := range pxyCfgs {
 		if _, ok := pm.proxies[name]; !ok {
-			pxy := NewProxyWrapper(cfg, pm.HandleEvent, pm.logPrefix)
+			pxy := NewProxyWrapper(pm.ctx, cfg, pm.clientCfg, pm.HandleEvent, pm.serverUDPPort)
 			pm.proxies[name] = pxy
 			addPxyNames = append(addPxyNames, name)
 
@@ -134,6 +141,6 @@ func (pm *ProxyManager) Reload(pxyCfgs map[string]config.ProxyConf) {
 		}
 	}
 	if len(addPxyNames) > 0 {
-		pm.Info("proxy added: %v", addPxyNames)
+		xl.Info("proxy added: %v", addPxyNames)
 	}
 }

client/proxy/proxy_wrapper.go

@@ -1,7 +1,9 @@
 package proxy
 
 import (
+	"context"
 	"fmt"
+	"net"
 	"sync"
 	"sync/atomic"
 	"time"
@@ -10,8 +12,7 @@ import (
 	"github.com/fatedier/frp/client/health"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/models/msg"
-	"github.com/fatedier/frp/utils/log"
-	frpNet "github.com/fatedier/frp/utils/net"
+	"github.com/fatedier/frp/utils/xlog"
 
 	"github.com/fatedier/golib/errors"
 )
@@ -62,11 +63,13 @@ type ProxyWrapper struct {
 	healthNotifyCh   chan struct{}
 	mu               sync.RWMutex
 
-	log.Logger
+	xl  *xlog.Logger
+	ctx context.Context
 }
 
-func NewProxyWrapper(cfg config.ProxyConf, eventHandler event.EventHandler, logPrefix string) *ProxyWrapper {
+func NewProxyWrapper(ctx context.Context, cfg config.ProxyConf, clientCfg config.ClientCommonConf, eventHandler event.EventHandler, serverUDPPort int) *ProxyWrapper {
 	baseInfo := cfg.GetBaseInfo()
+	xl := xlog.FromContextSafe(ctx).Spawn().AppendPrefix(baseInfo.ProxyName)
 	pw := &ProxyWrapper{
 		ProxyStatus: ProxyStatus{
 			Name:   baseInfo.ProxyName,
@@ -77,20 +80,19 @@ func NewProxyWrapper(cfg config.ProxyConf, eventHandler event.EventHandler, logP
 		closeCh:        make(chan struct{}),
 		healthNotifyCh: make(chan struct{}),
 		handler:        eventHandler,
-		Logger:         log.NewPrefixLogger(logPrefix),
+		xl:             xl,
+		ctx:            xlog.NewContext(ctx, xl),
 	}
-	pw.AddLogPrefix(pw.Name)
 
 	if baseInfo.HealthCheckType != "" {
 		pw.health = 1 // means failed
-		pw.monitor = health.NewHealthCheckMonitor(baseInfo.HealthCheckType, baseInfo.HealthCheckIntervalS,
+		pw.monitor = health.NewHealthCheckMonitor(pw.ctx, baseInfo.HealthCheckType, baseInfo.HealthCheckIntervalS,
 			baseInfo.HealthCheckTimeoutS, baseInfo.HealthCheckMaxFailed, baseInfo.HealthCheckAddr,
 			baseInfo.HealthCheckUrl, pw.statusNormalCallback, pw.statusFailedCallback)
-		pw.monitor.SetLogger(pw.Logger)
-		pw.Trace("enable health check monitor")
+		xl.Trace("enable health check monitor")
 	}
 
-	pw.pxy = NewProxy(pw.Cfg)
+	pw.pxy = NewProxy(pw.ctx, pw.Cfg, clientCfg, serverUDPPort)
 	return pw
 }
 
@@ -147,6 +149,7 @@ func (pw *ProxyWrapper) Stop() {
 }
 
 func (pw *ProxyWrapper) checkWorker() {
+	xl := pw.xl
 	if pw.monitor != nil {
 		// let monitor do check request first
 		time.Sleep(500 * time.Millisecond)
@@ -161,7 +164,7 @@ func (pw *ProxyWrapper) checkWorker() {
 				(pw.Status == ProxyStatusWaitStart && now.After(pw.lastSendStartMsg.Add(waitResponseTimeout))) ||
 				(pw.Status == ProxyStatusStartErr && now.After(pw.lastStartErr.Add(startErrTimeout))) {
 
-				pw.Trace("change status from [%s] to [%s]", pw.Status, ProxyStatusWaitStart)
+				xl.Trace("change status from [%s] to [%s]", pw.Status, ProxyStatusWaitStart)
 				pw.Status = ProxyStatusWaitStart
 
 				var newProxyMsg msg.NewProxy
@@ -180,7 +183,7 @@ func (pw *ProxyWrapper) checkWorker() {
 						ProxyName: pw.Name,
 					},
 				})
-				pw.Trace("change status from [%s] to [%s]", pw.Status, ProxyStatusCheckFailed)
+				xl.Trace("change status from [%s] to [%s]", pw.Status, ProxyStatusCheckFailed)
 				pw.Status = ProxyStatusCheckFailed
 			}
 			pw.mu.Unlock()
@@ -196,6 +199,7 @@ func (pw *ProxyWrapper) checkWorker() {
 }
 
 func (pw *ProxyWrapper) statusNormalCallback() {
+	xl := pw.xl
 	atomic.StoreUint32(&pw.health, 0)
 	errors.PanicToError(func() {
 		select {
@@ -203,10 +207,11 @@ func (pw *ProxyWrapper) statusNormalCallback() {
 		default:
 		}
 	})
-	pw.Info("health check success")
+	xl.Info("health check success")
 }
 
 func (pw *ProxyWrapper) statusFailedCallback() {
+	xl := pw.xl
 	atomic.StoreUint32(&pw.health, 1)
 	errors.PanicToError(func() {
 		select {
@@ -214,15 +219,16 @@ func (pw *ProxyWrapper) statusFailedCallback() {
 		default:
 		}
 	})
-	pw.Info("health check failed")
+	xl.Info("health check failed")
 }
 
-func (pw *ProxyWrapper) InWorkConn(workConn frpNet.Conn, m *msg.StartWorkConn) {
+func (pw *ProxyWrapper) InWorkConn(workConn net.Conn, m *msg.StartWorkConn) {
+	xl := pw.xl
 	pw.mu.RLock()
 	pxy := pw.pxy
 	pw.mu.RUnlock()
 	if pxy != nil {
-		workConn.Debug("start a new work connection, localAddr: %s remoteAddr: %s", workConn.LocalAddr().String(), workConn.RemoteAddr().String())
+		xl.Debug("start a new work connection, localAddr: %s remoteAddr: %s", workConn.LocalAddr().String(), workConn.RemoteAddr().String())
 		go pxy.InWorkConn(workConn, m)
 	} else {
 		workConn.Close()

client/service.go

@@ -15,26 +15,29 @@
 package client
 
 import (
+	"context"
 	"crypto/tls"
 	"fmt"
 	"io/ioutil"
+	"net"
 	"runtime"
 	"sync"
 	"sync/atomic"
 	"time"
 
 	"github.com/fatedier/frp/assets"
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/models/msg"
 	"github.com/fatedier/frp/utils/log"
 	frpNet "github.com/fatedier/frp/utils/net"
 	"github.com/fatedier/frp/utils/util"
 	"github.com/fatedier/frp/utils/version"
+	"github.com/fatedier/frp/utils/xlog"
 
 	fmux "github.com/hashicorp/yamux"
 )
 
+// Service is a client service.
 type Service struct {
 	// uniq id got from frps, attach it in loginMsg
 	runId string
@@ -43,27 +46,37 @@ type Service struct {
 	ctl   *Control
 	ctlMu sync.RWMutex
 
+	cfg         config.ClientCommonConf
 	pxyCfgs     map[string]config.ProxyConf
 	visitorCfgs map[string]config.VisitorConf
 	cfgMu       sync.RWMutex
 
-	exit     uint32 // 0 means not exit
-	closedCh chan int
+	// The configuration file used to initialize this client, or an empty
+	// string if no configuration file was used.
+	cfgFile string
+
+	// This is configured by the login response from frps
+	serverUDPPort int
+
+	exit uint32 // 0 means not exit
+
+	// service context
+	ctx context.Context
+	// call cancel to stop service
+	cancel context.CancelFunc
 }
 
-func NewService(pxyCfgs map[string]config.ProxyConf, visitorCfgs map[string]config.VisitorConf) (svr *Service, err error) {
-	// Init assets
-	err = assets.Load("")
-	if err != nil {
-		err = fmt.Errorf("Load assets error: %v", err)
-		return
-	}
+func NewService(cfg config.ClientCommonConf, pxyCfgs map[string]config.ProxyConf, visitorCfgs map[string]config.VisitorConf, cfgFile string) (svr *Service, err error) {
 
+	ctx, cancel := context.WithCancel(context.Background())
 	svr = &Service{
+		cfg:         cfg,
+		cfgFile:     cfgFile,
 		pxyCfgs:     pxyCfgs,
 		visitorCfgs: visitorCfgs,
 		exit:        0,
-		closedCh:    make(chan int),
+		ctx:         xlog.NewContext(ctx, xlog.New()),
+		cancel:      cancel,
 	}
 	return
 }
@@ -75,22 +88,24 @@ func (svr *Service) GetController() *Control {
 }
 
 func (svr *Service) Run() error {
-	// first login
+	xl := xlog.FromContextSafe(svr.ctx)
+
+	// login to frps
 	for {
 		conn, session, err := svr.login()
 		if err != nil {
-			log.Warn("login to server failed: %v", err)
+			xl.Warn("login to server failed: %v", err)
 
 			// if login_fail_exit is true, just exit this program
 			// otherwise sleep a while and try again to connect to server
-			if g.GlbClientCfg.LoginFailExit {
+			if svr.cfg.LoginFailExit {
 				return err
 			} else {
 				time.Sleep(10 * time.Second)
 			}
 		} else {
 			// login success
-			ctl := NewControl(svr.runId, conn, session, svr.pxyCfgs, svr.visitorCfgs)
+			ctl := NewControl(svr.ctx, svr.runId, conn, session, svr.cfg, svr.pxyCfgs, svr.visitorCfgs, svr.serverUDPPort)
 			ctl.Run()
 			svr.ctlMu.Lock()
 			svr.ctl = ctl
@@ -101,19 +116,25 @@ func (svr *Service) Run() error {
 
 	go svr.keepControllerWorking()
 
-	if g.GlbClientCfg.AdminPort != 0 {
-		err := svr.RunAdminServer(g.GlbClientCfg.AdminAddr, g.GlbClientCfg.AdminPort)
+	if svr.cfg.AdminPort != 0 {
+		// Init admin server assets
+		err := assets.Load(svr.cfg.AssetsDir)
+		if err != nil {
+			return fmt.Errorf("Load assets error: %v", err)
+		}
+
+		err = svr.RunAdminServer(svr.cfg.AdminAddr, svr.cfg.AdminPort)
 		if err != nil {
 			log.Warn("run admin server error: %v", err)
 		}
-		log.Info("admin server listen on %s:%d", g.GlbClientCfg.AdminAddr, g.GlbClientCfg.AdminPort)
+		log.Info("admin server listen on %s:%d", svr.cfg.AdminAddr, svr.cfg.AdminPort)
 	}
-
-	<-svr.closedCh
+	<-svr.ctx.Done()
 	return nil
 }
 
 func (svr *Service) keepControllerWorking() {
+	xl := xlog.FromContextSafe(svr.ctx)
 	maxDelayTime := 20 * time.Second
 	delayTime := time.Second
 
@@ -124,10 +145,10 @@ func (svr *Service) keepControllerWorking() {
 		}
 
 		for {
-			log.Info("try to reconnect to server...")
+			xl.Info("try to reconnect to server...")
 			conn, session, err := svr.login()
 			if err != nil {
-				log.Warn("reconnect to server error: %v", err)
+				xl.Warn("reconnect to server error: %v", err)
 				time.Sleep(delayTime)
 				delayTime = delayTime * 2
 				if delayTime > maxDelayTime {
@@ -138,7 +159,7 @@ func (svr *Service) keepControllerWorking() {
 			// reconnect success, init delayTime
 			delayTime = time.Second
 
-			ctl := NewControl(svr.runId, conn, session, svr.pxyCfgs, svr.visitorCfgs)
+			ctl := NewControl(svr.ctx, svr.runId, conn, session, svr.cfg, svr.pxyCfgs, svr.visitorCfgs, svr.serverUDPPort)
 			ctl.Run()
 			svr.ctlMu.Lock()
 			svr.ctl = ctl
@@ -151,15 +172,16 @@ func (svr *Service) keepControllerWorking() {
 // login creates a connection to frps and registers it self as a client
 // conn: control connection
 // session: if it's not nil, using tcp mux
-func (svr *Service) login() (conn frpNet.Conn, session *fmux.Session, err error) {
+func (svr *Service) login() (conn net.Conn, session *fmux.Session, err error) {
+	xl := xlog.FromContextSafe(svr.ctx)
 	var tlsConfig *tls.Config
-	if g.GlbClientCfg.TLSEnable {
+	if svr.cfg.TLSEnable {
 		tlsConfig = &tls.Config{
 			InsecureSkipVerify: true,
 		}
 	}
-	conn, err = frpNet.ConnectServerByProxyWithTLS(g.GlbClientCfg.HttpProxy, g.GlbClientCfg.Protocol,
-		fmt.Sprintf("%s:%d", g.GlbClientCfg.ServerAddr, g.GlbClientCfg.ServerPort), tlsConfig)
+	conn, err = frpNet.ConnectServerByProxyWithTLS(svr.cfg.HttpProxy, svr.cfg.Protocol,
+		fmt.Sprintf("%s:%d", svr.cfg.ServerAddr, svr.cfg.ServerPort), tlsConfig)
 	if err != nil {
 		return
 	}
@@ -167,10 +189,13 @@ func (svr *Service) login() (conn frpNet.Conn, session *fmux.Session, err error)
 	defer func() {
 		if err != nil {
 			conn.Close()
+			if session != nil {
+				session.Close()
+			}
 		}
 	}()
 
-	if g.GlbClientCfg.TcpMux {
+	if svr.cfg.TcpMux {
 		fmuxCfg := fmux.DefaultConfig()
 		fmuxCfg.KeepAliveInterval = 20 * time.Second
 		fmuxCfg.LogOutput = ioutil.Discard
@@ -184,17 +209,17 @@ func (svr *Service) login() (conn frpNet.Conn, session *fmux.Session, err error)
 			err = errRet
 			return
 		}
-		conn = frpNet.WrapConn(stream)
+		conn = stream
 	}
 
 	now := time.Now().Unix()
 	loginMsg := &msg.Login{
 		Arch:         runtime.GOARCH,
 		Os:           runtime.GOOS,
-		PoolCount:    g.GlbClientCfg.PoolCount,
-		User:         g.GlbClientCfg.User,
+		PoolCount:    svr.cfg.PoolCount,
+		User:         svr.cfg.User,
 		Version:      version.Full(),
-		PrivilegeKey: util.GetAuthKey(g.GlbClientCfg.Token, now),
+		PrivilegeKey: util.GetAuthKey(svr.cfg.Token, now),
 		Timestamp:    now,
 		RunId:        svr.runId,
 	}
@@ -212,13 +237,16 @@ func (svr *Service) login() (conn frpNet.Conn, session *fmux.Session, err error)
 
 	if loginRespMsg.Error != "" {
 		err = fmt.Errorf("%s", loginRespMsg.Error)
-		log.Error("%s", loginRespMsg.Error)
+		xl.Error("%s", loginRespMsg.Error)
 		return
 	}
 
 	svr.runId = loginRespMsg.RunId
-	g.GlbClientCfg.ServerUdpPort = loginRespMsg.ServerUdpPort
-	log.Info("login to server success, get run id [%s], server udp port [%d]", loginRespMsg.RunId, loginRespMsg.ServerUdpPort)
+	xl.ResetPrefixes()
+	xl.AppendPrefix(svr.runId)
+
+	svr.serverUDPPort = loginRespMsg.ServerUdpPort
+	xl.Info("login to server success, get run id [%s], server udp port [%d]", loginRespMsg.RunId, loginRespMsg.ServerUdpPort)
 	return
 }
 
@@ -234,5 +262,5 @@ func (svr *Service) ReloadConf(pxyCfgs map[string]config.ProxyConf, visitorCfgs
 func (svr *Service) Close() {
 	atomic.StoreUint32(&svr.exit, 1)
 	svr.ctl.Close()
-	close(svr.closedCh)
+	svr.cancel()
 }

client/visitor.go

@@ -16,6 +16,7 @@ package client
 
 import (
 	"bytes"
+	"context"
 	"fmt"
 	"io"
 	"io/ioutil"
@@ -23,12 +24,11 @@ import (
 	"sync"
 	"time"
 
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/models/msg"
-	"github.com/fatedier/frp/utils/log"
 	frpNet "github.com/fatedier/frp/utils/net"
 	"github.com/fatedier/frp/utils/util"
+	"github.com/fatedier/frp/utils/xlog"
 
 	frpIo "github.com/fatedier/golib/io"
 	"github.com/fatedier/golib/pool"
@@ -39,13 +39,13 @@ import (
 type Visitor interface {
 	Run() error
 	Close()
-	log.Logger
 }
 
-func NewVisitor(ctl *Control, cfg config.VisitorConf) (visitor Visitor) {
+func NewVisitor(ctx context.Context, ctl *Control, cfg config.VisitorConf) (visitor Visitor) {
+	xl := xlog.FromContextSafe(ctx).Spawn().AppendPrefix(cfg.GetBaseInfo().ProxyName)
 	baseVisitor := BaseVisitor{
-		ctl:    ctl,
-		Logger: log.NewPrefixLogger(cfg.GetBaseInfo().ProxyName),
+		ctl: ctl,
+		ctx: xlog.NewContext(ctx, xl),
 	}
 	switch cfg := cfg.(type) {
 	case *config.StcpVisitorConf:
@@ -64,10 +64,11 @@ func NewVisitor(ctl *Control, cfg config.VisitorConf) (visitor Visitor) {
 
 type BaseVisitor struct {
 	ctl    *Control
-	l      frpNet.Listener
+	l      net.Listener
 	closed bool
-	mu     sync.RWMutex
-	log.Logger
+
+	mu  sync.RWMutex
+	ctx context.Context
 }
 
 type StcpVisitor struct {
@@ -77,7 +78,7 @@ type StcpVisitor struct {
 }
 
 func (sv *StcpVisitor) Run() (err error) {
-	sv.l, err = frpNet.ListenTcp(sv.cfg.BindAddr, sv.cfg.BindPort)
+	sv.l, err = net.Listen("tcp", fmt.Sprintf("%s:%d", sv.cfg.BindAddr, sv.cfg.BindPort))
 	if err != nil {
 		return
 	}
@@ -91,10 +92,11 @@ func (sv *StcpVisitor) Close() {
 }
 
 func (sv *StcpVisitor) worker() {
+	xl := xlog.FromContextSafe(sv.ctx)
 	for {
 		conn, err := sv.l.Accept()
 		if err != nil {
-			sv.Warn("stcp local listener closed")
+			xl.Warn("stcp local listener closed")
 			return
 		}
 
@@ -102,10 +104,11 @@ func (sv *StcpVisitor) worker() {
 	}
 }
 
-func (sv *StcpVisitor) handleConn(userConn frpNet.Conn) {
+func (sv *StcpVisitor) handleConn(userConn net.Conn) {
+	xl := xlog.FromContextSafe(sv.ctx)
 	defer userConn.Close()
 
-	sv.Debug("get a new stcp user connection")
+	xl.Debug("get a new stcp user connection")
 	visitorConn, err := sv.ctl.connectServer()
 	if err != nil {
 		return
@@ -122,7 +125,7 @@ func (sv *StcpVisitor) handleConn(userConn frpNet.Conn) {
 	}
 	err = msg.WriteMsg(visitorConn, newVisitorConnMsg)
 	if err != nil {
-		sv.Warn("send newVisitorConnMsg to server error: %v", err)
+		xl.Warn("send newVisitorConnMsg to server error: %v", err)
 		return
 	}
 
@@ -130,13 +133,13 @@ func (sv *StcpVisitor) handleConn(userConn frpNet.Conn) {
 	visitorConn.SetReadDeadline(time.Now().Add(10 * time.Second))
 	err = msg.ReadMsgInto(visitorConn, &newVisitorConnRespMsg)
 	if err != nil {
-		sv.Warn("get newVisitorConnRespMsg error: %v", err)
+		xl.Warn("get newVisitorConnRespMsg error: %v", err)
 		return
 	}
 	visitorConn.SetReadDeadline(time.Time{})
 
 	if newVisitorConnRespMsg.Error != "" {
-		sv.Warn("start new visitor connection error: %s", newVisitorConnRespMsg.Error)
+		xl.Warn("start new visitor connection error: %s", newVisitorConnRespMsg.Error)
 		return
 	}
 
@@ -145,7 +148,7 @@ func (sv *StcpVisitor) handleConn(userConn frpNet.Conn) {
 	if sv.cfg.UseEncryption {
 		remote, err = frpIo.WithEncryption(remote, []byte(sv.cfg.Sk))
 		if err != nil {
-			sv.Error("create encryption stream error: %v", err)
+			xl.Error("create encryption stream error: %v", err)
 			return
 		}
 	}
@@ -164,7 +167,7 @@ type XtcpVisitor struct {
 }
 
 func (sv *XtcpVisitor) Run() (err error) {
-	sv.l, err = frpNet.ListenTcp(sv.cfg.BindAddr, sv.cfg.BindPort)
+	sv.l, err = net.Listen("tcp", fmt.Sprintf("%s:%d", sv.cfg.BindAddr, sv.cfg.BindPort))
 	if err != nil {
 		return
 	}
@@ -178,10 +181,11 @@ func (sv *XtcpVisitor) Close() {
 }
 
 func (sv *XtcpVisitor) worker() {
+	xl := xlog.FromContextSafe(sv.ctx)
 	for {
 		conn, err := sv.l.Accept()
 		if err != nil {
-			sv.Warn("xtcp local listener closed")
+			xl.Warn("xtcp local listener closed")
 			return
 		}
 
@@ -189,25 +193,26 @@ func (sv *XtcpVisitor) worker() {
 	}
 }
 
-func (sv *XtcpVisitor) handleConn(userConn frpNet.Conn) {
+func (sv *XtcpVisitor) handleConn(userConn net.Conn) {
+	xl := xlog.FromContextSafe(sv.ctx)
 	defer userConn.Close()
 
-	sv.Debug("get a new xtcp user connection")
-	if g.GlbClientCfg.ServerUdpPort == 0 {
-		sv.Error("xtcp is not supported by server")
+	xl.Debug("get a new xtcp user connection")
+	if sv.ctl.serverUDPPort == 0 {
+		xl.Error("xtcp is not supported by server")
 		return
 	}
 
 	raddr, err := net.ResolveUDPAddr("udp",
-		fmt.Sprintf("%s:%d", g.GlbClientCfg.ServerAddr, g.GlbClientCfg.ServerUdpPort))
+		fmt.Sprintf("%s:%d", sv.ctl.clientCfg.ServerAddr, sv.ctl.serverUDPPort))
 	if err != nil {
-		sv.Error("resolve server UDP addr error")
+		xl.Error("resolve server UDP addr error")
 		return
 	}
 
 	visitorConn, err := net.DialUDP("udp", nil, raddr)
 	if err != nil {
-		sv.Warn("dial server udp addr error: %v", err)
+		xl.Warn("dial server udp addr error: %v", err)
 		return
 	}
 	defer visitorConn.Close()
@@ -220,7 +225,7 @@ func (sv *XtcpVisitor) handleConn(userConn frpNet.Conn) {
 	}
 	err = msg.WriteMsg(visitorConn, natHoleVisitorMsg)
 	if err != nil {
-		sv.Warn("send natHoleVisitorMsg to server error: %v", err)
+		xl.Warn("send natHoleVisitorMsg to server error: %v", err)
 		return
 	}
 
@@ -230,24 +235,24 @@ func (sv *XtcpVisitor) handleConn(userConn frpNet.Conn) {
 	buf := pool.GetBuf(1024)
 	n, err := visitorConn.Read(buf)
 	if err != nil {
-		sv.Warn("get natHoleRespMsg error: %v", err)
+		xl.Warn("get natHoleRespMsg error: %v", err)
 		return
 	}
 
 	err = msg.ReadMsgInto(bytes.NewReader(buf[:n]), &natHoleRespMsg)
 	if err != nil {
-		sv.Warn("get natHoleRespMsg error: %v", err)
+		xl.Warn("get natHoleRespMsg error: %v", err)
 		return
 	}
 	visitorConn.SetReadDeadline(time.Time{})
 	pool.PutBuf(buf)
 
 	if natHoleRespMsg.Error != "" {
-		sv.Error("natHoleRespMsg get error info: %s", natHoleRespMsg.Error)
+		xl.Error("natHoleRespMsg get error info: %s", natHoleRespMsg.Error)
 		return
 	}
 
-	sv.Trace("get natHoleRespMsg, sid [%s], client address [%s], visitor address [%s]", natHoleRespMsg.Sid, natHoleRespMsg.ClientAddr, natHoleRespMsg.VisitorAddr)
+	xl.Trace("get natHoleRespMsg, sid [%s], client address [%s], visitor address [%s]", natHoleRespMsg.Sid, natHoleRespMsg.ClientAddr, natHoleRespMsg.VisitorAddr)
 
 	// Close visitorConn, so we can use it's local address.
 	visitorConn.Close()
@@ -256,12 +261,12 @@ func (sv *XtcpVisitor) handleConn(userConn frpNet.Conn) {
 	laddr, _ := net.ResolveUDPAddr("udp", visitorConn.LocalAddr().String())
 	daddr, err := net.ResolveUDPAddr("udp", natHoleRespMsg.ClientAddr)
 	if err != nil {
-		sv.Error("resolve client udp address error: %v", err)
+		xl.Error("resolve client udp address error: %v", err)
 		return
 	}
 	lConn, err := net.DialUDP("udp", laddr, daddr)
 	if err != nil {
-		sv.Error("dial client udp address error: %v", err)
+		xl.Error("dial client udp address error: %v", err)
 		return
 	}
 	defer lConn.Close()
@@ -273,53 +278,53 @@ func (sv *XtcpVisitor) handleConn(userConn frpNet.Conn) {
 	lConn.SetReadDeadline(time.Now().Add(8 * time.Second))
 	n, err = lConn.Read(sidBuf)
 	if err != nil {
-		sv.Warn("get sid from client error: %v", err)
+		xl.Warn("get sid from client error: %v", err)
 		return
 	}
 	lConn.SetReadDeadline(time.Time{})
 	if string(sidBuf[:n]) != natHoleRespMsg.Sid {
-		sv.Warn("incorrect sid from client")
+		xl.Warn("incorrect sid from client")
 		return
 	}
 	pool.PutBuf(sidBuf)
 
-	sv.Info("nat hole connection make success, sid [%s]", natHoleRespMsg.Sid)
+	xl.Info("nat hole connection make success, sid [%s]", natHoleRespMsg.Sid)
 
 	// wrap kcp connection
 	var remote io.ReadWriteCloser
 	remote, err = frpNet.NewKcpConnFromUdp(lConn, true, natHoleRespMsg.ClientAddr)
 	if err != nil {
-		sv.Error("create kcp connection from udp connection error: %v", err)
+		xl.Error("create kcp connection from udp connection error: %v", err)
 		return
 	}
 
-	if sv.cfg.UseEncryption {
-		remote, err = frpIo.WithEncryption(remote, []byte(sv.cfg.Sk))
-		if err != nil {
-			sv.Error("create encryption stream error: %v", err)
-			return
-		}
-	}
-
-	if sv.cfg.UseCompression {
-		remote = frpIo.WithCompression(remote)
-	}
-
 	fmuxCfg := fmux.DefaultConfig()
 	fmuxCfg.KeepAliveInterval = 5 * time.Second
 	fmuxCfg.LogOutput = ioutil.Discard
 	sess, err := fmux.Client(remote, fmuxCfg)
 	if err != nil {
-		sv.Error("create yamux session error: %v", err)
+		xl.Error("create yamux session error: %v", err)
 		return
 	}
 	defer sess.Close()
 	muxConn, err := sess.Open()
 	if err != nil {
-		sv.Error("open yamux stream error: %v", err)
+		xl.Error("open yamux stream error: %v", err)
 		return
 	}
 
-	frpIo.Join(userConn, muxConn)
-	sv.Debug("join connections closed")
+	var muxConnRWCloser io.ReadWriteCloser = muxConn
+	if sv.cfg.UseEncryption {
+		muxConnRWCloser, err = frpIo.WithEncryption(muxConnRWCloser, []byte(sv.cfg.Sk))
+		if err != nil {
+			xl.Error("create encryption stream error: %v", err)
+			return
+		}
+	}
+	if sv.cfg.UseCompression {
+		muxConnRWCloser = frpIo.WithCompression(muxConnRWCloser)
+	}
+
+	frpIo.Join(userConn, muxConnRWCloser)
+	xl.Debug("join connections closed")
 }

client/visitor_manager.go

@@ -15,11 +15,12 @@
 package client
 
 import (
+	"context"
 	"sync"
 	"time"
 
 	"github.com/fatedier/frp/models/config"
-	"github.com/fatedier/frp/utils/log"
+	"github.com/fatedier/frp/utils/xlog"
 )
 
 type VisitorManager struct {
@@ -30,26 +31,29 @@ type VisitorManager struct {
 
 	checkInterval time.Duration
 
-	mu sync.Mutex
+	mu  sync.Mutex
+	ctx context.Context
 }
 
-func NewVisitorManager(ctl *Control) *VisitorManager {
+func NewVisitorManager(ctx context.Context, ctl *Control) *VisitorManager {
 	return &VisitorManager{
 		ctl:           ctl,
 		cfgs:          make(map[string]config.VisitorConf),
 		visitors:      make(map[string]Visitor),
 		checkInterval: 10 * time.Second,
+		ctx:           ctx,
 	}
 }
 
 func (vm *VisitorManager) Run() {
+	xl := xlog.FromContextSafe(vm.ctx)
 	for {
 		time.Sleep(vm.checkInterval)
 		vm.mu.Lock()
 		for _, cfg := range vm.cfgs {
 			name := cfg.GetBaseInfo().ProxyName
 			if _, exist := vm.visitors[name]; !exist {
-				log.Info("try to start visitor [%s]", name)
+				xl.Info("try to start visitor [%s]", name)
 				vm.startVisitor(cfg)
 			}
 		}
@@ -59,19 +63,21 @@ func (vm *VisitorManager) Run() {
 
 // Hold lock before calling this function.
 func (vm *VisitorManager) startVisitor(cfg config.VisitorConf) (err error) {
+	xl := xlog.FromContextSafe(vm.ctx)
 	name := cfg.GetBaseInfo().ProxyName
-	visitor := NewVisitor(vm.ctl, cfg)
+	visitor := NewVisitor(vm.ctx, vm.ctl, cfg)
 	err = visitor.Run()
 	if err != nil {
-		visitor.Warn("start error: %v", err)
+		xl.Warn("start error: %v", err)
 	} else {
 		vm.visitors[name] = visitor
-		visitor.Info("start visitor success")
+		xl.Info("start visitor success")
 	}
 	return
 }
 
 func (vm *VisitorManager) Reload(cfgs map[string]config.VisitorConf) {
+	xl := xlog.FromContextSafe(vm.ctx)
 	vm.mu.Lock()
 	defer vm.mu.Unlock()
 
@@ -97,7 +103,7 @@ func (vm *VisitorManager) Reload(cfgs map[string]config.VisitorConf) {
 		}
 	}
 	if len(delNames) > 0 {
-		log.Info("visitor removed: %v", delNames)
+		xl.Info("visitor removed: %v", delNames)
 	}
 
 	addNames := make([]string, 0)
@@ -109,7 +115,7 @@ func (vm *VisitorManager) Reload(cfgs map[string]config.VisitorConf) {
 		}
 	}
 	if len(addNames) > 0 {
-		log.Info("visitor added: %v", addNames)
+		xl.Info("visitor added: %v", addNames)
 	}
 	return
 }

cmd/frpc/main.go

@@ -15,6 +15,9 @@
 package main
 
 import (
+	"math/rand"
+	"time"
+
 	_ "github.com/fatedier/frp/assets/frpc/statik"
 	"github.com/fatedier/frp/cmd/frpc/sub"
 
@@ -23,6 +26,7 @@ import (
 
 func main() {
 	crypto.DefaultSalt = "frp"
+	rand.Seed(time.Now().UnixNano())
 
 	sub.Execute()
 }

cmd/frpc/sub/http.go

@@ -33,6 +33,7 @@ func init() {
 	httpCmd.PersistentFlags().StringVarP(&logLevel, "log_level", "", "info", "log level")
 	httpCmd.PersistentFlags().StringVarP(&logFile, "log_file", "", "console", "console or file path")
 	httpCmd.PersistentFlags().IntVarP(&logMaxDays, "log_max_days", "", 3, "log file reversed days")
+	httpCmd.PersistentFlags().BoolVarP(&disableLogColor, "disable_log_color", "", false, "disable log color in console")
 
 	httpCmd.PersistentFlags().StringVarP(&proxyName, "proxy_name", "n", "", "proxy name")
 	httpCmd.PersistentFlags().StringVarP(&localIp, "local_ip", "i", "127.0.0.1", "local ip")
@@ -53,7 +54,7 @@ var httpCmd = &cobra.Command{
 	Use:   "http",
 	Short: "Run frpc with a single http proxy",
 	RunE: func(cmd *cobra.Command, args []string) error {
-		err := parseClientCommonCfg(CfgFileTypeCmd, "")
+		clientCfg, err := parseClientCommonCfg(CfgFileTypeCmd, "")
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)
@@ -86,7 +87,7 @@ var httpCmd = &cobra.Command{
 		proxyConfs := map[string]config.ProxyConf{
 			cfg.ProxyName: cfg,
 		}
-		err = startService(proxyConfs, nil)
+		err = startService(clientCfg, proxyConfs, nil, "")
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)

cmd/frpc/sub/https.go

@@ -33,6 +33,7 @@ func init() {
 	httpsCmd.PersistentFlags().StringVarP(&logLevel, "log_level", "", "info", "log level")
 	httpsCmd.PersistentFlags().StringVarP(&logFile, "log_file", "", "console", "console or file path")
 	httpsCmd.PersistentFlags().IntVarP(&logMaxDays, "log_max_days", "", 3, "log file reversed days")
+	httpsCmd.PersistentFlags().BoolVarP(&disableLogColor, "disable_log_color", "", false, "disable log color in console")
 
 	httpsCmd.PersistentFlags().StringVarP(&proxyName, "proxy_name", "n", "", "proxy name")
 	httpsCmd.PersistentFlags().StringVarP(&localIp, "local_ip", "i", "127.0.0.1", "local ip")
@@ -49,7 +50,7 @@ var httpsCmd = &cobra.Command{
 	Use:   "https",
 	Short: "Run frpc with a single https proxy",
 	RunE: func(cmd *cobra.Command, args []string) error {
-		err := parseClientCommonCfg(CfgFileTypeCmd, "")
+		clientCfg, err := parseClientCommonCfg(CfgFileTypeCmd, "")
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)
@@ -78,7 +79,7 @@ var httpsCmd = &cobra.Command{
 		proxyConfs := map[string]config.ProxyConf{
 			cfg.ProxyName: cfg,
 		}
-		err = startService(proxyConfs, nil)
+		err = startService(clientCfg, proxyConfs, nil, "")
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)

cmd/frpc/sub/reload.go

@@ -24,7 +24,6 @@ import (
 
 	"github.com/spf13/cobra"
 
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 )
 
@@ -42,13 +41,13 @@ var reloadCmd = &cobra.Command{
 			os.Exit(1)
 		}
 
-		err = parseClientCommonCfg(CfgFileTypeIni, iniContent)
+		clientCfg, err := parseClientCommonCfg(CfgFileTypeIni, iniContent)
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)
 		}
 
-		err = reload()
+		err = reload(clientCfg)
 		if err != nil {
 			fmt.Printf("frpc reload error: %v\n", err)
 			os.Exit(1)
@@ -58,35 +57,34 @@ var reloadCmd = &cobra.Command{
 	},
 }
 
-func reload() error {
-	if g.GlbClientCfg.AdminPort == 0 {
+func reload(clientCfg config.ClientCommonConf) error {
+	if clientCfg.AdminPort == 0 {
 		return fmt.Errorf("admin_port shoud be set if you want to use reload feature")
 	}
 
 	req, err := http.NewRequest("GET", "http://"+
-		g.GlbClientCfg.AdminAddr+":"+fmt.Sprintf("%d", g.GlbClientCfg.AdminPort)+"/api/reload", nil)
+		clientCfg.AdminAddr+":"+fmt.Sprintf("%d", clientCfg.AdminPort)+"/api/reload", nil)
 	if err != nil {
 		return err
 	}
 
-	authStr := "Basic " + base64.StdEncoding.EncodeToString([]byte(g.GlbClientCfg.AdminUser+":"+
-		g.GlbClientCfg.AdminPwd))
+	authStr := "Basic " + base64.StdEncoding.EncodeToString([]byte(clientCfg.AdminUser+":"+
+		clientCfg.AdminPwd))
 
 	req.Header.Add("Authorization", authStr)
 	resp, err := http.DefaultClient.Do(req)
 	if err != nil {
 		return err
-	} else {
-		if resp.StatusCode == 200 {
-			return nil
-		}
-
-		defer resp.Body.Close()
-		body, err := ioutil.ReadAll(resp.Body)
-		if err != nil {
-			return err
-		}
-		return fmt.Errorf("code [%d], %s", resp.StatusCode, strings.TrimSpace(string(body)))
 	}
-	return nil
+	defer resp.Body.Close()
+
+	if resp.StatusCode == 200 {
+		return nil
+	}
+
+	body, err := ioutil.ReadAll(resp.Body)
+	if err != nil {
+		return err
+	}
+	return fmt.Errorf("code [%d], %s", resp.StatusCode, strings.TrimSpace(string(body)))
 }

cmd/frpc/sub/root.go

@@ -28,7 +28,6 @@ import (
 	"github.com/spf13/cobra"
 
 	"github.com/fatedier/frp/client"
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/utils/log"
 	"github.com/fatedier/frp/utils/version"
@@ -43,13 +42,14 @@ var (
 	cfgFile     string
 	showVersion bool
 
-	serverAddr string
-	user       string
-	protocol   string
-	token      string
-	logLevel   string
-	logFile    string
-	logMaxDays int
+	serverAddr      string
+	user            string
+	protocol        string
+	token           string
+	logLevel        string
+	logFile         string
+	logMaxDays      int
+	disableLogColor bool
 
 	proxyName         string
 	localIp           string
@@ -113,59 +113,62 @@ func handleSignal(svr *client.Service) {
 	close(kcpDoneCh)
 }
 
-func parseClientCommonCfg(fileType int, content string) (err error) {
+func parseClientCommonCfg(fileType int, content string) (cfg config.ClientCommonConf, err error) {
 	if fileType == CfgFileTypeIni {
-		err = parseClientCommonCfgFromIni(content)
+		cfg, err = parseClientCommonCfgFromIni(content)
 	} else if fileType == CfgFileTypeCmd {
-		err = parseClientCommonCfgFromCmd()
+		cfg, err = parseClientCommonCfgFromCmd()
 	}
 	if err != nil {
 		return
 	}
 
-	err = g.GlbClientCfg.ClientCommonConf.Check()
+	err = cfg.Check()
 	if err != nil {
 		return
 	}
 	return
 }
 
-func parseClientCommonCfgFromIni(content string) (err error) {
-	cfg, err := config.UnmarshalClientConfFromIni(&g.GlbClientCfg.ClientCommonConf, content)
+func parseClientCommonCfgFromIni(content string) (config.ClientCommonConf, error) {
+	cfg, err := config.UnmarshalClientConfFromIni(content)
 	if err != nil {
-		return err
+		return config.ClientCommonConf{}, err
 	}
-	g.GlbClientCfg.ClientCommonConf = *cfg
-	return
+	return cfg, err
 }
 
-func parseClientCommonCfgFromCmd() (err error) {
+func parseClientCommonCfgFromCmd() (cfg config.ClientCommonConf, err error) {
+	cfg = config.GetDefaultClientConf()
+
 	strs := strings.Split(serverAddr, ":")
 	if len(strs) < 2 {
 		err = fmt.Errorf("invalid server_addr")
 		return
 	}
 	if strs[0] != "" {
-		g.GlbClientCfg.ServerAddr = strs[0]
+		cfg.ServerAddr = strs[0]
 	}
-	g.GlbClientCfg.ServerPort, err = strconv.Atoi(strs[1])
+	cfg.ServerPort, err = strconv.Atoi(strs[1])
 	if err != nil {
 		err = fmt.Errorf("invalid server_addr")
 		return
 	}
 
-	g.GlbClientCfg.User = user
-	g.GlbClientCfg.Protocol = protocol
-	g.GlbClientCfg.Token = token
-	g.GlbClientCfg.LogLevel = logLevel
-	g.GlbClientCfg.LogFile = logFile
-	g.GlbClientCfg.LogMaxDays = int64(logMaxDays)
+	cfg.User = user
+	cfg.Protocol = protocol
+	cfg.Token = token
+	cfg.LogLevel = logLevel
+	cfg.LogFile = logFile
+	cfg.LogMaxDays = int64(logMaxDays)
 	if logFile == "console" {
-		g.GlbClientCfg.LogWay = "console"
+		cfg.LogWay = "console"
 	} else {
-		g.GlbClientCfg.LogWay = "file"
+		cfg.LogWay = "file"
 	}
-	return nil
+	cfg.DisableLogColor = disableLogColor
+
+	return
 }
 
 func runClient(cfgFilePath string) (err error) {
@@ -174,26 +177,27 @@ func runClient(cfgFilePath string) (err error) {
 	if err != nil {
 		return
 	}
-	g.GlbClientCfg.CfgFile = cfgFilePath
 
-	err = parseClientCommonCfg(CfgFileTypeIni, content)
+	cfg, err := parseClientCommonCfg(CfgFileTypeIni, content)
 	if err != nil {
 		return
 	}
 
-	pxyCfgs, visitorCfgs, err := config.LoadAllConfFromIni(g.GlbClientCfg.User, content, g.GlbClientCfg.Start)
+	pxyCfgs, visitorCfgs, err := config.LoadAllConfFromIni(cfg.User, content, cfg.Start)
 	if err != nil {
 		return err
 	}
 
-	err = startService(pxyCfgs, visitorCfgs)
+	err = startService(cfg, pxyCfgs, visitorCfgs, cfgFilePath)
 	return
 }
 
-func startService(pxyCfgs map[string]config.ProxyConf, visitorCfgs map[string]config.VisitorConf) (err error) {
-	log.InitLog(g.GlbClientCfg.LogWay, g.GlbClientCfg.LogFile, g.GlbClientCfg.LogLevel, g.GlbClientCfg.LogMaxDays)
-	if g.GlbClientCfg.DnsServer != "" {
-		s := g.GlbClientCfg.DnsServer
+func startService(cfg config.ClientCommonConf, pxyCfgs map[string]config.ProxyConf, visitorCfgs map[string]config.VisitorConf, cfgFile string) (err error) {
+	log.InitLog(cfg.LogWay, cfg.LogFile, cfg.LogLevel,
+		cfg.LogMaxDays, cfg.DisableLogColor)
+
+	if cfg.DnsServer != "" {
+		s := cfg.DnsServer
 		if !strings.Contains(s, ":") {
 			s += ":53"
 		}
@@ -205,19 +209,19 @@ func startService(pxyCfgs map[string]config.ProxyConf, visitorCfgs map[string]co
 			},
 		}
 	}
-	svr, errRet := client.NewService(pxyCfgs, visitorCfgs)
+	svr, errRet := client.NewService(cfg, pxyCfgs, visitorCfgs, cfgFile)
 	if errRet != nil {
 		err = errRet
 		return
 	}
 
 	// Capture the exit signal if we use kcp.
-	if g.GlbClientCfg.Protocol == "kcp" {
+	if cfg.Protocol == "kcp" {
 		go handleSignal(svr)
 	}
 
 	err = svr.Run()
-	if g.GlbClientCfg.Protocol == "kcp" {
+	if cfg.Protocol == "kcp" {
 		<-kcpDoneCh
 	}
 	return

cmd/frpc/sub/status.go

@@ -27,7 +27,6 @@ import (
 	"github.com/spf13/cobra"
 
 	"github.com/fatedier/frp/client"
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 )
 
@@ -45,13 +44,13 @@ var statusCmd = &cobra.Command{
 			os.Exit(1)
 		}
 
-		err = parseClientCommonCfg(CfgFileTypeIni, iniContent)
+		clientCfg, err := parseClientCommonCfg(CfgFileTypeIni, iniContent)
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)
 		}
 
-		err = status()
+		err = status(clientCfg)
 		if err != nil {
 			fmt.Printf("frpc get status error: %v\n", err)
 			os.Exit(1)
@@ -60,94 +59,96 @@ var statusCmd = &cobra.Command{
 	},
 }
 
-func status() error {
-	if g.GlbClientCfg.AdminPort == 0 {
+func status(clientCfg config.ClientCommonConf) error {
+	if clientCfg.AdminPort == 0 {
 		return fmt.Errorf("admin_port shoud be set if you want to get proxy status")
 	}
 
 	req, err := http.NewRequest("GET", "http://"+
-		g.GlbClientCfg.AdminAddr+":"+fmt.Sprintf("%d", g.GlbClientCfg.AdminPort)+"/api/status", nil)
+		clientCfg.AdminAddr+":"+fmt.Sprintf("%d", clientCfg.AdminPort)+"/api/status", nil)
 	if err != nil {
 		return err
 	}
 
-	authStr := "Basic " + base64.StdEncoding.EncodeToString([]byte(g.GlbClientCfg.AdminUser+":"+
-		g.GlbClientCfg.AdminPwd))
+	authStr := "Basic " + base64.StdEncoding.EncodeToString([]byte(clientCfg.AdminUser+":"+
+		clientCfg.AdminPwd))
 
 	req.Header.Add("Authorization", authStr)
 	resp, err := http.DefaultClient.Do(req)
 	if err != nil {
 		return err
-	} else {
-		if resp.StatusCode != 200 {
-			return fmt.Errorf("admin api status code [%d]", resp.StatusCode)
-		}
-		defer resp.Body.Close()
-		body, err := ioutil.ReadAll(resp.Body)
-		if err != nil {
-			return err
-		}
-		res := &client.StatusResp{}
-		err = json.Unmarshal(body, &res)
-		if err != nil {
-			return fmt.Errorf("unmarshal http response error: %s", strings.TrimSpace(string(body)))
-		}
-
-		fmt.Println("Proxy Status...")
-		if len(res.Tcp) > 0 {
-			fmt.Printf("TCP")
-			tbl := table.New("Name", "Status", "LocalAddr", "Plugin", "RemoteAddr", "Error")
-			for _, ps := range res.Tcp {
-				tbl.AddRow(ps.Name, ps.Status, ps.LocalAddr, ps.Plugin, ps.RemoteAddr, ps.Err)
-			}
-			tbl.Print()
-			fmt.Println("")
-		}
-		if len(res.Udp) > 0 {
-			fmt.Printf("UDP")
-			tbl := table.New("Name", "Status", "LocalAddr", "Plugin", "RemoteAddr", "Error")
-			for _, ps := range res.Udp {
-				tbl.AddRow(ps.Name, ps.Status, ps.LocalAddr, ps.Plugin, ps.RemoteAddr, ps.Err)
-			}
-			tbl.Print()
-			fmt.Println("")
-		}
-		if len(res.Http) > 0 {
-			fmt.Printf("HTTP")
-			tbl := table.New("Name", "Status", "LocalAddr", "Plugin", "RemoteAddr", "Error")
-			for _, ps := range res.Http {
-				tbl.AddRow(ps.Name, ps.Status, ps.LocalAddr, ps.Plugin, ps.RemoteAddr, ps.Err)
-			}
-			tbl.Print()
-			fmt.Println("")
-		}
-		if len(res.Https) > 0 {
-			fmt.Printf("HTTPS")
-			tbl := table.New("Name", "Status", "LocalAddr", "Plugin", "RemoteAddr", "Error")
-			for _, ps := range res.Https {
-				tbl.AddRow(ps.Name, ps.Status, ps.LocalAddr, ps.Plugin, ps.RemoteAddr, ps.Err)
-			}
-			tbl.Print()
-			fmt.Println("")
-		}
-		if len(res.Stcp) > 0 {
-			fmt.Printf("STCP")
-			tbl := table.New("Name", "Status", "LocalAddr", "Plugin", "RemoteAddr", "Error")
-			for _, ps := range res.Stcp {
-				tbl.AddRow(ps.Name, ps.Status, ps.LocalAddr, ps.Plugin, ps.RemoteAddr, ps.Err)
-			}
-			tbl.Print()
-			fmt.Println("")
-		}
-		if len(res.Xtcp) > 0 {
-			fmt.Printf("XTCP")
-			tbl := table.New("Name", "Status", "LocalAddr", "Plugin", "RemoteAddr", "Error")
-			for _, ps := range res.Xtcp {
-				tbl.AddRow(ps.Name, ps.Status, ps.LocalAddr, ps.Plugin, ps.RemoteAddr, ps.Err)
-			}
-			tbl.Print()
-			fmt.Println("")
-		}
 	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != 200 {
+		return fmt.Errorf("admin api status code [%d]", resp.StatusCode)
+	}
+
+	body, err := ioutil.ReadAll(resp.Body)
+	if err != nil {
+		return err
+	}
+	res := &client.StatusResp{}
+	err = json.Unmarshal(body, &res)
+	if err != nil {
+		return fmt.Errorf("unmarshal http response error: %s", strings.TrimSpace(string(body)))
+	}
+
+	fmt.Println("Proxy Status...")
+	if len(res.Tcp) > 0 {
+		fmt.Printf("TCP")
+		tbl := table.New("Name", "Status", "LocalAddr", "Plugin", "RemoteAddr", "Error")
+		for _, ps := range res.Tcp {
+			tbl.AddRow(ps.Name, ps.Status, ps.LocalAddr, ps.Plugin, ps.RemoteAddr, ps.Err)
+		}
+		tbl.Print()
+		fmt.Println("")
+	}
+	if len(res.Udp) > 0 {
+		fmt.Printf("UDP")
+		tbl := table.New("Name", "Status", "LocalAddr", "Plugin", "RemoteAddr", "Error")
+		for _, ps := range res.Udp {
+			tbl.AddRow(ps.Name, ps.Status, ps.LocalAddr, ps.Plugin, ps.RemoteAddr, ps.Err)
+		}
+		tbl.Print()
+		fmt.Println("")
+	}
+	if len(res.Http) > 0 {
+		fmt.Printf("HTTP")
+		tbl := table.New("Name", "Status", "LocalAddr", "Plugin", "RemoteAddr", "Error")
+		for _, ps := range res.Http {
+			tbl.AddRow(ps.Name, ps.Status, ps.LocalAddr, ps.Plugin, ps.RemoteAddr, ps.Err)
+		}
+		tbl.Print()
+		fmt.Println("")
+	}
+	if len(res.Https) > 0 {
+		fmt.Printf("HTTPS")
+		tbl := table.New("Name", "Status", "LocalAddr", "Plugin", "RemoteAddr", "Error")
+		for _, ps := range res.Https {
+			tbl.AddRow(ps.Name, ps.Status, ps.LocalAddr, ps.Plugin, ps.RemoteAddr, ps.Err)
+		}
+		tbl.Print()
+		fmt.Println("")
+	}
+	if len(res.Stcp) > 0 {
+		fmt.Printf("STCP")
+		tbl := table.New("Name", "Status", "LocalAddr", "Plugin", "RemoteAddr", "Error")
+		for _, ps := range res.Stcp {
+			tbl.AddRow(ps.Name, ps.Status, ps.LocalAddr, ps.Plugin, ps.RemoteAddr, ps.Err)
+		}
+		tbl.Print()
+		fmt.Println("")
+	}
+	if len(res.Xtcp) > 0 {
+		fmt.Printf("XTCP")
+		tbl := table.New("Name", "Status", "LocalAddr", "Plugin", "RemoteAddr", "Error")
+		for _, ps := range res.Xtcp {
+			tbl.AddRow(ps.Name, ps.Status, ps.LocalAddr, ps.Plugin, ps.RemoteAddr, ps.Err)
+		}
+		tbl.Print()
+		fmt.Println("")
+	}
+
 	return nil
 }

cmd/frpc/sub/stcp.go

@@ -32,6 +32,7 @@ func init() {
 	stcpCmd.PersistentFlags().StringVarP(&logLevel, "log_level", "", "info", "log level")
 	stcpCmd.PersistentFlags().StringVarP(&logFile, "log_file", "", "console", "console or file path")
 	stcpCmd.PersistentFlags().IntVarP(&logMaxDays, "log_max_days", "", 3, "log file reversed days")
+	stcpCmd.PersistentFlags().BoolVarP(&disableLogColor, "disable_log_color", "", false, "disable log color in console")
 
 	stcpCmd.PersistentFlags().StringVarP(&proxyName, "proxy_name", "n", "", "proxy name")
 	stcpCmd.PersistentFlags().StringVarP(&role, "role", "", "server", "role")
@@ -51,7 +52,7 @@ var stcpCmd = &cobra.Command{
 	Use:   "stcp",
 	Short: "Run frpc with a single stcp proxy",
 	RunE: func(cmd *cobra.Command, args []string) error {
-		err := parseClientCommonCfg(CfgFileTypeCmd, "")
+		clientCfg, err := parseClientCommonCfg(CfgFileTypeCmd, "")
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)
@@ -103,7 +104,7 @@ var stcpCmd = &cobra.Command{
 			os.Exit(1)
 		}
 
-		err = startService(proxyConfs, visitorConfs)
+		err = startService(clientCfg, proxyConfs, visitorConfs, "")
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)

cmd/frpc/sub/tcp.go

@@ -32,6 +32,7 @@ func init() {
 	tcpCmd.PersistentFlags().StringVarP(&logLevel, "log_level", "", "info", "log level")
 	tcpCmd.PersistentFlags().StringVarP(&logFile, "log_file", "", "console", "console or file path")
 	tcpCmd.PersistentFlags().IntVarP(&logMaxDays, "log_max_days", "", 3, "log file reversed days")
+	tcpCmd.PersistentFlags().BoolVarP(&disableLogColor, "disable_log_color", "", false, "disable log color in console")
 
 	tcpCmd.PersistentFlags().StringVarP(&proxyName, "proxy_name", "n", "", "proxy name")
 	tcpCmd.PersistentFlags().StringVarP(&localIp, "local_ip", "i", "127.0.0.1", "local ip")
@@ -47,7 +48,7 @@ var tcpCmd = &cobra.Command{
 	Use:   "tcp",
 	Short: "Run frpc with a single tcp proxy",
 	RunE: func(cmd *cobra.Command, args []string) error {
-		err := parseClientCommonCfg(CfgFileTypeCmd, "")
+		clientCfg, err := parseClientCommonCfg(CfgFileTypeCmd, "")
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)
@@ -75,7 +76,7 @@ var tcpCmd = &cobra.Command{
 		proxyConfs := map[string]config.ProxyConf{
 			cfg.ProxyName: cfg,
 		}
-		err = startService(proxyConfs, nil)
+		err = startService(clientCfg, proxyConfs, nil, "")
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)

cmd/frpc/sub/udp.go

@@ -32,6 +32,7 @@ func init() {
 	udpCmd.PersistentFlags().StringVarP(&logLevel, "log_level", "", "info", "log level")
 	udpCmd.PersistentFlags().StringVarP(&logFile, "log_file", "", "console", "console or file path")
 	udpCmd.PersistentFlags().IntVarP(&logMaxDays, "log_max_days", "", 3, "log file reversed days")
+	udpCmd.PersistentFlags().BoolVarP(&disableLogColor, "disable_log_color", "", false, "disable log color in console")
 
 	udpCmd.PersistentFlags().StringVarP(&proxyName, "proxy_name", "n", "", "proxy name")
 	udpCmd.PersistentFlags().StringVarP(&localIp, "local_ip", "i", "127.0.0.1", "local ip")
@@ -47,7 +48,7 @@ var udpCmd = &cobra.Command{
 	Use:   "udp",
 	Short: "Run frpc with a single udp proxy",
 	RunE: func(cmd *cobra.Command, args []string) error {
-		err := parseClientCommonCfg(CfgFileTypeCmd, "")
+		clientCfg, err := parseClientCommonCfg(CfgFileTypeCmd, "")
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)
@@ -75,7 +76,7 @@ var udpCmd = &cobra.Command{
 		proxyConfs := map[string]config.ProxyConf{
 			cfg.ProxyName: cfg,
 		}
-		err = startService(proxyConfs, nil)
+		err = startService(clientCfg, proxyConfs, nil, "")
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)

cmd/frpc/sub/xtcp.go

@@ -32,6 +32,7 @@ func init() {
 	xtcpCmd.PersistentFlags().StringVarP(&logLevel, "log_level", "", "info", "log level")
 	xtcpCmd.PersistentFlags().StringVarP(&logFile, "log_file", "", "console", "console or file path")
 	xtcpCmd.PersistentFlags().IntVarP(&logMaxDays, "log_max_days", "", 3, "log file reversed days")
+	xtcpCmd.PersistentFlags().BoolVarP(&disableLogColor, "disable_log_color", "", false, "disable log color in console")
 
 	xtcpCmd.PersistentFlags().StringVarP(&proxyName, "proxy_name", "n", "", "proxy name")
 	xtcpCmd.PersistentFlags().StringVarP(&role, "role", "", "server", "role")
@@ -51,7 +52,7 @@ var xtcpCmd = &cobra.Command{
 	Use:   "xtcp",
 	Short: "Run frpc with a single xtcp proxy",
 	RunE: func(cmd *cobra.Command, args []string) error {
-		err := parseClientCommonCfg(CfgFileTypeCmd, "")
+		clientCfg, err := parseClientCommonCfg(CfgFileTypeCmd, "")
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)
@@ -103,7 +104,7 @@ var xtcpCmd = &cobra.Command{
 			os.Exit(1)
 		}
 
-		err = startService(proxyConfs, visitorConfs)
+		err = startService(clientCfg, proxyConfs, visitorConfs, "")
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)

cmd/frps/main.go

@@ -15,6 +15,9 @@
 package main
 
 import (
+	"math/rand"
+	"time"
+
 	"github.com/fatedier/golib/crypto"
 
 	_ "github.com/fatedier/frp/assets/frps/statik"
@@ -22,6 +25,7 @@ import (
 
 func main() {
 	crypto.DefaultSalt = "frp"
+	rand.Seed(time.Now().UnixNano())
 
 	Execute()
 }

cmd/frps/root.go

@@ -20,7 +20,6 @@ import (
 
 	"github.com/spf13/cobra"
 
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/server"
 	"github.com/fatedier/frp/utils/log"
@@ -53,6 +52,7 @@ var (
 	logFile           string
 	logLevel          string
 	logMaxDays        int64
+	disableLogColor   bool
 	token             string
 	subDomainHost     string
 	tcpMux            bool
@@ -79,7 +79,9 @@ func init() {
 	rootCmd.PersistentFlags().StringVarP(&dashboardPwd, "dashboard_pwd", "", "admin", "dashboard password")
 	rootCmd.PersistentFlags().StringVarP(&logFile, "log_file", "", "console", "log file")
 	rootCmd.PersistentFlags().StringVarP(&logLevel, "log_level", "", "info", "log level")
-	rootCmd.PersistentFlags().Int64VarP(&logMaxDays, "log_max_days", "", 3, "log_max_days")
+	rootCmd.PersistentFlags().Int64VarP(&logMaxDays, "log_max_days", "", 3, "log max days")
+	rootCmd.PersistentFlags().BoolVarP(&disableLogColor, "disable_log_color", "", false, "disable log color in console")
+
 	rootCmd.PersistentFlags().StringVarP(&token, "token", "t", "", "auth token")
 	rootCmd.PersistentFlags().StringVarP(&subDomainHost, "subdomain_host", "", "", "subdomain host")
 	rootCmd.PersistentFlags().StringVarP(&allowPorts, "allow_ports", "", "", "allow ports")
@@ -95,6 +97,7 @@ var rootCmd = &cobra.Command{
 			return nil
 		}
 
+		var cfg config.ServerCommonConf
 		var err error
 		if cfgFile != "" {
 			var content string
@@ -102,16 +105,15 @@ var rootCmd = &cobra.Command{
 			if err != nil {
 				return err
 			}
-			g.GlbServerCfg.CfgFile = cfgFile
-			err = parseServerCommonCfg(CfgFileTypeIni, content)
+			cfg, err = parseServerCommonCfg(CfgFileTypeIni, content)
 		} else {
-			err = parseServerCommonCfg(CfgFileTypeCmd, "")
+			cfg, err = parseServerCommonCfg(CfgFileTypeCmd, "")
 		}
 		if err != nil {
 			return err
 		}
 
-		err = runServer()
+		err = runServer(cfg)
 		if err != nil {
 			fmt.Println(err)
 			os.Exit(1)
@@ -126,52 +128,51 @@ func Execute() {
 	}
 }
 
-func parseServerCommonCfg(fileType int, content string) (err error) {
+func parseServerCommonCfg(fileType int, content string) (cfg config.ServerCommonConf, err error) {
 	if fileType == CfgFileTypeIni {
-		err = parseServerCommonCfgFromIni(content)
+		cfg, err = parseServerCommonCfgFromIni(content)
 	} else if fileType == CfgFileTypeCmd {
-		err = parseServerCommonCfgFromCmd()
+		cfg, err = parseServerCommonCfgFromCmd()
 	}
 	if err != nil {
 		return
 	}
 
-	err = g.GlbServerCfg.ServerCommonConf.Check()
+	err = cfg.Check()
 	if err != nil {
 		return
 	}
-
-	config.InitServerCfg(&g.GlbServerCfg.ServerCommonConf)
 	return
 }
 
-func parseServerCommonCfgFromIni(content string) (err error) {
-	cfg, err := config.UnmarshalServerConfFromIni(&g.GlbServerCfg.ServerCommonConf, content)
+func parseServerCommonCfgFromIni(content string) (config.ServerCommonConf, error) {
+	cfg, err := config.UnmarshalServerConfFromIni(content)
 	if err != nil {
-		return err
+		return config.ServerCommonConf{}, err
 	}
-	g.GlbServerCfg.ServerCommonConf = *cfg
-	return
+	return cfg, nil
 }
 
-func parseServerCommonCfgFromCmd() (err error) {
-	g.GlbServerCfg.BindAddr = bindAddr
-	g.GlbServerCfg.BindPort = bindPort
-	g.GlbServerCfg.BindUdpPort = bindUdpPort
-	g.GlbServerCfg.KcpBindPort = kcpBindPort
-	g.GlbServerCfg.ProxyBindAddr = proxyBindAddr
-	g.GlbServerCfg.VhostHttpPort = vhostHttpPort
-	g.GlbServerCfg.VhostHttpsPort = vhostHttpsPort
-	g.GlbServerCfg.VhostHttpTimeout = vhostHttpTimeout
-	g.GlbServerCfg.DashboardAddr = dashboardAddr
-	g.GlbServerCfg.DashboardPort = dashboardPort
-	g.GlbServerCfg.DashboardUser = dashboardUser
-	g.GlbServerCfg.DashboardPwd = dashboardPwd
-	g.GlbServerCfg.LogFile = logFile
-	g.GlbServerCfg.LogLevel = logLevel
-	g.GlbServerCfg.LogMaxDays = logMaxDays
-	g.GlbServerCfg.Token = token
-	g.GlbServerCfg.SubDomainHost = subDomainHost
+func parseServerCommonCfgFromCmd() (cfg config.ServerCommonConf, err error) {
+	cfg = config.GetDefaultServerConf()
+
+	cfg.BindAddr = bindAddr
+	cfg.BindPort = bindPort
+	cfg.BindUdpPort = bindUdpPort
+	cfg.KcpBindPort = kcpBindPort
+	cfg.ProxyBindAddr = proxyBindAddr
+	cfg.VhostHttpPort = vhostHttpPort
+	cfg.VhostHttpsPort = vhostHttpsPort
+	cfg.VhostHttpTimeout = vhostHttpTimeout
+	cfg.DashboardAddr = dashboardAddr
+	cfg.DashboardPort = dashboardPort
+	cfg.DashboardUser = dashboardUser
+	cfg.DashboardPwd = dashboardPwd
+	cfg.LogFile = logFile
+	cfg.LogLevel = logLevel
+	cfg.LogMaxDays = logMaxDays
+	cfg.Token = token
+	cfg.SubDomainHost = subDomainHost
 	if len(allowPorts) > 0 {
 		// e.g. 1000-2000,2001,2002,3000-4000
 		ports, errRet := util.ParseRangeNumbers(allowPorts)
@@ -181,28 +182,27 @@ func parseServerCommonCfgFromCmd() (err error) {
 		}
 
 		for _, port := range ports {
-			g.GlbServerCfg.AllowPorts[int(port)] = struct{}{}
+			cfg.AllowPorts[int(port)] = struct{}{}
 		}
 	}
-	g.GlbServerCfg.MaxPortsPerClient = maxPortsPerClient
+	cfg.MaxPortsPerClient = maxPortsPerClient
 
 	if logFile == "console" {
-		g.GlbServerCfg.LogWay = "console"
+		cfg.LogWay = "console"
 	} else {
-		g.GlbServerCfg.LogWay = "file"
+		cfg.LogWay = "file"
 	}
+	cfg.DisableLogColor = disableLogColor
 	return
 }
 
-func runServer() (err error) {
-	log.InitLog(g.GlbServerCfg.LogWay, g.GlbServerCfg.LogFile, g.GlbServerCfg.LogLevel,
-		g.GlbServerCfg.LogMaxDays)
-	svr, err := server.NewService()
+func runServer(cfg config.ServerCommonConf) (err error) {
+	log.InitLog(cfg.LogWay, cfg.LogFile, cfg.LogLevel, cfg.LogMaxDays, cfg.DisableLogColor)
+	svr, err := server.NewService(cfg)
 	if err != nil {
 		return err
 	}
-	log.Info("Start frps success")
-	server.ServerService = svr
+	log.Info("start frps success")
 	svr.Run()
 	return
 }

conf/frpc_full.ini

@@ -18,6 +18,9 @@ log_level = info
 
 log_max_days = 3
 
+# disable log colors when log_file is console, default is false
+disable_log_color = false
+
 # for authentication
 token = 12345678
 
@@ -26,6 +29,8 @@ admin_addr = 127.0.0.1
 admin_port = 7400
 admin_user = admin
 admin_pwd = admin
+# Admin assets directory. By default, these assets are bundled with frpc.
+# assets_dir = ./static
 
 # connections will be established in advance, default value is zero
 pool_count = 5
@@ -50,7 +55,7 @@ tls_enable = true
 # specify a dns server, so frpc will use this instead of default one
 # dns_server = 8.8.8.8
 
-# proxy names you want to start divided by ','
+# proxy names you want to start seperated by ','
 # default is empty, means all proxies
 # start = ssh,dns
 
@@ -66,6 +71,8 @@ tls_enable = true
 type = tcp
 local_ip = 127.0.0.1
 local_port = 22
+# limit bandwith for this proxy, unit is KB and MB
+bandwith_limit = 1MB
 # true or false, if true, messages between frps and frpc will be encrypted, default is false
 use_encryption = false
 # if true, message will be compressed
@@ -198,6 +205,15 @@ plugin_local_addr = 127.0.0.1:80
 plugin_crt_path = ./server.crt
 plugin_key_path = ./server.key
 plugin_host_header_rewrite = 127.0.0.1
+plugin_header_X-From-Where = frp
+
+[plugin_http2https]
+type = http
+custom_domains = test.yourdomain.com
+plugin = http2https
+plugin_local_addr = 127.0.0.1:443
+plugin_host_header_rewrite = 127.0.0.1
+plugin_header_X-From-Where = frp
 
 [secret_tcp]
 # If the type is secret tcp, remote_port is useless

conf/frps_full.ini

@@ -43,6 +43,9 @@ log_level = info
 
 log_max_days = 3
 
+# disable log colors when log_file is console, default is false
+disable_log_color = false
+
 # auth token
 token = 12345678
 

g/g.go

@@ -1,32 +0,0 @@
-package g
-
-import (
-	"github.com/fatedier/frp/models/config"
-)
-
-var (
-	GlbClientCfg *ClientCfg
-	GlbServerCfg *ServerCfg
-)
-
-func init() {
-	GlbClientCfg = &ClientCfg{
-		ClientCommonConf: *config.GetDefaultClientConf(),
-	}
-	GlbServerCfg = &ServerCfg{
-		ServerCommonConf: *config.GetDefaultServerConf(),
-	}
-}
-
-type ClientCfg struct {
-	config.ClientCommonConf
-
-	CfgFile       string
-	ServerUdpPort int // this is configured by login response from frps
-}
-
-type ServerCfg struct {
-	config.ServerCommonConf
-
-	CfgFile string
-}

go.mod

@@ -4,14 +4,12 @@ go 1.12
 
 require (
 	github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5
-	github.com/davecgh/go-spew v1.1.0 // indirect
 	github.com/fatedier/beego v0.0.0-20171024143340-6c6a4f5bd5eb
 	github.com/fatedier/golib v0.0.0-20181107124048-ff8cd814b049
-	github.com/fatedier/kcp-go v2.0.4-0.20190317085623-2063a803e6fe+incompatible
+	github.com/fatedier/kcp-go v2.0.4-0.20190803094908-fe8645b0a904+incompatible
 	github.com/golang/snappy v0.0.0-20170215233205-553a64147049 // indirect
-	github.com/gorilla/context v1.1.1 // indirect
-	github.com/gorilla/mux v1.6.2
-	github.com/gorilla/websocket v1.2.0
+	github.com/gorilla/mux v1.7.3
+	github.com/gorilla/websocket v1.4.0
 	github.com/hashicorp/yamux v0.0.0-20181012175058-2f1d1f20f75d
 	github.com/inconshreveable/mousetrap v1.0.0 // indirect
 	github.com/klauspost/cpuid v1.2.0 // indirect
@@ -19,16 +17,17 @@ require (
 	github.com/mattn/go-runewidth v0.0.4 // indirect
 	github.com/pires/go-proxyproto v0.0.0-20190111085350-4d51b51e3bfc
 	github.com/pkg/errors v0.8.0 // indirect
-	github.com/pmezard/go-difflib v1.0.0 // indirect
 	github.com/rakyll/statik v0.1.1
 	github.com/rodaine/table v1.0.0
 	github.com/spf13/cobra v0.0.3
 	github.com/spf13/pflag v1.0.1 // indirect
-	github.com/stretchr/testify v1.2.1
+	github.com/stretchr/testify v1.3.0
 	github.com/templexxx/cpufeat v0.0.0-20170927014610-3794dfbfb047 // indirect
 	github.com/templexxx/xor v0.0.0-20170926022130-0af8e873c554 // indirect
 	github.com/tjfoc/gmsm v0.0.0-20171124023159-98aa888b79d8 // indirect
 	github.com/vaughan0/go-ini v0.0.0-20130923145212-a98ad7ee00ec
-	golang.org/x/crypto v0.0.0-20180505025534-4ec37c66abab // indirect
-	golang.org/x/net v0.0.0-20180524181706-dfa909b99c79
+	github.com/xtaci/lossyconn v0.0.0-20190602105132-8df528c0c9ae // indirect
+	golang.org/x/net v0.0.0-20190724013045-ca1201d0de80
+	golang.org/x/text v0.3.2 // indirect
+	golang.org/x/time v0.0.0-20191024005414-555d28b269f0
 )

go.sum

@@ -3,13 +3,13 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
 github.com/fatedier/beego v0.0.0-20171024143340-6c6a4f5bd5eb/go.mod h1:wx3gB6dbIfBRcucp94PI9Bt3I0F2c/MyNEWuhzpWiwk=
 github.com/fatedier/golib v0.0.0-20181107124048-ff8cd814b049 h1:teH578mf2ii42NHhIp3PhgvjU5bv+NFMq9fSQR8NaG8=
 github.com/fatedier/golib v0.0.0-20181107124048-ff8cd814b049/go.mod h1:DqIrnl0rp3Zybg9zbJmozTy1n8fYJoX+QoAj9slIkKM=
-github.com/fatedier/kcp-go v2.0.4-0.20190317085623-2063a803e6fe+incompatible h1:pNNeBKz1jtMDupiwvtEGFTujA3J86xoEXGSkwVeYFsw=
-github.com/fatedier/kcp-go v2.0.4-0.20190317085623-2063a803e6fe+incompatible/go.mod h1:YpCOaxj7vvMThhIQ9AfTOPW2sfztQR5WDfs7AflSy4s=
+github.com/fatedier/kcp-go v2.0.4-0.20190803094908-fe8645b0a904+incompatible h1:ssXat9YXFvigNge/IkkZvFMn8yeYKFX+uI6wn2mLJ74=
+github.com/fatedier/kcp-go v2.0.4-0.20190803094908-fe8645b0a904+incompatible/go.mod h1:YpCOaxj7vvMThhIQ9AfTOPW2sfztQR5WDfs7AflSy4s=
 github.com/golang/snappy v0.0.0-20170215233205-553a64147049/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
-github.com/gorilla/context v1.1.1 h1:AWwleXJkX/nhcU9bZSnZoi3h/qGYqQAGhq6zZe/aQW8=
-github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg=
-github.com/gorilla/mux v1.6.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
-github.com/gorilla/websocket v1.2.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
+github.com/gorilla/mux v1.7.3 h1:gnP5JzjVOuiZD07fKKToCAOjS0yOpj/qPETTXCCS6hw=
+github.com/gorilla/mux v1.7.3/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
+github.com/gorilla/websocket v1.4.0 h1:WDFjx/TMzVgy9VdMMQi2K2Emtwi2QcUQsztZ/zLaH/Q=
+github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
 github.com/hashicorp/yamux v0.0.0-20181012175058-2f1d1f20f75d h1:kJCB4vdITiW1eC1vq2e6IsrXKrZit1bv/TDYFGMp4BQ=
 github.com/hashicorp/yamux v0.0.0-20181012175058-2f1d1f20f75d/go.mod h1:+NfK9FKeTrX5uv1uIXGdwYDTeHna2qgaIlx54MXqjAM=
 github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
@@ -28,10 +28,24 @@ github.com/rakyll/statik v0.1.1/go.mod h1:OEi9wJV/fMUAGx1eNjq75DKDsJVuEv1U0oYdX6
 github.com/rodaine/table v1.0.0/go.mod h1:YAUzwPOji0DUJNEvggdxyQcUAl4g3hDRcFlyjnnR51I=
 github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
 github.com/spf13/pflag v1.0.1/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
-github.com/stretchr/testify v1.2.1/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
+github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
 github.com/templexxx/cpufeat v0.0.0-20170927014610-3794dfbfb047/go.mod h1:wM7WEvslTq+iOEAMDLSzhVuOt5BRZ05WirO+b09GHQU=
 github.com/templexxx/xor v0.0.0-20170926022130-0af8e873c554/go.mod h1:5XA7W9S6mni3h5uvOC75dA3m9CCCaS83lltmc0ukdi4=
 github.com/tjfoc/gmsm v0.0.0-20171124023159-98aa888b79d8/go.mod h1:XxO4hdhhrzAd+G4CjDqaOkd0hUzmtPR/d3EiBBMn/wc=
 github.com/vaughan0/go-ini v0.0.0-20130923145212-a98ad7ee00ec/go.mod h1:owBmyHYMLkxyrugmfwE/DLJyW8Ro9mkphwuVErQ0iUw=
-golang.org/x/crypto v0.0.0-20180505025534-4ec37c66abab/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
-golang.org/x/net v0.0.0-20180524181706-dfa909b99c79/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+github.com/xtaci/lossyconn v0.0.0-20190602105132-8df528c0c9ae h1:J0GxkO96kL4WF+AIT3M4mfUVinOCPgf2uUWYFUzN0sM=
+github.com/xtaci/lossyconn v0.0.0-20190602105132-8df528c0c9ae/go.mod h1:gXtu8J62kEgmN++bm9BVICuT/e8yiLI2KFobd/TRFsE=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2 h1:VklqNMn3ovrHsnt90PveolxSbWFaJdECFbxSq0Mqo2M=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/net v0.0.0-20190724013045-ca1201d0de80 h1:Ao/3l156eZf2AW5wK8a7/smtodRU+gha3+BeqJ69lRk=
+golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a h1:1BGLXjeY4akVXGgbC9HugT3Jv3hCI0z56oJR5vAMgBU=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
+golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
+golang.org/x/time v0.0.0-20191024005414-555d28b269f0 h1:/5xXl8Y5W96D+TtHSlonuFqGHIWVuyCkGJLwGh9JJFs=
+golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=

models/config/client_common.go

@@ -23,34 +23,103 @@ import (
 	ini "github.com/vaughan0/go-ini"
 )
 
-// client common config
+// ClientCommonConf contains information for a client service. It is
+// recommended to use GetDefaultClientConf instead of creating this object
+// directly, so that all unspecified fields have reasonable default values.
 type ClientCommonConf struct {
-	ServerAddr        string              `json:"server_addr"`
-	ServerPort        int                 `json:"server_port"`
-	HttpProxy         string              `json:"http_proxy"`
-	LogFile           string              `json:"log_file"`
-	LogWay            string              `json:"log_way"`
-	LogLevel          string              `json:"log_level"`
-	LogMaxDays        int64               `json:"log_max_days"`
-	Token             string              `json:"token"`
-	AdminAddr         string              `json:"admin_addr"`
-	AdminPort         int                 `json:"admin_port"`
-	AdminUser         string              `json:"admin_user"`
-	AdminPwd          string              `json:"admin_pwd"`
-	PoolCount         int                 `json:"pool_count"`
-	TcpMux            bool                `json:"tcp_mux"`
-	User              string              `json:"user"`
-	DnsServer         string              `json:"dns_server"`
-	LoginFailExit     bool                `json:"login_fail_exit"`
-	Start             map[string]struct{} `json:"start"`
-	Protocol          string              `json:"protocol"`
-	TLSEnable         bool                `json:"tls_enable"`
-	HeartBeatInterval int64               `json:"heartbeat_interval"`
-	HeartBeatTimeout  int64               `json:"heartbeat_timeout"`
+	// ServerAddr specifies the address of the server to connect to. By
+	// default, this value is "0.0.0.0".
+	ServerAddr string `json:"server_addr"`
+	// ServerPort specifies the port to connect to the server on. By default,
+	// this value is 7000.
+	ServerPort int `json:"server_port"`
+	// HttpProxy specifies a proxy address to connect to the server through. If
+	// this value is "", the server will be connected to directly. By default,
+	// this value is read from the "http_proxy" environment variable.
+	HttpProxy string `json:"http_proxy"`
+	// LogFile specifies a file where logs will be written to. This value will
+	// only be used if LogWay is set appropriately. By default, this value is
+	// "console".
+	LogFile string `json:"log_file"`
+	// LogWay specifies the way logging is managed. Valid values are "console"
+	// or "file". If "console" is used, logs will be printed to stdout. If
+	// "file" is used, logs will be printed to LogFile. By default, this value
+	// is "console".
+	LogWay string `json:"log_way"`
+	// LogLevel specifies the minimum log level. Valid values are "trace",
+	// "debug", "info", "warn", and "error". By default, this value is "info".
+	LogLevel string `json:"log_level"`
+	// LogMaxDays specifies the maximum number of days to store log information
+	// before deletion. This is only used if LogWay == "file". By default, this
+	// value is 0.
+	LogMaxDays int64 `json:"log_max_days"`
+	// DisableLogColor disables log colors when LogWay == "console" when set to
+	// true. By default, this value is false.
+	DisableLogColor bool `json:"disable_log_color"`
+	// Token specifies the authorization token used to create keys to be sent
+	// to the server. The server must have a matching token for authorization
+	// to succeed.  By default, this value is "".
+	Token string `json:"token"`
+	// AdminAddr specifies the address that the admin server binds to. By
+	// default, this value is "127.0.0.1".
+	AdminAddr string `json:"admin_addr"`
+	// AdminPort specifies the port for the admin server to listen on. If this
+	// value is 0, the admin server will not be started. By default, this value
+	// is 0.
+	AdminPort int `json:"admin_port"`
+	// AdminUser specifies the username that the admin server will use for
+	// login. By default, this value is "admin".
+	AdminUser string `json:"admin_user"`
+	// AdminPwd specifies the password that the admin server will use for
+	// login. By default, this value is "admin".
+	AdminPwd string `json:"admin_pwd"`
+	// AssetsDir specifies the local directory that the admin server will load
+	// resources from. If this value is "", assets will be loaded from the
+	// bundled executable using statik. By default, this value is "".
+	AssetsDir string `json:"assets_dir"`
+	// PoolCount specifies the number of connections the client will make to
+	// the server in advance. By default, this value is 0.
+	PoolCount int `json:"pool_count"`
+	// TcpMux toggles TCP stream multiplexing. This allows multiple requests
+	// from a client to share a single TCP connection. If this value is true,
+	// the server must have TCP multiplexing enabled as well. By default, this
+	// value is true.
+	TcpMux bool `json:"tcp_mux"`
+	// User specifies a prefix for proxy names to distinguish them from other
+	// clients. If this value is not "", proxy names will automatically be
+	// changed to "{user}.{proxy_name}". By default, this value is "".
+	User string `json:"user"`
+	// DnsServer specifies a DNS server address for FRPC to use. If this value
+	// is "", the default DNS will be used. By default, this value is "".
+	DnsServer string `json:"dns_server"`
+	// LoginFailExit controls whether or not the client should exit after a
+	// failed login attempt. If false, the client will retry until a login
+	// attempt succeeds. By default, this value is true.
+	LoginFailExit bool `json:"login_fail_exit"`
+	// Start specifies a set of enabled proxies by name. If this set is empty,
+	// all supplied proxies are enabled. By default, this value is an empty
+	// set.
+	Start map[string]struct{} `json:"start"`
+	// Protocol specifies the protocol to use when interacting with the server.
+	// Valid values are "tcp", "kcp", and "websocket". By default, this value
+	// is "tcp".
+	Protocol string `json:"protocol"`
+	// TLSEnable specifies whether or not TLS should be used when communicating
+	// with the server.
+	TLSEnable bool `json:"tls_enable"`
+	// HeartBeatInterval specifies at what interval heartbeats are sent to the
+	// server, in seconds. It is not recommended to change this value. By
+	// default, this value is 30.
+	HeartBeatInterval int64 `json:"heartbeat_interval"`
+	// HeartBeatTimeout specifies the maximum allowed heartbeat response delay
+	// before the connection is terminated, in seconds. It is not recommended
+	// to change this value. By default, this value is 90.
+	HeartBeatTimeout int64 `json:"heartbeat_timeout"`
 }
 
-func GetDefaultClientConf() *ClientCommonConf {
-	return &ClientCommonConf{
+// GetDefaultClientConf returns a client configuration with default values.
+func GetDefaultClientConf() ClientCommonConf {
+	return ClientCommonConf{
 		ServerAddr:        "0.0.0.0",
 		ServerPort:        7000,
 		HttpProxy:         os.Getenv("http_proxy"),
@@ -58,11 +127,13 @@ func GetDefaultClientConf() *ClientCommonConf {
 		LogWay:            "console",
 		LogLevel:          "info",
 		LogMaxDays:        3,
+		DisableLogColor:   false,
 		Token:             "",
 		AdminAddr:         "127.0.0.1",
 		AdminPort:         0,
 		AdminUser:         "",
 		AdminPwd:          "",
+		AssetsDir:         "",
 		PoolCount:         1,
 		TcpMux:            true,
 		User:              "",
@@ -76,16 +147,12 @@ func GetDefaultClientConf() *ClientCommonConf {
 	}
 }
 
-func UnmarshalClientConfFromIni(defaultCfg *ClientCommonConf, content string) (cfg *ClientCommonConf, err error) {
-	cfg = defaultCfg
-	if cfg == nil {
-		cfg = GetDefaultClientConf()
-	}
+func UnmarshalClientConfFromIni(content string) (cfg ClientCommonConf, err error) {
+	cfg = GetDefaultClientConf()
 
 	conf, err := ini.Load(strings.NewReader(content))
 	if err != nil {
-		err = fmt.Errorf("parse ini conf file error: %v", err)
-		return nil, err
+		return ClientCommonConf{}, fmt.Errorf("parse ini conf file error: %v", err)
 	}
 
 	var (
@@ -106,6 +173,10 @@ func UnmarshalClientConfFromIni(defaultCfg *ClientCommonConf, content string) (c
 		cfg.ServerPort = int(v)
 	}
 
+	if tmpStr, ok = conf.Get("common", "disable_log_color"); ok && tmpStr == "true" {
+		cfg.DisableLogColor = true
+	}
+
 	if tmpStr, ok = conf.Get("common", "http_proxy"); ok {
 		cfg.HttpProxy = tmpStr
 	}
@@ -154,6 +225,10 @@ func UnmarshalClientConfFromIni(defaultCfg *ClientCommonConf, content string) (c
 		cfg.AdminPwd = tmpStr
 	}
 
+	if tmpStr, ok = conf.Get("common", "assets_dir"); ok {
+		cfg.AssetsDir = tmpStr
+	}
+
 	if tmpStr, ok = conf.Get("common", "pool_count"); ok {
 		if v, err = strconv.ParseInt(tmpStr, 10, 64); err == nil {
 			cfg.PoolCount = int(v)

models/config/proxy.go

@@ -58,11 +58,11 @@ type ProxyConf interface {
 	UnmarshalFromIni(prefix string, name string, conf ini.Section) error
 	MarshalToMsg(pMsg *msg.NewProxy)
 	CheckForCli() error
-	CheckForSvr() error
+	CheckForSvr(serverCfg ServerCommonConf) error
 	Compare(conf ProxyConf) bool
 }
 
-func NewProxyConfFromMsg(pMsg *msg.NewProxy) (cfg ProxyConf, err error) {
+func NewProxyConfFromMsg(pMsg *msg.NewProxy, serverCfg ServerCommonConf) (cfg ProxyConf, err error) {
 	if pMsg.ProxyType == "" {
 		pMsg.ProxyType = consts.TcpProxy
 	}
@@ -73,7 +73,7 @@ func NewProxyConfFromMsg(pMsg *msg.NewProxy) (cfg ProxyConf, err error) {
 		return
 	}
 	cfg.UnmarshalFromMsg(pMsg)
-	err = cfg.CheckForSvr()
+	err = cfg.CheckForSvr(serverCfg)
 	return
 }
 
@@ -97,18 +97,39 @@ func NewProxyConfFromIni(prefix string, name string, section ini.Section) (cfg P
 	return
 }
 
-// BaseProxy info
+// BaseProxyConf provides configuration info that is common to all proxy types.
 type BaseProxyConf struct {
+	// ProxyName is the name of this proxy.
 	ProxyName string `json:"proxy_name"`
+	// ProxyType specifies the type of this proxy. Valid values include "tcp",
+	// "udp", "http", "https", "stcp", and "xtcp". By default, this value is
+	// "tcp".
 	ProxyType string `json:"proxy_type"`
 
-	UseEncryption  bool   `json:"use_encryption"`
-	UseCompression bool   `json:"use_compression"`
-	Group          string `json:"group"`
-	GroupKey       string `json:"group_key"`
+	// UseEncryption controls whether or not communication with the server will
+	// be encrypted. Encryption is done using the tokens supplied in the server
+	// and client configuration. By default, this value is false.
+	UseEncryption bool `json:"use_encryption"`
+	// UseCompression controls whether or not communication with the server
+	// will be compressed. By default, this value is false.
+	UseCompression bool `json:"use_compression"`
+	// Group specifies which group the proxy is a part of. The server will use
+	// this information to load balance proxies in the same group. If the value
+	// is "", this proxy will not be in a group. By default, this value is "".
+	Group string `json:"group"`
+	// GroupKey specifies a group key, which should be the same among proxies
+	// of the same group. By default, this value is "".
+	GroupKey string `json:"group_key"`
 
-	// only used for client
+	// ProxyProtocolVersion specifies which protocol version to use. Valid
+	// values include "v1", "v2", and "". If the value is "", a protocol
+	// version will be automatically selected. By default, this value is "".
 	ProxyProtocolVersion string `json:"proxy_protocol_version"`
+
+	// BandwidthLimit limit the proxy bandwidth
+	// 0 means no limit
+	BandwidthLimit BandwidthQuantity `json:"bandwidth_limit"`
+
 	LocalSvrConf
 	HealthCheckConf
 }
@@ -124,7 +145,8 @@ func (cfg *BaseProxyConf) compare(cmp *BaseProxyConf) bool {
 		cfg.UseCompression != cmp.UseCompression ||
 		cfg.Group != cmp.Group ||
 		cfg.GroupKey != cmp.GroupKey ||
-		cfg.ProxyProtocolVersion != cmp.ProxyProtocolVersion {
+		cfg.ProxyProtocolVersion != cmp.ProxyProtocolVersion ||
+		cfg.BandwidthLimit.Equal(&cmp.BandwidthLimit) {
 		return false
 	}
 	if !cfg.LocalSvrConf.compare(&cmp.LocalSvrConf) {
@@ -149,6 +171,7 @@ func (cfg *BaseProxyConf) UnmarshalFromIni(prefix string, name string, section i
 	var (
 		tmpStr string
 		ok     bool
+		err    error
 	)
 	cfg.ProxyName = prefix + name
 	cfg.ProxyType = section["type"]
@@ -167,11 +190,15 @@ func (cfg *BaseProxyConf) UnmarshalFromIni(prefix string, name string, section i
 	cfg.GroupKey = section["group_key"]
 	cfg.ProxyProtocolVersion = section["proxy_protocol_version"]
 
-	if err := cfg.LocalSvrConf.UnmarshalFromIni(prefix, name, section); err != nil {
+	if cfg.BandwidthLimit, err = NewBandwidthQuantity(section["bandwidth_limit"]); err != nil {
 		return err
 	}
 
-	if err := cfg.HealthCheckConf.UnmarshalFromIni(prefix, name, section); err != nil {
+	if err = cfg.LocalSvrConf.UnmarshalFromIni(prefix, name, section); err != nil {
+		return err
+	}
+
+	if err = cfg.HealthCheckConf.UnmarshalFromIni(prefix, name, section); err != nil {
 		return err
 	}
 
@@ -308,21 +335,21 @@ func (cfg *DomainConf) checkForCli() (err error) {
 	return
 }
 
-func (cfg *DomainConf) checkForSvr() (err error) {
+func (cfg *DomainConf) checkForSvr(serverCfg ServerCommonConf) (err error) {
 	if err = cfg.check(); err != nil {
 		return
 	}
 
 	for _, domain := range cfg.CustomDomains {
-		if subDomainHost != "" && len(strings.Split(subDomainHost, ".")) < len(strings.Split(domain, ".")) {
-			if strings.Contains(domain, subDomainHost) {
-				return fmt.Errorf("custom domain [%s] should not belong to subdomain_host [%s]", domain, subDomainHost)
+		if serverCfg.SubDomainHost != "" && len(strings.Split(serverCfg.SubDomainHost, ".")) < len(strings.Split(domain, ".")) {
+			if strings.Contains(domain, serverCfg.SubDomainHost) {
+				return fmt.Errorf("custom domain [%s] should not belong to subdomain_host [%s]", domain, serverCfg.SubDomainHost)
 			}
 		}
 	}
 
 	if cfg.SubDomain != "" {
-		if subDomainHost == "" {
+		if serverCfg.SubDomainHost == "" {
 			return fmt.Errorf("subdomain is not supported because this feature is not enabled in remote frps")
 		}
 		if strings.Contains(cfg.SubDomain, ".") || strings.Contains(cfg.SubDomain, "*") {
@@ -332,12 +359,20 @@ func (cfg *DomainConf) checkForSvr() (err error) {
 	return
 }
 
-// Local service info
+// LocalSvrConf configures what location the client will proxy to, or what
+// plugin will be used.
 type LocalSvrConf struct {
-	LocalIp   string `json:"local_ip"`
-	LocalPort int    `json:"local_port"`
+	// LocalIp specifies the IP address or host name to proxy to.
+	LocalIp string `json:"local_ip"`
+	// LocalPort specifies the port to proxy to.
+	LocalPort int `json:"local_port"`
 
-	Plugin       string            `json:"plugin"`
+	// Plugin specifies what plugin should be used for proxying. If this value
+	// is set, the LocalIp and LocalPort values will be ignored. By default,
+	// this value is "".
+	Plugin string `json:"plugin"`
+	// PluginParams specify parameters to be passed to the plugin, if one is
+	// being used. By default, this value is an empty map.
 	PluginParams map[string]string `json:"plugin_params"`
 }
 
@@ -399,15 +434,35 @@ func (cfg *LocalSvrConf) checkForCli() (err error) {
 	return
 }
 
-// Health check info
+// HealthCheckConf configures health checking. This can be useful for load
+// balancing purposes to detect and remove proxies to failing services.
 type HealthCheckConf struct {
-	HealthCheckType      string `json:"health_check_type"` // tcp | http
-	HealthCheckTimeoutS  int    `json:"health_check_timeout_s"`
-	HealthCheckMaxFailed int    `json:"health_check_max_failed"`
-	HealthCheckIntervalS int    `json:"health_check_interval_s"`
-	HealthCheckUrl       string `json:"health_check_url"`
-
-	// local_ip + local_port
+	// HealthCheckType specifies what protocol to use for health checking.
+	// Valid values include "tcp", "http", and "". If this value is "", health
+	// checking will not be performed. By default, this value is "".
+	//
+	// If the type is "tcp", a connection will be attempted to the target
+	// server. If a connection cannot be established, the health check fails.
+	//
+	// If the type is "http", a GET request will be made to the endpoint
+	// specified by HealthCheckUrl. If the response is not a 200, the health
+	// check fails.
+	HealthCheckType string `json:"health_check_type"` // tcp | http
+	// HealthCheckTimeoutS specifies the number of seconds to wait for a health
+	// check attempt to connect. If the timeout is reached, this counts as a
+	// health check failure. By default, this value is 3.
+	HealthCheckTimeoutS int `json:"health_check_timeout_s"`
+	// HealthCheckMaxFailed specifies the number of allowed failures before the
+	// proxy is stopped. By default, this value is 1.
+	HealthCheckMaxFailed int `json:"health_check_max_failed"`
+	// HealthCheckIntervalS specifies the time in seconds between health
+	// checks. By default, this value is 10.
+	HealthCheckIntervalS int `json:"health_check_interval_s"`
+	// HealthCheckUrl specifies the address to send health checks to if the
+	// health check type is "http".
+	HealthCheckUrl string `json:"health_check_url"`
+	// HealthCheckAddr specifies the address to connect to if the health check
+	// type is "tcp".
 	HealthCheckAddr string `json:"-"`
 }
 
@@ -504,7 +559,7 @@ func (cfg *TcpProxyConf) CheckForCli() (err error) {
 	return
 }
 
-func (cfg *TcpProxyConf) CheckForSvr() error { return nil }
+func (cfg *TcpProxyConf) CheckForSvr(serverCfg ServerCommonConf) error { return nil }
 
 // UDP
 type UdpProxyConf struct {
@@ -552,7 +607,7 @@ func (cfg *UdpProxyConf) CheckForCli() (err error) {
 	return
 }
 
-func (cfg *UdpProxyConf) CheckForSvr() error { return nil }
+func (cfg *UdpProxyConf) CheckForSvr(serverCfg ServerCommonConf) error { return nil }
 
 // HTTP
 type HttpProxyConf struct {
@@ -657,11 +712,11 @@ func (cfg *HttpProxyConf) CheckForCli() (err error) {
 	return
 }
 
-func (cfg *HttpProxyConf) CheckForSvr() (err error) {
-	if vhostHttpPort == 0 {
+func (cfg *HttpProxyConf) CheckForSvr(serverCfg ServerCommonConf) (err error) {
+	if serverCfg.VhostHttpPort == 0 {
 		return fmt.Errorf("type [http] not support when vhost_http_port is not set")
 	}
-	if err = cfg.DomainConf.checkForSvr(); err != nil {
+	if err = cfg.DomainConf.checkForSvr(serverCfg); err != nil {
 		err = fmt.Errorf("proxy [%s] domain conf check error: %v", cfg.ProxyName, err)
 		return
 	}
@@ -717,11 +772,11 @@ func (cfg *HttpsProxyConf) CheckForCli() (err error) {
 	return
 }
 
-func (cfg *HttpsProxyConf) CheckForSvr() (err error) {
-	if vhostHttpsPort == 0 {
+func (cfg *HttpsProxyConf) CheckForSvr(serverCfg ServerCommonConf) (err error) {
+	if serverCfg.VhostHttpsPort == 0 {
 		return fmt.Errorf("type [https] not support when vhost_https_port is not set")
 	}
-	if err = cfg.DomainConf.checkForSvr(); err != nil {
+	if err = cfg.DomainConf.checkForSvr(serverCfg); err != nil {
 		err = fmt.Errorf("proxy [%s] domain conf check error: %v", cfg.ProxyName, err)
 		return
 	}
@@ -790,7 +845,7 @@ func (cfg *StcpProxyConf) CheckForCli() (err error) {
 	return
 }
 
-func (cfg *StcpProxyConf) CheckForSvr() (err error) {
+func (cfg *StcpProxyConf) CheckForSvr(serverCfg ServerCommonConf) (err error) {
 	return
 }
 
@@ -857,7 +912,7 @@ func (cfg *XtcpProxyConf) CheckForCli() (err error) {
 	return
 }
 
-func (cfg *XtcpProxyConf) CheckForSvr() (err error) {
+func (cfg *XtcpProxyConf) CheckForSvr(serverCfg ServerCommonConf) (err error) {
 	return
 }
 

models/config/server_common.go

@@ -24,62 +24,122 @@ import (
 	"github.com/fatedier/frp/utils/util"
 )
 
-var (
-	// server global configure used for generate proxy conf used in frps
-	proxyBindAddr  string
-	subDomainHost  string
-	vhostHttpPort  int
-	vhostHttpsPort int
-)
-
-func InitServerCfg(cfg *ServerCommonConf) {
-	proxyBindAddr = cfg.ProxyBindAddr
-	subDomainHost = cfg.SubDomainHost
-	vhostHttpPort = cfg.VhostHttpPort
-	vhostHttpsPort = cfg.VhostHttpsPort
-}
-
-// common config
+// ServerCommonConf contains information for a server service. It is
+// recommended to use GetDefaultServerConf instead of creating this object
+// directly, so that all unspecified fields have reasonable default values.
 type ServerCommonConf struct {
-	BindAddr      string `json:"bind_addr"`
-	BindPort      int    `json:"bind_port"`
-	BindUdpPort   int    `json:"bind_udp_port"`
-	KcpBindPort   int    `json:"kcp_bind_port"`
+	// BindAddr specifies the address that the server binds to. By default,
+	// this value is "0.0.0.0".
+	BindAddr string `json:"bind_addr"`
+	// BindPort specifies the port that the server listens on. By default, this
+	// value is 7000.
+	BindPort int `json:"bind_port"`
+	// BindUdpPort specifies the UDP port that the server listens on. If this
+	// value is 0, the server will not listen for UDP connections. By default,
+	// this value is 0
+	BindUdpPort int `json:"bind_udp_port"`
+	// BindKcpPort specifies the KCP port that the server listens on. If this
+	// value is 0, the server will not listen for KCP connections. By default,
+	// this value is 0.
+	KcpBindPort int `json:"kcp_bind_port"`
+	// ProxyBindAddr specifies the address that the proxy binds to. This value
+	// may be the same as BindAddr. By default, this value is "0.0.0.0".
 	ProxyBindAddr string `json:"proxy_bind_addr"`
 
-	// If VhostHttpPort equals 0, don't listen a public port for http protocol.
+	// VhostHttpPort specifies the port that the server listens for HTTP Vhost
+	// requests. If this value is 0, the server will not listen for HTTP
+	// requests. By default, this value is 0.
 	VhostHttpPort int `json:"vhost_http_port"`
 
-	// if VhostHttpsPort equals 0, don't listen a public port for https protocol
+	// VhostHttpsPort specifies the port that the server listens for HTTPS
+	// Vhost requests. If this value is 0, the server will not listen for HTTPS
+	// requests. By default, this value is 0.
 	VhostHttpsPort int `json:"vhost_https_port"`
 
+	// VhostHttpTimeout specifies the response header timeout for the Vhost
+	// HTTP server, in seconds. By default, this value is 60.
 	VhostHttpTimeout int64 `json:"vhost_http_timeout"`
 
+	// DashboardAddr specifies the address that the dashboard binds to. By
+	// default, this value is "0.0.0.0".
 	DashboardAddr string `json:"dashboard_addr"`
 
-	// if DashboardPort equals 0, dashboard is not available
-	DashboardPort int    `json:"dashboard_port"`
+	// DashboardPort specifies the port that the dashboard listens on. If this
+	// value is 0, the dashboard will not be started. By default, this value is
+	// 0.
+	DashboardPort int `json:"dashboard_port"`
+	// DashboardUser specifies the username that the dashboard will use for
+	// login. By default, this value is "admin".
 	DashboardUser string `json:"dashboard_user"`
-	DashboardPwd  string `json:"dashboard_pwd"`
-	AssetsDir     string `json:"asserts_dir"`
-	LogFile       string `json:"log_file"`
-	LogWay        string `json:"log_way"` // console or file
-	LogLevel      string `json:"log_level"`
-	LogMaxDays    int64  `json:"log_max_days"`
-	Token         string `json:"token"`
+	// DashboardUser specifies the password that the dashboard will use for
+	// login. By default, this value is "admin".
+	DashboardPwd string `json:"dashboard_pwd"`
+	// AssetsDir specifies the local directory that the dashboard will load
+	// resources from. If this value is "", assets will be loaded from the
+	// bundled executable using statik. By default, this value is "".
+	AssetsDir string `json:"asserts_dir"`
+	// LogFile specifies a file where logs will be written to. This value will
+	// only be used if LogWay is set appropriately. By default, this value is
+	// "console".
+	LogFile string `json:"log_file"`
+	// LogWay specifies the way logging is managed. Valid values are "console"
+	// or "file". If "console" is used, logs will be printed to stdout. If
+	// "file" is used, logs will be printed to LogFile. By default, this value
+	// is "console".
+	LogWay string `json:"log_way"`
+	// LogLevel specifies the minimum log level. Valid values are "trace",
+	// "debug", "info", "warn", and "error". By default, this value is "info".
+	LogLevel string `json:"log_level"`
+	// LogMaxDays specifies the maximum number of days to store log information
+	// before deletion. This is only used if LogWay == "file". By default, this
+	// value is 0.
+	LogMaxDays int64 `json:"log_max_days"`
+	// DisableLogColor disables log colors when LogWay == "console" when set to
+	// true. By default, this value is false.
+	DisableLogColor bool `json:"disable_log_color"`
+	// Token specifies the authorization token used to authenticate keys
+	// received from clients. Clients must have a matching token to be
+	// authorized to use the server. By default, this value is "".
+	Token string `json:"token"`
+	// SubDomainHost specifies the domain that will be attached to sub-domains
+	// requested by the client when using Vhost proxying. For example, if this
+	// value is set to "frps.com" and the client requested the subdomain
+	// "test", the resulting URL would be "test.frps.com". By default, this
+	// value is "".
 	SubDomainHost string `json:"subdomain_host"`
-	TcpMux        bool   `json:"tcp_mux"`
+	// TcpMux toggles TCP stream multiplexing. This allows multiple requests
+	// from a client to share a single TCP connection. By default, this value
+	// is true.
+	TcpMux bool `json:"tcp_mux"`
+	// Custom404Page specifies a path to a custom 404 page to display. If this
+	// value is "", a default page will be displayed. By default, this value is
+	// "".
 	Custom404Page string `json:"custom_404_page"`
 
-	AllowPorts        map[int]struct{}
-	MaxPoolCount      int64 `json:"max_pool_count"`
+	// AllowPorts specifies a set of ports that clients are able to proxy to.
+	// If the length of this value is 0, all ports are allowed. By default,
+	// this value is an empty set.
+	AllowPorts map[int]struct{}
+	// MaxPoolCount specifies the maximum pool size for each proxy. By default,
+	// this value is 5.
+	MaxPoolCount int64 `json:"max_pool_count"`
+	// MaxPortsPerClient specifies the maximum number of ports a single client
+	// may proxy to. If this value is 0, no limit will be applied. By default,
+	// this value is 0.
 	MaxPortsPerClient int64 `json:"max_ports_per_client"`
-	HeartBeatTimeout  int64 `json:"heart_beat_timeout"`
-	UserConnTimeout   int64 `json:"user_conn_timeout"`
+	// HeartBeatTimeout specifies the maximum time to wait for a heartbeat
+	// before terminating the connection. It is not recommended to change this
+	// value. By default, this value is 90.
+	HeartBeatTimeout int64 `json:"heart_beat_timeout"`
+	// UserConnTimeout specifies the maximum time to wait for a work
+	// connection. By default, this value is 10.
+	UserConnTimeout int64 `json:"user_conn_timeout"`
 }
 
-func GetDefaultServerConf() *ServerCommonConf {
-	return &ServerCommonConf{
+// GetDefaultServerConf returns a server configuration with reasonable
+// defaults.
+func GetDefaultServerConf() ServerCommonConf {
+	return ServerCommonConf{
 		BindAddr:          "0.0.0.0",
 		BindPort:          7000,
 		BindUdpPort:       0,
@@ -97,6 +157,7 @@ func GetDefaultServerConf() *ServerCommonConf {
 		LogWay:            "console",
 		LogLevel:          "info",
 		LogMaxDays:        3,
+		DisableLogColor:   false,
 		Token:             "",
 		SubDomainHost:     "",
 		TcpMux:            true,
@@ -109,16 +170,15 @@ func GetDefaultServerConf() *ServerCommonConf {
 	}
 }
 
-func UnmarshalServerConfFromIni(defaultCfg *ServerCommonConf, content string) (cfg *ServerCommonConf, err error) {
-	cfg = defaultCfg
-	if cfg == nil {
-		cfg = GetDefaultServerConf()
-	}
+// UnmarshalServerConfFromIni parses the contents of a server configuration ini
+// file and returns the resulting server configuration.
+func UnmarshalServerConfFromIni(content string) (cfg ServerCommonConf, err error) {
+	cfg = GetDefaultServerConf()
 
 	conf, err := ini.Load(strings.NewReader(content))
 	if err != nil {
 		err = fmt.Errorf("parse ini conf file error: %v", err)
-		return nil, err
+		return ServerCommonConf{}, err
 	}
 
 	var (
@@ -244,6 +304,10 @@ func UnmarshalServerConfFromIni(defaultCfg *ServerCommonConf, content string) (c
 		}
 	}
 
+	if tmpStr, ok = conf.Get("common", "disable_log_color"); ok && tmpStr == "true" {
+		cfg.DisableLogColor = true
+	}
+
 	cfg.Token, _ = conf.Get("common", "token")
 
 	if allowPortsStr, ok := conf.Get("common", "allow_ports"); ok {

models/config/types.go

@@ -0,0 +1,112 @@
+// Copyright 2019 fatedier, fatedier@gmail.com
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package config
+
+import (
+	"encoding/json"
+	"errors"
+	"strconv"
+	"strings"
+)
+
+const (
+	MB = 1024 * 1024
+	KB = 1024
+)
+
+type BandwidthQuantity struct {
+	s string // MB or KB
+
+	i int64 // bytes
+}
+
+func NewBandwidthQuantity(s string) (BandwidthQuantity, error) {
+	q := BandwidthQuantity{}
+	err := q.UnmarshalString(s)
+	if err != nil {
+		return q, err
+	}
+	return q, nil
+}
+
+func (q *BandwidthQuantity) Equal(u *BandwidthQuantity) bool {
+	if q == nil && u == nil {
+		return true
+	}
+	if q != nil && u != nil {
+		return q.i == u.i
+	}
+	return false
+}
+
+func (q *BandwidthQuantity) String() string {
+	return q.s
+}
+
+func (q *BandwidthQuantity) UnmarshalString(s string) error {
+	s = strings.TrimSpace(s)
+	if s == "" {
+		return nil
+	}
+
+	var (
+		base int64
+		f    float64
+		err  error
+	)
+	if strings.HasSuffix(s, "MB") {
+		base = MB
+		fstr := strings.TrimSuffix(s, "MB")
+		f, err = strconv.ParseFloat(fstr, 64)
+		if err != nil {
+			return err
+		}
+	} else if strings.HasSuffix(s, "KB") {
+		base = KB
+		fstr := strings.TrimSuffix(s, "KB")
+		f, err = strconv.ParseFloat(fstr, 64)
+		if err != nil {
+			return err
+		}
+	} else {
+		return errors.New("unit not support")
+	}
+
+	q.s = s
+	q.i = int64(f * float64(base))
+	return nil
+}
+
+func (q *BandwidthQuantity) UnmarshalJSON(b []byte) error {
+	if len(b) == 4 && string(b) == "null" {
+		return nil
+	}
+
+	var str string
+	err := json.Unmarshal(b, &str)
+	if err != nil {
+		return err
+	}
+
+	return q.UnmarshalString(str)
+}
+
+func (q *BandwidthQuantity) MarshalJSON() ([]byte, error) {
+	return []byte("\"" + q.s + "\""), nil
+}
+
+func (q *BandwidthQuantity) Bytes() int64 {
+	return q.i
+}

models/config/types_test.go

@@ -0,0 +1,40 @@
+// Copyright 2019 fatedier, fatedier@gmail.com
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package config
+
+import (
+	"encoding/json"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+)
+
+type Wrap struct {
+	B   BandwidthQuantity `json:"b"`
+	Int int               `json:"int"`
+}
+
+func TestBandwidthQuantity(t *testing.T) {
+	assert := assert.New(t)
+
+	var w Wrap
+	err := json.Unmarshal([]byte(`{"b":"1KB","int":5}`), &w)
+	assert.NoError(err)
+	assert.EqualValues(1*KB, w.B.Bytes())
+
+	buf, err := json.Marshal(&w)
+	assert.NoError(err)
+	assert.Equal(`{"b":"1KB","int":5}`, string(buf))
+}

models/plugin/http2https.go

@@ -0,0 +1,112 @@
+// Copyright 2019 fatedier, fatedier@gmail.com
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package plugin
+
+import (
+	"crypto/tls"
+	"fmt"
+	"io"
+	"net"
+	"net/http"
+	"net/http/httputil"
+	"strings"
+
+	frpNet "github.com/fatedier/frp/utils/net"
+)
+
+const PluginHTTP2HTTPS = "http2https"
+
+func init() {
+	Register(PluginHTTP2HTTPS, NewHTTP2HTTPSPlugin)
+}
+
+type HTTP2HTTPSPlugin struct {
+	hostHeaderRewrite string
+	localAddr         string
+	headers           map[string]string
+
+	l *Listener
+	s *http.Server
+}
+
+func NewHTTP2HTTPSPlugin(params map[string]string) (Plugin, error) {
+	localAddr := params["plugin_local_addr"]
+	hostHeaderRewrite := params["plugin_host_header_rewrite"]
+	headers := make(map[string]string)
+	for k, v := range params {
+		if !strings.HasPrefix(k, "plugin_header_") {
+			continue
+		}
+		if k = strings.TrimPrefix(k, "plugin_header_"); k != "" {
+			headers[k] = v
+		}
+	}
+
+	if localAddr == "" {
+		return nil, fmt.Errorf("plugin_local_addr is required")
+	}
+
+	listener := NewProxyListener()
+
+	p := &HTTPS2HTTPPlugin{
+		localAddr:         localAddr,
+		hostHeaderRewrite: hostHeaderRewrite,
+		headers:           headers,
+		l:                 listener,
+	}
+
+	tr := &http.Transport{
+		TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
+	}
+
+	rp := &httputil.ReverseProxy{
+		Director: func(req *http.Request) {
+			req.URL.Scheme = "https"
+			req.URL.Host = p.localAddr
+			if p.hostHeaderRewrite != "" {
+				req.Host = p.hostHeaderRewrite
+			}
+			for k, v := range p.headers {
+				req.Header.Set(k, v)
+			}
+		},
+		Transport: tr,
+	}
+
+	p.s = &http.Server{
+		Handler: rp,
+	}
+
+	go p.s.Serve(listener)
+
+	return p, nil
+}
+
+
+func (p *HTTP2HTTPSPlugin) Handle(conn io.ReadWriteCloser, realConn net.Conn, extraBufToLocal []byte) {
+	wrapConn := frpNet.WrapReadWriteCloserToConn(conn, realConn)
+	p.l.PutConn(wrapConn)
+}
+
+func (p *HTTP2HTTPSPlugin) Name() string {
+	return PluginHTTP2HTTPS
+}
+
+func (p *HTTP2HTTPSPlugin) Close() error {
+	if err := p.s.Close();err != nil {
+		return err
+	}
+	return nil
+}

models/plugin/http_proxy.go

@@ -64,7 +64,7 @@ func (hp *HttpProxy) Name() string {
 	return PluginHttpProxy
 }
 
-func (hp *HttpProxy) Handle(conn io.ReadWriteCloser, realConn frpNet.Conn, extraBufToLocal []byte) {
+func (hp *HttpProxy) Handle(conn io.ReadWriteCloser, realConn net.Conn, extraBufToLocal []byte) {
 	wrapConn := frpNet.WrapReadWriteCloserToConn(conn, realConn)
 
 	sc, rd := gnet.NewSharedConn(wrapConn)

models/plugin/https2http.go

@@ -18,8 +18,10 @@ import (
 	"crypto/tls"
 	"fmt"
 	"io"
+	"net"
 	"net/http"
 	"net/http/httputil"
+	"strings"
 
 	frpNet "github.com/fatedier/frp/utils/net"
 )
@@ -35,6 +37,7 @@ type HTTPS2HTTPPlugin struct {
 	keyPath           string
 	hostHeaderRewrite string
 	localAddr         string
+	headers           map[string]string
 
 	l *Listener
 	s *http.Server
@@ -45,6 +48,15 @@ func NewHTTPS2HTTPPlugin(params map[string]string) (Plugin, error) {
 	keyPath := params["plugin_key_path"]
 	localAddr := params["plugin_local_addr"]
 	hostHeaderRewrite := params["plugin_host_header_rewrite"]
+	headers := make(map[string]string)
+	for k, v := range params {
+		if !strings.HasPrefix(k, "plugin_header_") {
+			continue
+		}
+		if k = strings.TrimPrefix(k, "plugin_header_"); k != "" {
+			headers[k] = v
+		}
+	}
 
 	if crtPath == "" {
 		return nil, fmt.Errorf("plugin_crt_path is required")
@@ -63,6 +75,7 @@ func NewHTTPS2HTTPPlugin(params map[string]string) (Plugin, error) {
 		keyPath:           keyPath,
 		localAddr:         localAddr,
 		hostHeaderRewrite: hostHeaderRewrite,
+		headers:           headers,
 		l:                 listener,
 	}
 
@@ -73,6 +86,9 @@ func NewHTTPS2HTTPPlugin(params map[string]string) (Plugin, error) {
 			if p.hostHeaderRewrite != "" {
 				req.Host = p.hostHeaderRewrite
 			}
+			for k, v := range p.headers {
+				req.Header.Set(k, v)
+			}
 		},
 	}
 
@@ -100,7 +116,7 @@ func (p *HTTPS2HTTPPlugin) genTLSConfig() (*tls.Config, error) {
 	return config, nil
 }
 
-func (p *HTTPS2HTTPPlugin) Handle(conn io.ReadWriteCloser, realConn frpNet.Conn, extraBufToLocal []byte) {
+func (p *HTTPS2HTTPPlugin) Handle(conn io.ReadWriteCloser, realConn net.Conn, extraBufToLocal []byte) {
 	wrapConn := frpNet.WrapReadWriteCloserToConn(conn, realConn)
 	p.l.PutConn(wrapConn)
 }
@@ -110,5 +126,8 @@ func (p *HTTPS2HTTPPlugin) Name() string {
 }
 
 func (p *HTTPS2HTTPPlugin) Close() error {
+	if err := p.s.Close();err != nil {
+		return err
+	}
 	return nil
 }

models/plugin/plugin.go

@@ -20,8 +20,6 @@ import (
 	"net"
 	"sync"
 
-	frpNet "github.com/fatedier/frp/utils/net"
-
 	"github.com/fatedier/golib/errors"
 )
 
@@ -46,7 +44,9 @@ func Create(name string, params map[string]string) (p Plugin, err error) {
 
 type Plugin interface {
 	Name() string
-	Handle(conn io.ReadWriteCloser, realConn frpNet.Conn, extraBufToLocal []byte)
+
+	// extraBufToLocal will send to local connection first, then join conn with local connection
+	Handle(conn io.ReadWriteCloser, realConn net.Conn, extraBufToLocal []byte)
 	Close() error
 }
 

models/plugin/socks5.go

@@ -18,6 +18,7 @@ import (
 	"io"
 	"io/ioutil"
 	"log"
+	"net"
 
 	frpNet "github.com/fatedier/frp/utils/net"
 
@@ -53,7 +54,7 @@ func NewSocks5Plugin(params map[string]string) (p Plugin, err error) {
 	return
 }
 
-func (sp *Socks5Plugin) Handle(conn io.ReadWriteCloser, realConn frpNet.Conn, extraBufToLocal []byte) {
+func (sp *Socks5Plugin) Handle(conn io.ReadWriteCloser, realConn net.Conn, extraBufToLocal []byte) {
 	defer conn.Close()
 	wrapConn := frpNet.WrapReadWriteCloserToConn(conn, realConn)
 	sp.Server.ServeConn(wrapConn)

models/plugin/static_file.go

@@ -16,6 +16,7 @@ package plugin
 
 import (
 	"io"
+	"net"
 	"net/http"
 
 	frpNet "github.com/fatedier/frp/utils/net"
@@ -72,7 +73,7 @@ func NewStaticFilePlugin(params map[string]string) (Plugin, error) {
 	return sp, nil
 }
 
-func (sp *StaticFilePlugin) Handle(conn io.ReadWriteCloser, realConn frpNet.Conn, extraBufToLocal []byte) {
+func (sp *StaticFilePlugin) Handle(conn io.ReadWriteCloser, realConn net.Conn, extraBufToLocal []byte) {
 	wrapConn := frpNet.WrapReadWriteCloserToConn(conn, realConn)
 	sp.l.PutConn(wrapConn)
 }

models/plugin/unix_domain_socket.go

@@ -19,8 +19,6 @@ import (
 	"io"
 	"net"
 
-	frpNet "github.com/fatedier/frp/utils/net"
-
 	frpIo "github.com/fatedier/golib/io"
 )
 
@@ -53,7 +51,7 @@ func NewUnixDomainSocketPlugin(params map[string]string) (p Plugin, err error) {
 	return
 }
 
-func (uds *UnixDomainSocketPlugin) Handle(conn io.ReadWriteCloser, realConn frpNet.Conn, extraBufToLocal []byte) {
+func (uds *UnixDomainSocketPlugin) Handle(conn io.ReadWriteCloser, realConn net.Conn, extraBufToLocal []byte) {
 	localConn, err := net.DialUnix("unix", nil, uds.UnixAddr)
 	if err != nil {
 		return

models/proto/udp/udp.go

@@ -117,6 +117,7 @@ func Forwarder(dstAddr *net.UDPAddr, readCh <-chan *msg.UdpPacket, sendCh chan<-
 			if !ok {
 				udpConn, err = net.DialUDP("udp", nil, dstAddr)
 				if err != nil {
+					mu.Unlock()
 					continue
 				}
 				udpConnMap[udpMsg.RemoteAddr.String()] = udpConn

server/control.go

@@ -15,13 +15,14 @@
 package server
 
 import (
+	"context"
 	"fmt"
 	"io"
+	"net"
 	"runtime/debug"
 	"sync"
 	"time"
 
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/models/consts"
 	frpErr "github.com/fatedier/frp/models/errors"
@@ -29,8 +30,8 @@ import (
 	"github.com/fatedier/frp/server/controller"
 	"github.com/fatedier/frp/server/proxy"
 	"github.com/fatedier/frp/server/stats"
-	"github.com/fatedier/frp/utils/net"
 	"github.com/fatedier/frp/utils/version"
+	"github.com/fatedier/frp/utils/xlog"
 
 	"github.com/fatedier/golib/control/shutdown"
 	"github.com/fatedier/golib/crypto"
@@ -129,11 +130,22 @@ type Control struct {
 	allShutdown     *shutdown.Shutdown
 
 	mu sync.RWMutex
+
+	// Server configuration information
+	serverCfg config.ServerCommonConf
+
+	xl  *xlog.Logger
+	ctx context.Context
 }
 
-func NewControl(rc *controller.ResourceController, pxyManager *proxy.ProxyManager,
-	statsCollector stats.Collector, ctlConn net.Conn, loginMsg *msg.Login) *Control {
+func NewControl(ctx context.Context, rc *controller.ResourceController, pxyManager *proxy.ProxyManager,
+	statsCollector stats.Collector, ctlConn net.Conn, loginMsg *msg.Login,
+	serverCfg config.ServerCommonConf) *Control {
 
+	poolCount := loginMsg.PoolCount
+	if poolCount > int(serverCfg.MaxPoolCount) {
+		poolCount = int(serverCfg.MaxPoolCount)
+	}
 	return &Control{
 		rc:              rc,
 		pxyManager:      pxyManager,
@@ -142,9 +154,9 @@ func NewControl(rc *controller.ResourceController, pxyManager *proxy.ProxyManage
 		loginMsg:        loginMsg,
 		sendCh:          make(chan msg.Message, 10),
 		readCh:          make(chan msg.Message, 10),
-		workConnCh:      make(chan net.Conn, loginMsg.PoolCount+10),
+		workConnCh:      make(chan net.Conn, poolCount+10),
 		proxies:         make(map[string]proxy.Proxy),
-		poolCount:       loginMsg.PoolCount,
+		poolCount:       poolCount,
 		portsUsedNum:    0,
 		lastPing:        time.Now(),
 		runId:           loginMsg.RunId,
@@ -153,6 +165,9 @@ func NewControl(rc *controller.ResourceController, pxyManager *proxy.ProxyManage
 		writerShutdown:  shutdown.New(),
 		managerShutdown: shutdown.New(),
 		allShutdown:     shutdown.New(),
+		serverCfg:       serverCfg,
+		xl:              xlog.FromContextSafe(ctx),
+		ctx:             ctx,
 	}
 }
 
@@ -161,7 +176,7 @@ func (ctl *Control) Start() {
 	loginRespMsg := &msg.LoginResp{
 		Version:       version.Full(),
 		RunId:         ctl.runId,
-		ServerUdpPort: g.GlbServerCfg.BindUdpPort,
+		ServerUdpPort: ctl.serverCfg.BindUdpPort,
 		Error:         "",
 	}
 	msg.WriteMsg(ctl.conn, loginRespMsg)
@@ -177,18 +192,19 @@ func (ctl *Control) Start() {
 }
 
 func (ctl *Control) RegisterWorkConn(conn net.Conn) {
+	xl := ctl.xl
 	defer func() {
 		if err := recover(); err != nil {
-			ctl.conn.Error("panic error: %v", err)
-			ctl.conn.Error(string(debug.Stack()))
+			xl.Error("panic error: %v", err)
+			xl.Error(string(debug.Stack()))
 		}
 	}()
 
 	select {
 	case ctl.workConnCh <- conn:
-		ctl.conn.Debug("new work connection registered")
+		xl.Debug("new work connection registered")
 	default:
-		ctl.conn.Debug("work connection pool is full, discarding")
+		xl.Debug("work connection pool is full, discarding")
 		conn.Close()
 	}
 }
@@ -198,10 +214,11 @@ func (ctl *Control) RegisterWorkConn(conn net.Conn) {
 // and wait until it is available.
 // return an error if wait timeout
 func (ctl *Control) GetWorkConn() (workConn net.Conn, err error) {
+	xl := ctl.xl
 	defer func() {
 		if err := recover(); err != nil {
-			ctl.conn.Error("panic error: %v", err)
-			ctl.conn.Error(string(debug.Stack()))
+			xl.Error("panic error: %v", err)
+			xl.Error(string(debug.Stack()))
 		}
 	}()
 
@@ -213,14 +230,14 @@ func (ctl *Control) GetWorkConn() (workConn net.Conn, err error) {
 			err = frpErr.ErrCtlClosed
 			return
 		}
-		ctl.conn.Debug("get work connection from pool")
+		xl.Debug("get work connection from pool")
 	default:
 		// no work connections available in the poll, send message to frpc to get more
 		err = errors.PanicToError(func() {
 			ctl.sendCh <- &msg.ReqWorkConn{}
 		})
 		if err != nil {
-			ctl.conn.Error("%v", err)
+			xl.Error("%v", err)
 			return
 		}
 
@@ -228,13 +245,13 @@ func (ctl *Control) GetWorkConn() (workConn net.Conn, err error) {
 		case workConn, ok = <-ctl.workConnCh:
 			if !ok {
 				err = frpErr.ErrCtlClosed
-				ctl.conn.Warn("no work connections avaiable, %v", err)
+				xl.Warn("no work connections avaiable, %v", err)
 				return
 			}
 
-		case <-time.After(time.Duration(g.GlbServerCfg.UserConnTimeout) * time.Second):
+		case <-time.After(time.Duration(ctl.serverCfg.UserConnTimeout) * time.Second):
 			err = fmt.Errorf("timeout trying to get work connection")
-			ctl.conn.Warn("%v", err)
+			xl.Warn("%v", err)
 			return
 		}
 	}
@@ -247,35 +264,37 @@ func (ctl *Control) GetWorkConn() (workConn net.Conn, err error) {
 }
 
 func (ctl *Control) Replaced(newCtl *Control) {
-	ctl.conn.Info("Replaced by client [%s]", newCtl.runId)
+	xl := ctl.xl
+	xl.Info("Replaced by client [%s]", newCtl.runId)
 	ctl.runId = ""
 	ctl.allShutdown.Start()
 }
 
 func (ctl *Control) writer() {
+	xl := ctl.xl
 	defer func() {
 		if err := recover(); err != nil {
-			ctl.conn.Error("panic error: %v", err)
-			ctl.conn.Error(string(debug.Stack()))
+			xl.Error("panic error: %v", err)
+			xl.Error(string(debug.Stack()))
 		}
 	}()
 
 	defer ctl.allShutdown.Start()
 	defer ctl.writerShutdown.Done()
 
-	encWriter, err := crypto.NewWriter(ctl.conn, []byte(g.GlbServerCfg.Token))
+	encWriter, err := crypto.NewWriter(ctl.conn, []byte(ctl.serverCfg.Token))
 	if err != nil {
-		ctl.conn.Error("crypto new writer error: %v", err)
+		xl.Error("crypto new writer error: %v", err)
 		ctl.allShutdown.Start()
 		return
 	}
 	for {
 		if m, ok := <-ctl.sendCh; !ok {
-			ctl.conn.Info("control writer is closing")
+			xl.Info("control writer is closing")
 			return
 		} else {
 			if err := msg.WriteMsg(encWriter, m); err != nil {
-				ctl.conn.Warn("write message to control connection error: %v", err)
+				xl.Warn("write message to control connection error: %v", err)
 				return
 			}
 		}
@@ -283,24 +302,25 @@ func (ctl *Control) writer() {
 }
 
 func (ctl *Control) reader() {
+	xl := ctl.xl
 	defer func() {
 		if err := recover(); err != nil {
-			ctl.conn.Error("panic error: %v", err)
-			ctl.conn.Error(string(debug.Stack()))
+			xl.Error("panic error: %v", err)
+			xl.Error(string(debug.Stack()))
 		}
 	}()
 
 	defer ctl.allShutdown.Start()
 	defer ctl.readerShutdown.Done()
 
-	encReader := crypto.NewReader(ctl.conn, []byte(g.GlbServerCfg.Token))
+	encReader := crypto.NewReader(ctl.conn, []byte(ctl.serverCfg.Token))
 	for {
 		if m, err := msg.ReadMsg(encReader); err != nil {
 			if err == io.EOF {
-				ctl.conn.Debug("control connection closed")
+				xl.Debug("control connection closed")
 				return
 			} else {
-				ctl.conn.Warn("read error: %v", err)
+				xl.Warn("read error: %v", err)
 				ctl.conn.Close()
 				return
 			}
@@ -311,10 +331,11 @@ func (ctl *Control) reader() {
 }
 
 func (ctl *Control) stoper() {
+	xl := ctl.xl
 	defer func() {
 		if err := recover(); err != nil {
-			ctl.conn.Error("panic error: %v", err)
-			ctl.conn.Error(string(debug.Stack()))
+			xl.Error("panic error: %v", err)
+			xl.Error(string(debug.Stack()))
 		}
 	}()
 
@@ -347,7 +368,7 @@ func (ctl *Control) stoper() {
 	}
 
 	ctl.allShutdown.Done()
-	ctl.conn.Info("client exit success")
+	xl.Info("client exit success")
 
 	ctl.statsCollector.Mark(stats.TypeCloseClient, &stats.CloseClientPayload{})
 }
@@ -358,10 +379,11 @@ func (ctl *Control) WaitClosed() {
 }
 
 func (ctl *Control) manager() {
+	xl := ctl.xl
 	defer func() {
 		if err := recover(); err != nil {
-			ctl.conn.Error("panic error: %v", err)
-			ctl.conn.Error(string(debug.Stack()))
+			xl.Error("panic error: %v", err)
+			xl.Error(string(debug.Stack()))
 		}
 	}()
 
@@ -374,8 +396,8 @@ func (ctl *Control) manager() {
 	for {
 		select {
 		case <-heartbeat.C:
-			if time.Since(ctl.lastPing) > time.Duration(g.GlbServerCfg.HeartBeatTimeout)*time.Second {
-				ctl.conn.Warn("heartbeat timeout")
+			if time.Since(ctl.lastPing) > time.Duration(ctl.serverCfg.HeartBeatTimeout)*time.Second {
+				xl.Warn("heartbeat timeout")
 				return
 			}
 		case rawMsg, ok := <-ctl.readCh:
@@ -392,10 +414,10 @@ func (ctl *Control) manager() {
 				}
 				if err != nil {
 					resp.Error = err.Error()
-					ctl.conn.Warn("new proxy [%s] error: %v", m.ProxyName, err)
+					xl.Warn("new proxy [%s] error: %v", m.ProxyName, err)
 				} else {
 					resp.RemoteAddr = remoteAddr
-					ctl.conn.Info("new proxy [%s] success", m.ProxyName)
+					xl.Info("new proxy [%s] success", m.ProxyName)
 					ctl.statsCollector.Mark(stats.TypeNewProxy, &stats.NewProxyPayload{
 						Name:      m.ProxyName,
 						ProxyType: m.ProxyType,
@@ -404,10 +426,10 @@ func (ctl *Control) manager() {
 				ctl.sendCh <- resp
 			case *msg.CloseProxy:
 				ctl.CloseProxy(m)
-				ctl.conn.Info("close proxy [%s] success", m.ProxyName)
+				xl.Info("close proxy [%s] success", m.ProxyName)
 			case *msg.Ping:
 				ctl.lastPing = time.Now()
-				ctl.conn.Debug("receive heartbeat")
+				xl.Debug("receive heartbeat")
 				ctl.sendCh <- &msg.Pong{}
 			}
 		}
@@ -417,22 +439,22 @@ func (ctl *Control) manager() {
 func (ctl *Control) RegisterProxy(pxyMsg *msg.NewProxy) (remoteAddr string, err error) {
 	var pxyConf config.ProxyConf
 	// Load configures from NewProxy message and check.
-	pxyConf, err = config.NewProxyConfFromMsg(pxyMsg)
+	pxyConf, err = config.NewProxyConfFromMsg(pxyMsg, ctl.serverCfg)
 	if err != nil {
 		return
 	}
 
 	// NewProxy will return a interface Proxy.
 	// In fact it create different proxies by different proxy type, we just call run() here.
-	pxy, err := proxy.NewProxy(ctl.runId, ctl.rc, ctl.statsCollector, ctl.poolCount, ctl.GetWorkConn, pxyConf)
+	pxy, err := proxy.NewProxy(ctl.ctx, ctl.runId, ctl.rc, ctl.statsCollector, ctl.poolCount, ctl.GetWorkConn, pxyConf, ctl.serverCfg)
 	if err != nil {
 		return remoteAddr, err
 	}
 
 	// Check ports used number in each client
-	if g.GlbServerCfg.MaxPortsPerClient > 0 {
+	if ctl.serverCfg.MaxPortsPerClient > 0 {
 		ctl.mu.Lock()
-		if ctl.portsUsedNum+pxy.GetUsedPortsNum() > int(g.GlbServerCfg.MaxPortsPerClient) {
+		if ctl.portsUsedNum+pxy.GetUsedPortsNum() > int(ctl.serverCfg.MaxPortsPerClient) {
 			ctl.mu.Unlock()
 			err = fmt.Errorf("exceed the max_ports_per_client")
 			return
@@ -478,7 +500,7 @@ func (ctl *Control) CloseProxy(closeMsg *msg.CloseProxy) (err error) {
 		return
 	}
 
-	if g.GlbServerCfg.MaxPortsPerClient > 0 {
+	if ctl.serverCfg.MaxPortsPerClient > 0 {
 		ctl.portsUsedNum = ctl.portsUsedNum - pxy.GetUsedPortsNum()
 	}
 	pxy.Close()

server/controller/resource.go

@@ -29,6 +29,9 @@ type ResourceController struct {
 	// Tcp Group Controller
 	TcpGroupCtl *group.TcpGroupCtl
 
+	// HTTP Group Controller
+	HTTPGroupCtl *group.HTTPGroupController
+
 	// Manage all tcp ports
 	TcpPortManager *ports.PortManager
 
@@ -38,7 +41,7 @@ type ResourceController struct {
 	// For http proxies, forwarding http requests
 	HttpReverseProxy *vhost.HttpReverseProxy
 
-	// For https proxies, route requests to different clients by hostname and other infomation
+	// For https proxies, route requests to different clients by hostname and other information
 	VhostHttpsMuxer *vhost.HttpsMuxer
 
 	// Controller for nat hole connections

server/controller/visitor.go

@@ -17,6 +17,7 @@ package controller
 import (
 	"fmt"
 	"io"
+	"net"
 	"sync"
 
 	frpNet "github.com/fatedier/frp/utils/net"
@@ -55,7 +56,7 @@ func (vm *VisitorManager) Listen(name string, sk string) (l *frpNet.CustomListen
 	return
 }
 
-func (vm *VisitorManager) NewConn(name string, conn frpNet.Conn, timestamp int64, signKey string,
+func (vm *VisitorManager) NewConn(name string, conn net.Conn, timestamp int64, signKey string,
 	useEncryption bool, useCompression bool) (err error) {
 
 	vm.mu.RLock()

server/dashboard.go

@@ -21,7 +21,6 @@ import (
 	"time"
 
 	"github.com/fatedier/frp/assets"
-	"github.com/fatedier/frp/g"
 	frpNet "github.com/fatedier/frp/utils/net"
 
 	"github.com/gorilla/mux"
@@ -36,7 +35,7 @@ func (svr *Service) RunDashboardServer(addr string, port int) (err error) {
 	// url router
 	router := mux.NewRouter()
 
-	user, passwd := g.GlbServerCfg.DashboardUser, g.GlbServerCfg.DashboardPwd
+	user, passwd := svr.cfg.DashboardUser, svr.cfg.DashboardPwd
 	router.Use(frpNet.NewHttpAuthMiddleware(user, passwd).Middleware)
 
 	// api, see dashboard_api.go

server/dashboard_api.go

@@ -18,7 +18,6 @@ import (
 	"encoding/json"
 	"net/http"
 
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/models/consts"
 	"github.com/fatedier/frp/utils/log"
@@ -63,19 +62,18 @@ func (svr *Service) ApiServerInfo(w http.ResponseWriter, r *http.Request) {
 	}()
 
 	log.Info("Http request: [%s]", r.URL.Path)
-	cfg := &g.GlbServerCfg.ServerCommonConf
 	serverStats := svr.statsCollector.GetServer()
 	svrResp := ServerInfoResp{
 		Version:           version.Full(),
-		BindPort:          cfg.BindPort,
-		BindUdpPort:       cfg.BindUdpPort,
-		VhostHttpPort:     cfg.VhostHttpPort,
-		VhostHttpsPort:    cfg.VhostHttpsPort,
-		KcpBindPort:       cfg.KcpBindPort,
-		SubdomainHost:     cfg.SubDomainHost,
-		MaxPoolCount:      cfg.MaxPoolCount,
-		MaxPortsPerClient: cfg.MaxPortsPerClient,
-		HeartBeatTimeout:  cfg.HeartBeatTimeout,
+		BindPort:          svr.cfg.BindPort,
+		BindUdpPort:       svr.cfg.BindUdpPort,
+		VhostHttpPort:     svr.cfg.VhostHttpPort,
+		VhostHttpsPort:    svr.cfg.VhostHttpsPort,
+		KcpBindPort:       svr.cfg.KcpBindPort,
+		SubdomainHost:     svr.cfg.SubDomainHost,
+		MaxPoolCount:      svr.cfg.MaxPoolCount,
+		MaxPortsPerClient: svr.cfg.MaxPortsPerClient,
+		HeartBeatTimeout:  svr.cfg.HeartBeatTimeout,
 
 		TotalTrafficIn:  serverStats.TotalTrafficIn,
 		TotalTrafficOut: serverStats.TotalTrafficOut,

server/group/group.go

@@ -23,4 +23,5 @@ var (
 	ErrGroupParamsInvalid = errors.New("group params invalid")
 	ErrListenerClosed     = errors.New("group listener closed")
 	ErrGroupDifferentPort = errors.New("group should have same remote port")
+	ErrProxyRepeated      = errors.New("group proxy repeated")
 )

server/group/http.go

@@ -0,0 +1,156 @@
+package group
+
+import (
+	"fmt"
+	"net"
+	"sync"
+	"sync/atomic"
+
+	"github.com/fatedier/frp/utils/vhost"
+)
+
+type HTTPGroupController struct {
+	groups map[string]*HTTPGroup
+
+	vhostRouter *vhost.VhostRouters
+
+	mu sync.Mutex
+}
+
+func NewHTTPGroupController(vhostRouter *vhost.VhostRouters) *HTTPGroupController {
+	return &HTTPGroupController{
+		groups:      make(map[string]*HTTPGroup),
+		vhostRouter: vhostRouter,
+	}
+}
+
+func (ctl *HTTPGroupController) Register(proxyName, group, groupKey string,
+	routeConfig vhost.VhostRouteConfig) (err error) {
+
+	indexKey := httpGroupIndex(group, routeConfig.Domain, routeConfig.Location)
+	ctl.mu.Lock()
+	g, ok := ctl.groups[indexKey]
+	if !ok {
+		g = NewHTTPGroup(ctl)
+		ctl.groups[indexKey] = g
+	}
+	ctl.mu.Unlock()
+
+	return g.Register(proxyName, group, groupKey, routeConfig)
+}
+
+func (ctl *HTTPGroupController) UnRegister(proxyName, group, domain, location string) {
+	indexKey := httpGroupIndex(group, domain, location)
+	ctl.mu.Lock()
+	defer ctl.mu.Unlock()
+	g, ok := ctl.groups[indexKey]
+	if !ok {
+		return
+	}
+
+	isEmpty := g.UnRegister(proxyName)
+	if isEmpty {
+		delete(ctl.groups, indexKey)
+	}
+}
+
+type HTTPGroup struct {
+	group    string
+	groupKey string
+	domain   string
+	location string
+
+	createFuncs map[string]vhost.CreateConnFunc
+	pxyNames    []string
+	index       uint64
+	ctl         *HTTPGroupController
+	mu          sync.RWMutex
+}
+
+func NewHTTPGroup(ctl *HTTPGroupController) *HTTPGroup {
+	return &HTTPGroup{
+		createFuncs: make(map[string]vhost.CreateConnFunc),
+		pxyNames:    make([]string, 0),
+		ctl:         ctl,
+	}
+}
+
+func (g *HTTPGroup) Register(proxyName, group, groupKey string,
+	routeConfig vhost.VhostRouteConfig) (err error) {
+
+	g.mu.Lock()
+	defer g.mu.Unlock()
+	if len(g.createFuncs) == 0 {
+		// the first proxy in this group
+		tmp := routeConfig // copy object
+		tmp.CreateConnFn = g.createConn
+		err = g.ctl.vhostRouter.Add(routeConfig.Domain, routeConfig.Location, &tmp)
+		if err != nil {
+			return
+		}
+
+		g.group = group
+		g.groupKey = groupKey
+		g.domain = routeConfig.Domain
+		g.location = routeConfig.Location
+	} else {
+		if g.group != group || g.domain != routeConfig.Domain || g.location != routeConfig.Location {
+			err = ErrGroupParamsInvalid
+			return
+		}
+		if g.groupKey != groupKey {
+			err = ErrGroupAuthFailed
+			return
+		}
+	}
+	if _, ok := g.createFuncs[proxyName]; ok {
+		err = ErrProxyRepeated
+		return
+	}
+	g.createFuncs[proxyName] = routeConfig.CreateConnFn
+	g.pxyNames = append(g.pxyNames, proxyName)
+	return nil
+}
+
+func (g *HTTPGroup) UnRegister(proxyName string) (isEmpty bool) {
+	g.mu.Lock()
+	defer g.mu.Unlock()
+	delete(g.createFuncs, proxyName)
+	for i, name := range g.pxyNames {
+		if name == proxyName {
+			g.pxyNames = append(g.pxyNames[:i], g.pxyNames[i+1:]...)
+			break
+		}
+	}
+
+	if len(g.createFuncs) == 0 {
+		isEmpty = true
+		g.ctl.vhostRouter.Del(g.domain, g.location)
+	}
+	return
+}
+
+func (g *HTTPGroup) createConn(remoteAddr string) (net.Conn, error) {
+	var f vhost.CreateConnFunc
+	newIndex := atomic.AddUint64(&g.index, 1)
+
+	g.mu.RLock()
+	group := g.group
+	domain := g.domain
+	location := g.location
+	if len(g.pxyNames) > 0 {
+		name := g.pxyNames[int(newIndex)%len(g.pxyNames)]
+		f, _ = g.createFuncs[name]
+	}
+	g.mu.RUnlock()
+
+	if f == nil {
+		return nil, fmt.Errorf("no CreateConnFunc for http group [%s], domain [%s], location [%s]", group, domain, location)
+	}
+
+	return f(remoteAddr)
+}
+
+func httpGroupIndex(group, domain, location string) string {
+	return fmt.Sprintf("%s_%s_%s", group, domain, location)
+}

server/group/tcp.go

@@ -24,46 +24,47 @@ import (
 	gerr "github.com/fatedier/golib/errors"
 )
 
-type TcpGroupListener struct {
-	groupName string
-	group     *TcpGroup
+// TcpGroupCtl manage all TcpGroups
+type TcpGroupCtl struct {
+	groups map[string]*TcpGroup
 
-	addr    net.Addr
-	closeCh chan struct{}
+	// portManager is used to manage port
+	portManager *ports.PortManager
+	mu          sync.Mutex
 }
 
-func newTcpGroupListener(name string, group *TcpGroup, addr net.Addr) *TcpGroupListener {
-	return &TcpGroupListener{
-		groupName: name,
-		group:     group,
-		addr:      addr,
-		closeCh:   make(chan struct{}),
+// NewTcpGroupCtl return a new TcpGroupCtl
+func NewTcpGroupCtl(portManager *ports.PortManager) *TcpGroupCtl {
+	return &TcpGroupCtl{
+		groups:      make(map[string]*TcpGroup),
+		portManager: portManager,
 	}
 }
 
-func (ln *TcpGroupListener) Accept() (c net.Conn, err error) {
-	var ok bool
-	select {
-	case <-ln.closeCh:
-		return nil, ErrListenerClosed
-	case c, ok = <-ln.group.Accept():
-		if !ok {
-			return nil, ErrListenerClosed
-		}
-		return c, nil
+// Listen is the wrapper for TcpGroup's Listen
+// If there are no group, we will create one here
+func (tgc *TcpGroupCtl) Listen(proxyName string, group string, groupKey string,
+	addr string, port int) (l net.Listener, realPort int, err error) {
+
+	tgc.mu.Lock()
+	tcpGroup, ok := tgc.groups[group]
+	if !ok {
+		tcpGroup = NewTcpGroup(tgc)
+		tgc.groups[group] = tcpGroup
 	}
+	tgc.mu.Unlock()
+
+	return tcpGroup.Listen(proxyName, group, groupKey, addr, port)
 }
 
-func (ln *TcpGroupListener) Addr() net.Addr {
-	return ln.addr
-}
-
-func (ln *TcpGroupListener) Close() (err error) {
-	close(ln.closeCh)
-	ln.group.CloseListener(ln)
-	return
+// RemoveGroup remove TcpGroup from controller
+func (tgc *TcpGroupCtl) RemoveGroup(group string) {
+	tgc.mu.Lock()
+	defer tgc.mu.Unlock()
+	delete(tgc.groups, group)
 }
 
+// TcpGroup route connections to different proxies
 type TcpGroup struct {
 	group    string
 	groupKey string
@@ -79,6 +80,7 @@ type TcpGroup struct {
 	mu       sync.Mutex
 }
 
+// NewTcpGroup return a new TcpGroup
 func NewTcpGroup(ctl *TcpGroupCtl) *TcpGroup {
 	return &TcpGroup{
 		lns:      make([]*TcpGroupListener, 0),
@@ -87,10 +89,14 @@ func NewTcpGroup(ctl *TcpGroupCtl) *TcpGroup {
 	}
 }
 
+// Listen will return a new TcpGroupListener
+// if TcpGroup already has a listener, just add a new TcpGroupListener to the queues
+// otherwise, listen on the real address
 func (tg *TcpGroup) Listen(proxyName string, group string, groupKey string, addr string, port int) (ln *TcpGroupListener, realPort int, err error) {
 	tg.mu.Lock()
 	defer tg.mu.Unlock()
 	if len(tg.lns) == 0 {
+		// the first listener, listen on the real address
 		realPort, err = tg.ctl.portManager.Acquire(proxyName, port)
 		if err != nil {
 			return
@@ -114,6 +120,7 @@ func (tg *TcpGroup) Listen(proxyName string, group string, groupKey string, addr
 		}
 		go tg.worker()
 	} else {
+		// address and port in the same group must be equal
 		if tg.group != group || tg.addr != addr {
 			err = ErrGroupParamsInvalid
 			return
@@ -133,6 +140,7 @@ func (tg *TcpGroup) Listen(proxyName string, group string, groupKey string, addr
 	return
 }
 
+// worker is called when the real tcp listener has been created
 func (tg *TcpGroup) worker() {
 	for {
 		c, err := tg.tcpLn.Accept()
@@ -152,6 +160,7 @@ func (tg *TcpGroup) Accept() <-chan net.Conn {
 	return tg.acceptCh
 }
 
+// CloseListener remove the TcpGroupListener from the TcpGroup
 func (tg *TcpGroup) CloseListener(ln *TcpGroupListener) {
 	tg.mu.Lock()
 	defer tg.mu.Unlock()
@@ -169,36 +178,47 @@ func (tg *TcpGroup) CloseListener(ln *TcpGroupListener) {
 	}
 }
 
-type TcpGroupCtl struct {
-	groups map[string]*TcpGroup
+// TcpGroupListener
+type TcpGroupListener struct {
+	groupName string
+	group     *TcpGroup
 
-	portManager *ports.PortManager
-	mu          sync.Mutex
+	addr    net.Addr
+	closeCh chan struct{}
 }
 
-func NewTcpGroupCtl(portManager *ports.PortManager) *TcpGroupCtl {
-	return &TcpGroupCtl{
-		groups:      make(map[string]*TcpGroup),
-		portManager: portManager,
+func newTcpGroupListener(name string, group *TcpGroup, addr net.Addr) *TcpGroupListener {
+	return &TcpGroupListener{
+		groupName: name,
+		group:     group,
+		addr:      addr,
+		closeCh:   make(chan struct{}),
 	}
 }
 
-func (tgc *TcpGroupCtl) Listen(proxyNanme string, group string, groupKey string,
-	addr string, port int) (l net.Listener, realPort int, err error) {
-
-	tgc.mu.Lock()
-	defer tgc.mu.Unlock()
-	if tcpGroup, ok := tgc.groups[group]; ok {
-		return tcpGroup.Listen(proxyNanme, group, groupKey, addr, port)
-	} else {
-		tcpGroup = NewTcpGroup(tgc)
-		tgc.groups[group] = tcpGroup
-		return tcpGroup.Listen(proxyNanme, group, groupKey, addr, port)
+// Accept will accept connections from TcpGroup
+func (ln *TcpGroupListener) Accept() (c net.Conn, err error) {
+	var ok bool
+	select {
+	case <-ln.closeCh:
+		return nil, ErrListenerClosed
+	case c, ok = <-ln.group.Accept():
+		if !ok {
+			return nil, ErrListenerClosed
+		}
+		return c, nil
 	}
 }
 
-func (tgc *TcpGroupCtl) RemoveGroup(group string) {
-	tgc.mu.Lock()
-	defer tgc.mu.Unlock()
-	delete(tgc.groups, group)
+func (ln *TcpGroupListener) Addr() net.Addr {
+	return ln.addr
+}
+
+// Close close the listener
+func (ln *TcpGroupListener) Close() (err error) {
+	close(ln.closeCh)
+
+	// remove self from TcpGroup
+	ln.group.CloseListener(ln)
+	return
 }

server/proxy/http.go

@@ -19,7 +19,6 @@ import (
 	"net"
 	"strings"
 
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/server/stats"
 	frpNet "github.com/fatedier/frp/utils/net"
@@ -37,6 +36,7 @@ type HttpProxy struct {
 }
 
 func (pxy *HttpProxy) Run() (remoteAddr string, err error) {
+	xl := pxy.xl
 	routeConfig := vhost.VhostRouteConfig{
 		RewriteHost:  pxy.cfg.HostHeaderRewrite,
 		Headers:      pxy.cfg.Headers,
@@ -50,6 +50,12 @@ func (pxy *HttpProxy) Run() (remoteAddr string, err error) {
 		locations = []string{""}
 	}
 
+	defer func() {
+		if err != nil {
+			pxy.Close()
+		}
+	}()
+
 	addrs := make([]string, 0)
 	for _, domain := range pxy.cfg.CustomDomains {
 		if domain == "" {
@@ -59,35 +65,63 @@ func (pxy *HttpProxy) Run() (remoteAddr string, err error) {
 		routeConfig.Domain = domain
 		for _, location := range locations {
 			routeConfig.Location = location
-			err = pxy.rc.HttpReverseProxy.Register(routeConfig)
-			if err != nil {
-				return
-			}
 			tmpDomain := routeConfig.Domain
 			tmpLocation := routeConfig.Location
-			addrs = append(addrs, util.CanonicalAddr(tmpDomain, int(g.GlbServerCfg.VhostHttpPort)))
-			pxy.closeFuncs = append(pxy.closeFuncs, func() {
-				pxy.rc.HttpReverseProxy.UnRegister(tmpDomain, tmpLocation)
-			})
-			pxy.Info("http proxy listen for host [%s] location [%s]", routeConfig.Domain, routeConfig.Location)
+
+			// handle group
+			if pxy.cfg.Group != "" {
+				err = pxy.rc.HTTPGroupCtl.Register(pxy.name, pxy.cfg.Group, pxy.cfg.GroupKey, routeConfig)
+				if err != nil {
+					return
+				}
+
+				pxy.closeFuncs = append(pxy.closeFuncs, func() {
+					pxy.rc.HTTPGroupCtl.UnRegister(pxy.name, pxy.cfg.Group, tmpDomain, tmpLocation)
+				})
+			} else {
+				// no group
+				err = pxy.rc.HttpReverseProxy.Register(routeConfig)
+				if err != nil {
+					return
+				}
+				pxy.closeFuncs = append(pxy.closeFuncs, func() {
+					pxy.rc.HttpReverseProxy.UnRegister(tmpDomain, tmpLocation)
+				})
+			}
+			addrs = append(addrs, util.CanonicalAddr(routeConfig.Domain, int(pxy.serverCfg.VhostHttpPort)))
+			xl.Info("http proxy listen for host [%s] location [%s] group [%s]", routeConfig.Domain, routeConfig.Location, pxy.cfg.Group)
 		}
 	}
 
 	if pxy.cfg.SubDomain != "" {
-		routeConfig.Domain = pxy.cfg.SubDomain + "." + g.GlbServerCfg.SubDomainHost
+		routeConfig.Domain = pxy.cfg.SubDomain + "." + pxy.serverCfg.SubDomainHost
 		for _, location := range locations {
 			routeConfig.Location = location
-			err = pxy.rc.HttpReverseProxy.Register(routeConfig)
-			if err != nil {
-				return
-			}
 			tmpDomain := routeConfig.Domain
 			tmpLocation := routeConfig.Location
-			addrs = append(addrs, util.CanonicalAddr(tmpDomain, g.GlbServerCfg.VhostHttpPort))
-			pxy.closeFuncs = append(pxy.closeFuncs, func() {
-				pxy.rc.HttpReverseProxy.UnRegister(tmpDomain, tmpLocation)
-			})
-			pxy.Info("http proxy listen for host [%s] location [%s]", routeConfig.Domain, routeConfig.Location)
+
+			// handle group
+			if pxy.cfg.Group != "" {
+				err = pxy.rc.HTTPGroupCtl.Register(pxy.name, pxy.cfg.Group, pxy.cfg.GroupKey, routeConfig)
+				if err != nil {
+					return
+				}
+
+				pxy.closeFuncs = append(pxy.closeFuncs, func() {
+					pxy.rc.HTTPGroupCtl.UnRegister(pxy.name, pxy.cfg.Group, tmpDomain, tmpLocation)
+				})
+			} else {
+				err = pxy.rc.HttpReverseProxy.Register(routeConfig)
+				if err != nil {
+					return
+				}
+				pxy.closeFuncs = append(pxy.closeFuncs, func() {
+					pxy.rc.HttpReverseProxy.UnRegister(tmpDomain, tmpLocation)
+				})
+			}
+			addrs = append(addrs, util.CanonicalAddr(tmpDomain, pxy.serverCfg.VhostHttpPort))
+
+			xl.Info("http proxy listen for host [%s] location [%s] group [%s]", routeConfig.Domain, routeConfig.Location, pxy.cfg.Group)
 		}
 	}
 	remoteAddr = strings.Join(addrs, ",")
@@ -98,10 +132,11 @@ func (pxy *HttpProxy) GetConf() config.ProxyConf {
 	return pxy.cfg
 }
 
-func (pxy *HttpProxy) GetRealConn(remoteAddr string) (workConn frpNet.Conn, err error) {
+func (pxy *HttpProxy) GetRealConn(remoteAddr string) (workConn net.Conn, err error) {
+	xl := pxy.xl
 	rAddr, errRet := net.ResolveTCPAddr("tcp", remoteAddr)
 	if errRet != nil {
-		pxy.Warn("resolve TCP addr [%s] error: %v", remoteAddr, errRet)
+		xl.Warn("resolve TCP addr [%s] error: %v", remoteAddr, errRet)
 		// we do not return error here since remoteAddr is not necessary for proxies without proxy protocol enabled
 	}
 
@@ -113,9 +148,9 @@ func (pxy *HttpProxy) GetRealConn(remoteAddr string) (workConn frpNet.Conn, err
 
 	var rwc io.ReadWriteCloser = tmpConn
 	if pxy.cfg.UseEncryption {
-		rwc, err = frpIo.WithEncryption(rwc, []byte(g.GlbServerCfg.Token))
+		rwc, err = frpIo.WithEncryption(rwc, []byte(pxy.serverCfg.Token))
 		if err != nil {
-			pxy.Error("create encryption stream error: %v", err)
+			xl.Error("create encryption stream error: %v", err)
 			return
 		}
 	}

server/proxy/https.go

@@ -17,7 +17,6 @@ package proxy
 import (
 	"strings"
 
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/utils/util"
 	"github.com/fatedier/frp/utils/vhost"
@@ -29,8 +28,14 @@ type HttpsProxy struct {
 }
 
 func (pxy *HttpsProxy) Run() (remoteAddr string, err error) {
+	xl := pxy.xl
 	routeConfig := &vhost.VhostRouteConfig{}
 
+	defer func() {
+		if err != nil {
+			pxy.Close()
+		}
+	}()
 	addrs := make([]string, 0)
 	for _, domain := range pxy.cfg.CustomDomains {
 		if domain == "" {
@@ -38,28 +43,26 @@ func (pxy *HttpsProxy) Run() (remoteAddr string, err error) {
 		}
 
 		routeConfig.Domain = domain
-		l, errRet := pxy.rc.VhostHttpsMuxer.Listen(routeConfig)
+		l, errRet := pxy.rc.VhostHttpsMuxer.Listen(pxy.ctx, routeConfig)
 		if errRet != nil {
 			err = errRet
 			return
 		}
-		l.AddLogPrefix(pxy.name)
-		pxy.Info("https proxy listen for host [%s]", routeConfig.Domain)
+		xl.Info("https proxy listen for host [%s]", routeConfig.Domain)
 		pxy.listeners = append(pxy.listeners, l)
-		addrs = append(addrs, util.CanonicalAddr(routeConfig.Domain, g.GlbServerCfg.VhostHttpsPort))
+		addrs = append(addrs, util.CanonicalAddr(routeConfig.Domain, pxy.serverCfg.VhostHttpsPort))
 	}
 
 	if pxy.cfg.SubDomain != "" {
-		routeConfig.Domain = pxy.cfg.SubDomain + "." + g.GlbServerCfg.SubDomainHost
-		l, errRet := pxy.rc.VhostHttpsMuxer.Listen(routeConfig)
+		routeConfig.Domain = pxy.cfg.SubDomain + "." + pxy.serverCfg.SubDomainHost
+		l, errRet := pxy.rc.VhostHttpsMuxer.Listen(pxy.ctx, routeConfig)
 		if errRet != nil {
 			err = errRet
 			return
 		}
-		l.AddLogPrefix(pxy.name)
-		pxy.Info("https proxy listen for host [%s]", routeConfig.Domain)
+		xl.Info("https proxy listen for host [%s]", routeConfig.Domain)
 		pxy.listeners = append(pxy.listeners, l)
-		addrs = append(addrs, util.CanonicalAddr(routeConfig.Domain, int(g.GlbServerCfg.VhostHttpsPort)))
+		addrs = append(addrs, util.CanonicalAddr(routeConfig.Domain, int(pxy.serverCfg.VhostHttpsPort)))
 	}
 
 	pxy.startListenHandler(pxy, HandleUserTcpConnection)

server/proxy/proxy.go

@@ -15,72 +15,83 @@
 package proxy
 
 import (
+	"context"
 	"fmt"
 	"io"
 	"net"
 	"strconv"
 	"sync"
 
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/models/msg"
 	"github.com/fatedier/frp/server/controller"
 	"github.com/fatedier/frp/server/stats"
-	"github.com/fatedier/frp/utils/log"
 	frpNet "github.com/fatedier/frp/utils/net"
+	"github.com/fatedier/frp/utils/xlog"
 
 	frpIo "github.com/fatedier/golib/io"
 )
 
-type GetWorkConnFn func() (frpNet.Conn, error)
+type GetWorkConnFn func() (net.Conn, error)
 
 type Proxy interface {
+	Context() context.Context
 	Run() (remoteAddr string, err error)
 	GetName() string
 	GetConf() config.ProxyConf
-	GetWorkConnFromPool(src, dst net.Addr) (workConn frpNet.Conn, err error)
+	GetWorkConnFromPool(src, dst net.Addr) (workConn net.Conn, err error)
 	GetUsedPortsNum() int
 	Close()
-	log.Logger
 }
 
 type BaseProxy struct {
 	name           string
 	rc             *controller.ResourceController
 	statsCollector stats.Collector
-	listeners      []frpNet.Listener
+	listeners      []net.Listener
 	usedPortsNum   int
 	poolCount      int
 	getWorkConnFn  GetWorkConnFn
+	serverCfg      config.ServerCommonConf
 
-	mu sync.RWMutex
-	log.Logger
+	mu  sync.RWMutex
+	xl  *xlog.Logger
+	ctx context.Context
 }
 
 func (pxy *BaseProxy) GetName() string {
 	return pxy.name
 }
 
+func (pxy *BaseProxy) Context() context.Context {
+	return pxy.ctx
+}
+
 func (pxy *BaseProxy) GetUsedPortsNum() int {
 	return pxy.usedPortsNum
 }
 
 func (pxy *BaseProxy) Close() {
-	pxy.Info("proxy closing")
+	xl := xlog.FromContextSafe(pxy.ctx)
+	xl.Info("proxy closing")
 	for _, l := range pxy.listeners {
 		l.Close()
 	}
 }
 
-func (pxy *BaseProxy) GetWorkConnFromPool(src, dst net.Addr) (workConn frpNet.Conn, err error) {
+// GetWorkConnFromPool try to get a new work connections from pool
+// for quickly response, we immediately send the StartWorkConn message to frpc after take out one from pool
+func (pxy *BaseProxy) GetWorkConnFromPool(src, dst net.Addr) (workConn net.Conn, err error) {
+	xl := xlog.FromContextSafe(pxy.ctx)
 	// try all connections from the pool
 	for i := 0; i < pxy.poolCount+1; i++ {
 		if workConn, err = pxy.getWorkConnFn(); err != nil {
-			pxy.Warn("failed to get work connection: %v", err)
+			xl.Warn("failed to get work connection: %v", err)
 			return
 		}
-		pxy.Info("get a new work connection: [%s]", workConn.RemoteAddr().String())
-		workConn.AddLogPrefix(pxy.GetName())
+		xl.Info("get a new work connection: [%s]", workConn.RemoteAddr().String())
+		xl.Spawn().AppendPrefix(pxy.GetName())
+		workConn = frpNet.NewContextConn(workConn, pxy.ctx)
 
 		var (
 			srcAddr    string
@@ -107,7 +118,7 @@ func (pxy *BaseProxy) GetWorkConnFromPool(src, dst net.Addr) (workConn frpNet.Co
 			DstPort:   uint16(dstPort),
 		})
 		if err != nil {
-			workConn.Warn("failed to send message to work connection from pool: %v, times: %d", err, i)
+			xl.Warn("failed to send message to work connection from pool: %v, times: %d", err, i)
 			workConn.Close()
 		} else {
 			break
@@ -115,7 +126,7 @@ func (pxy *BaseProxy) GetWorkConnFromPool(src, dst net.Addr) (workConn frpNet.Co
 	}
 
 	if err != nil {
-		pxy.Error("try to get work connection failed in the end")
+		xl.Error("try to get work connection failed in the end")
 		return
 	}
 	return
@@ -124,35 +135,39 @@ func (pxy *BaseProxy) GetWorkConnFromPool(src, dst net.Addr) (workConn frpNet.Co
 // startListenHandler start a goroutine handler for each listener.
 // p: p will just be passed to handler(Proxy, frpNet.Conn).
 // handler: each proxy type can set different handler function to deal with connections accepted from listeners.
-func (pxy *BaseProxy) startListenHandler(p Proxy, handler func(Proxy, frpNet.Conn, stats.Collector)) {
+func (pxy *BaseProxy) startListenHandler(p Proxy, handler func(Proxy, net.Conn, stats.Collector, config.ServerCommonConf)) {
+	xl := xlog.FromContextSafe(pxy.ctx)
 	for _, listener := range pxy.listeners {
-		go func(l frpNet.Listener) {
+		go func(l net.Listener) {
 			for {
 				// block
 				// if listener is closed, err returned
 				c, err := l.Accept()
 				if err != nil {
-					pxy.Info("listener is closed")
+					xl.Info("listener is closed")
 					return
 				}
-				pxy.Debug("get a user connection [%s]", c.RemoteAddr().String())
-				go handler(p, c, pxy.statsCollector)
+				xl.Debug("get a user connection [%s]", c.RemoteAddr().String())
+				go handler(p, c, pxy.statsCollector, pxy.serverCfg)
 			}
 		}(listener)
 	}
 }
 
-func NewProxy(runId string, rc *controller.ResourceController, statsCollector stats.Collector, poolCount int,
-	getWorkConnFn GetWorkConnFn, pxyConf config.ProxyConf) (pxy Proxy, err error) {
+func NewProxy(ctx context.Context, runId string, rc *controller.ResourceController, statsCollector stats.Collector, poolCount int,
+	getWorkConnFn GetWorkConnFn, pxyConf config.ProxyConf, serverCfg config.ServerCommonConf) (pxy Proxy, err error) {
 
+	xl := xlog.FromContextSafe(ctx).Spawn().AppendPrefix(pxyConf.GetBaseInfo().ProxyName)
 	basePxy := BaseProxy{
 		name:           pxyConf.GetBaseInfo().ProxyName,
 		rc:             rc,
 		statsCollector: statsCollector,
-		listeners:      make([]frpNet.Listener, 0),
+		listeners:      make([]net.Listener, 0),
 		poolCount:      poolCount,
 		getWorkConnFn:  getWorkConnFn,
-		Logger:         log.NewPrefixLogger(runId),
+		serverCfg:      serverCfg,
+		xl:             xl,
+		ctx:            xlog.NewContext(ctx, xl),
 	}
 	switch cfg := pxyConf.(type) {
 	case *config.TcpProxyConf:
@@ -190,13 +205,13 @@ func NewProxy(runId string, rc *controller.ResourceController, statsCollector st
 	default:
 		return pxy, fmt.Errorf("proxy type not support")
 	}
-	pxy.AddLogPrefix(pxy.GetName())
 	return
 }
 
 // HandleUserTcpConnection is used for incoming tcp user connections.
 // It can be used for tcp, http, https type.
-func HandleUserTcpConnection(pxy Proxy, userConn frpNet.Conn, statsCollector stats.Collector) {
+func HandleUserTcpConnection(pxy Proxy, userConn net.Conn, statsCollector stats.Collector, serverCfg config.ServerCommonConf) {
+	xl := xlog.FromContextSafe(pxy.Context())
 	defer userConn.Close()
 
 	// try all connections from the pool
@@ -208,17 +223,18 @@ func HandleUserTcpConnection(pxy Proxy, userConn frpNet.Conn, statsCollector sta
 
 	var local io.ReadWriteCloser = workConn
 	cfg := pxy.GetConf().GetBaseInfo()
+	xl.Trace("handler user tcp connection, use_encryption: %t, use_compression: %t", cfg.UseEncryption, cfg.UseCompression)
 	if cfg.UseEncryption {
-		local, err = frpIo.WithEncryption(local, []byte(g.GlbServerCfg.Token))
+		local, err = frpIo.WithEncryption(local, []byte(serverCfg.Token))
 		if err != nil {
-			pxy.Error("create encryption stream error: %v", err)
+			xl.Error("create encryption stream error: %v", err)
 			return
 		}
 	}
 	if cfg.UseCompression {
 		local = frpIo.WithCompression(local)
 	}
-	pxy.Debug("join connections, workConn(l[%s] r[%s]) userConn(l[%s] r[%s])", workConn.LocalAddr().String(),
+	xl.Debug("join connections, workConn(l[%s] r[%s]) userConn(l[%s] r[%s])", workConn.LocalAddr().String(),
 		workConn.RemoteAddr().String(), userConn.LocalAddr().String(), userConn.RemoteAddr().String())
 
 	statsCollector.Mark(stats.TypeOpenConnection, &stats.OpenConnectionPayload{ProxyName: pxy.GetName()})
@@ -232,7 +248,7 @@ func HandleUserTcpConnection(pxy Proxy, userConn frpNet.Conn, statsCollector sta
 		ProxyName:    pxy.GetName(),
 		TrafficBytes: outCount,
 	})
-	pxy.Debug("join connections closed")
+	xl.Debug("join connections closed")
 }
 
 type ProxyManager struct {

server/proxy/stcp.go

@@ -24,14 +24,14 @@ type StcpProxy struct {
 }
 
 func (pxy *StcpProxy) Run() (remoteAddr string, err error) {
+	xl := pxy.xl
 	listener, errRet := pxy.rc.VisitorManager.Listen(pxy.GetName(), pxy.cfg.Sk)
 	if errRet != nil {
 		err = errRet
 		return
 	}
-	listener.AddLogPrefix(pxy.name)
 	pxy.listeners = append(pxy.listeners, listener)
-	pxy.Info("stcp proxy custom listen success")
+	xl.Info("stcp proxy custom listen success")
 
 	pxy.startListenHandler(pxy, HandleUserTcpConnection)
 	return

server/proxy/tcp.go

@@ -16,10 +16,9 @@ package proxy
 
 import (
 	"fmt"
+	"net"
 
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
-	frpNet "github.com/fatedier/frp/utils/net"
 )
 
 type TcpProxy struct {
@@ -30,8 +29,9 @@ type TcpProxy struct {
 }
 
 func (pxy *TcpProxy) Run() (remoteAddr string, err error) {
+	xl := pxy.xl
 	if pxy.cfg.Group != "" {
-		l, realPort, errRet := pxy.rc.TcpGroupCtl.Listen(pxy.name, pxy.cfg.Group, pxy.cfg.GroupKey, g.GlbServerCfg.ProxyBindAddr, pxy.cfg.RemotePort)
+		l, realPort, errRet := pxy.rc.TcpGroupCtl.Listen(pxy.name, pxy.cfg.Group, pxy.cfg.GroupKey, pxy.serverCfg.ProxyBindAddr, pxy.cfg.RemotePort)
 		if errRet != nil {
 			err = errRet
 			return
@@ -42,10 +42,8 @@ func (pxy *TcpProxy) Run() (remoteAddr string, err error) {
 			}
 		}()
 		pxy.realPort = realPort
-		listener := frpNet.WrapLogListener(l)
-		listener.AddLogPrefix(pxy.name)
-		pxy.listeners = append(pxy.listeners, listener)
-		pxy.Info("tcp proxy listen port [%d] in group [%s]", pxy.cfg.RemotePort, pxy.cfg.Group)
+		pxy.listeners = append(pxy.listeners, l)
+		xl.Info("tcp proxy listen port [%d] in group [%s]", pxy.cfg.RemotePort, pxy.cfg.Group)
 	} else {
 		pxy.realPort, err = pxy.rc.TcpPortManager.Acquire(pxy.name, pxy.cfg.RemotePort)
 		if err != nil {
@@ -56,14 +54,13 @@ func (pxy *TcpProxy) Run() (remoteAddr string, err error) {
 				pxy.rc.TcpPortManager.Release(pxy.realPort)
 			}
 		}()
-		listener, errRet := frpNet.ListenTcp(g.GlbServerCfg.ProxyBindAddr, pxy.realPort)
+		listener, errRet := net.Listen("tcp", fmt.Sprintf("%s:%d", pxy.serverCfg.ProxyBindAddr, pxy.realPort))
 		if errRet != nil {
 			err = errRet
 			return
 		}
-		listener.AddLogPrefix(pxy.name)
 		pxy.listeners = append(pxy.listeners, listener)
-		pxy.Info("tcp proxy listen port [%d]", pxy.cfg.RemotePort)
+		xl.Info("tcp proxy listen port [%d]", pxy.cfg.RemotePort)
 	}
 
 	pxy.cfg.RemotePort = pxy.realPort

server/proxy/udp.go

@@ -20,7 +20,6 @@ import (
 	"net"
 	"time"
 
-	"github.com/fatedier/frp/g"
 	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/models/msg"
 	"github.com/fatedier/frp/models/proto/udp"
@@ -55,6 +54,7 @@ type UdpProxy struct {
 }
 
 func (pxy *UdpProxy) Run() (remoteAddr string, err error) {
+	xl := pxy.xl
 	pxy.realPort, err = pxy.rc.UdpPortManager.Acquire(pxy.name, pxy.cfg.RemotePort)
 	if err != nil {
 		return
@@ -67,7 +67,7 @@ func (pxy *UdpProxy) Run() (remoteAddr string, err error) {
 
 	remoteAddr = fmt.Sprintf(":%d", pxy.realPort)
 	pxy.cfg.RemotePort = pxy.realPort
-	addr, errRet := net.ResolveUDPAddr("udp", fmt.Sprintf("%s:%d", g.GlbServerCfg.ProxyBindAddr, pxy.realPort))
+	addr, errRet := net.ResolveUDPAddr("udp", fmt.Sprintf("%s:%d", pxy.serverCfg.ProxyBindAddr, pxy.realPort))
 	if errRet != nil {
 		err = errRet
 		return
@@ -75,10 +75,10 @@ func (pxy *UdpProxy) Run() (remoteAddr string, err error) {
 	udpConn, errRet := net.ListenUDP("udp", addr)
 	if errRet != nil {
 		err = errRet
-		pxy.Warn("listen udp port error: %v", err)
+		xl.Warn("listen udp port error: %v", err)
 		return
 	}
-	pxy.Info("udp proxy listen port [%d]", pxy.cfg.RemotePort)
+	xl.Info("udp proxy listen port [%d]", pxy.cfg.RemotePort)
 
 	pxy.udpConn = udpConn
 	pxy.sendCh = make(chan *msg.UdpPacket, 1024)
@@ -92,11 +92,11 @@ func (pxy *UdpProxy) Run() (remoteAddr string, err error) {
 				rawMsg msg.Message
 				errRet error
 			)
-			pxy.Trace("loop waiting message from udp workConn")
+			xl.Trace("loop waiting message from udp workConn")
 			// client will send heartbeat in workConn for keeping alive
 			conn.SetReadDeadline(time.Now().Add(time.Duration(60) * time.Second))
 			if rawMsg, errRet = msg.ReadMsg(conn); errRet != nil {
-				pxy.Warn("read from workConn for udp error: %v", errRet)
+				xl.Warn("read from workConn for udp error: %v", errRet)
 				conn.Close()
 				// notify proxy to start a new work connection
 				// ignore error here, it means the proxy is closed
@@ -108,11 +108,11 @@ func (pxy *UdpProxy) Run() (remoteAddr string, err error) {
 			conn.SetReadDeadline(time.Time{})
 			switch m := rawMsg.(type) {
 			case *msg.Ping:
-				pxy.Trace("udp work conn get ping message")
+				xl.Trace("udp work conn get ping message")
 				continue
 			case *msg.UdpPacket:
 				if errRet := errors.PanicToError(func() {
-					pxy.Trace("get udp message from workConn: %s", m.Content)
+					xl.Trace("get udp message from workConn: %s", m.Content)
 					pxy.readCh <- m
 					pxy.statsCollector.Mark(stats.TypeAddTrafficOut, &stats.AddTrafficOutPayload{
 						ProxyName:    pxy.GetName(),
@@ -120,7 +120,7 @@ func (pxy *UdpProxy) Run() (remoteAddr string, err error) {
 					})
 				}); errRet != nil {
 					conn.Close()
-					pxy.Info("reader goroutine for udp work connection closed")
+					xl.Info("reader goroutine for udp work connection closed")
 					return
 				}
 			}
@@ -134,15 +134,15 @@ func (pxy *UdpProxy) Run() (remoteAddr string, err error) {
 			select {
 			case udpMsg, ok := <-pxy.sendCh:
 				if !ok {
-					pxy.Info("sender goroutine for udp work connection closed")
+					xl.Info("sender goroutine for udp work connection closed")
 					return
 				}
 				if errRet = msg.WriteMsg(conn, udpMsg); errRet != nil {
-					pxy.Info("sender goroutine for udp work connection closed: %v", errRet)
+					xl.Info("sender goroutine for udp work connection closed: %v", errRet)
 					conn.Close()
 					return
 				} else {
-					pxy.Trace("send message to udp workConn: %s", udpMsg.Content)
+					xl.Trace("send message to udp workConn: %s", udpMsg.Content)
 					pxy.statsCollector.Mark(stats.TypeAddTrafficIn, &stats.AddTrafficInPayload{
 						ProxyName:    pxy.GetName(),
 						TrafficBytes: int64(len(udpMsg.Content)),
@@ -150,7 +150,7 @@ func (pxy *UdpProxy) Run() (remoteAddr string, err error) {
 					continue
 				}
 			case <-ctx.Done():
-				pxy.Info("sender goroutine for udp work connection closed")
+				xl.Info("sender goroutine for udp work connection closed")
 				return
 			}
 		}

server/proxy/xtcp.go

@@ -31,8 +31,10 @@ type XtcpProxy struct {
 }
 
 func (pxy *XtcpProxy) Run() (remoteAddr string, err error) {
+	xl := pxy.xl
+
 	if pxy.rc.NatHoleController == nil {
-		pxy.Error("udp port for xtcp is not specified.")
+		xl.Error("udp port for xtcp is not specified.")
 		err = fmt.Errorf("xtcp is not supported in frps")
 		return
 	}
@@ -53,7 +55,7 @@ func (pxy *XtcpProxy) Run() (remoteAddr string, err error) {
 				}
 				errRet = msg.WriteMsg(workConn, m)
 				if errRet != nil {
-					pxy.Warn("write nat hole sid package error, %v", errRet)
+					xl.Warn("write nat hole sid package error, %v", errRet)
 					workConn.Close()
 					break
 				}
@@ -61,12 +63,12 @@ func (pxy *XtcpProxy) Run() (remoteAddr string, err error) {
 				go func() {
 					raw, errRet := msg.ReadMsg(workConn)
 					if errRet != nil {
-						pxy.Warn("read nat hole client ok package error: %v", errRet)
+						xl.Warn("read nat hole client ok package error: %v", errRet)
 						workConn.Close()
 						return
 					}
 					if _, ok := raw.(*msg.NatHoleClientDetectOK); !ok {
-						pxy.Warn("read nat hole client ok package format error")
+						xl.Warn("read nat hole client ok package format error")
 						workConn.Close()
 						return
 					}

server/service.go

@@ -16,6 +16,7 @@ package server
 
 import (
 	"bytes"
+	"context"
 	"crypto/rand"
 	"crypto/rsa"
 	"crypto/tls"
@@ -29,7 +30,7 @@ import (
 	"time"
 
 	"github.com/fatedier/frp/assets"
-	"github.com/fatedier/frp/g"
+	"github.com/fatedier/frp/models/config"
 	"github.com/fatedier/frp/models/msg"
 	"github.com/fatedier/frp/models/nathole"
 	"github.com/fatedier/frp/server/controller"
@@ -42,6 +43,7 @@ import (
 	"github.com/fatedier/frp/utils/util"
 	"github.com/fatedier/frp/utils/version"
 	"github.com/fatedier/frp/utils/vhost"
+	"github.com/fatedier/frp/utils/xlog"
 
 	"github.com/fatedier/golib/net/mux"
 	fmux "github.com/hashicorp/yamux"
@@ -51,24 +53,22 @@ const (
 	connReadTimeout time.Duration = 10 * time.Second
 )
 
-var ServerService *Service
-
 // Server service
 type Service struct {
 	// Dispatch connections to different handlers listen on same port
 	muxer *mux.Mux
 
 	// Accept connections from client
-	listener frpNet.Listener
+	listener net.Listener
 
 	// Accept connections using kcp
-	kcpListener frpNet.Listener
+	kcpListener net.Listener
 
 	// Accept connections using websocket
-	websocketListener frpNet.Listener
+	websocketListener net.Listener
 
 	// Accept frp tls connections
-	tlsListener frpNet.Listener
+	tlsListener net.Listener
 
 	// Manage all controllers
 	ctlManager *ControlManager
@@ -76,6 +76,9 @@ type Service struct {
 	// Manage all proxies
 	pxyManager *proxy.ProxyManager
 
+	// HTTP vhost router
+	httpVhostRouter *vhost.VhostRouters
+
 	// All resource managers and controllers
 	rc *controller.ResourceController
 
@@ -83,10 +86,11 @@ type Service struct {
 	statsCollector stats.Collector
 
 	tlsConfig *tls.Config
+
+	cfg config.ServerCommonConf
 }
 
-func NewService() (svr *Service, err error) {
-	cfg := &g.GlbServerCfg.ServerCommonConf
+func NewService(cfg config.ServerCommonConf) (svr *Service, err error) {
 	svr = &Service{
 		ctlManager: NewControlManager(),
 		pxyManager: proxy.NewProxyManager(),
@@ -95,18 +99,16 @@ func NewService() (svr *Service, err error) {
 			TcpPortManager: ports.NewPortManager("tcp", cfg.ProxyBindAddr, cfg.AllowPorts),
 			UdpPortManager: ports.NewPortManager("udp", cfg.ProxyBindAddr, cfg.AllowPorts),
 		},
-		tlsConfig: generateTLSConfig(),
+		httpVhostRouter: vhost.NewVhostRouters(),
+		tlsConfig:       generateTLSConfig(),
+		cfg:             cfg,
 	}
 
 	// Init group controller
 	svr.rc.TcpGroupCtl = group.NewTcpGroupCtl(svr.rc.TcpPortManager)
 
-	// Init assets
-	err = assets.Load(cfg.AssetsDir)
-	if err != nil {
-		err = fmt.Errorf("Load assets error: %v", err)
-		return
-	}
+	// Init HTTP group controller
+	svr.rc.HTTPGroupCtl = group.NewHTTPGroupController(svr.httpVhostRouter)
 
 	// Init 404 not found page
 	vhost.NotFoundPagePath = cfg.Custom404Page
@@ -135,7 +137,7 @@ func NewService() (svr *Service, err error) {
 	go svr.muxer.Serve()
 	ln = svr.muxer.DefaultListener()
 
-	svr.listener = frpNet.WrapLogListener(ln)
+	svr.listener = ln
 	log.Info("frps tcp listen on %s:%d", cfg.BindAddr, cfg.BindPort)
 
 	// Listen for accepting connections from client using kcp protocol.
@@ -159,7 +161,7 @@ func NewService() (svr *Service, err error) {
 	if cfg.VhostHttpPort > 0 {
 		rp := vhost.NewHttpReverseProxy(vhost.HttpReverseProxyOptions{
 			ResponseHeaderTimeoutS: cfg.VhostHttpTimeout,
-		})
+		}, svr.httpVhostRouter)
 		svr.rc.HttpReverseProxy = rp
 
 		address := fmt.Sprintf("%s:%d", cfg.ProxyBindAddr, cfg.VhostHttpPort)
@@ -194,7 +196,7 @@ func NewService() (svr *Service, err error) {
 			}
 		}
 
-		svr.rc.VhostHttpsMuxer, err = vhost.NewHttpsMuxer(frpNet.WrapLogListener(l), 30*time.Second)
+		svr.rc.VhostHttpsMuxer, err = vhost.NewHttpsMuxer(l, 30*time.Second)
 		if err != nil {
 			err = fmt.Errorf("Create vhost httpsMuxer error, %v", err)
 			return
@@ -203,10 +205,9 @@ func NewService() (svr *Service, err error) {
 	}
 
 	// frp tls listener
-	tlsListener := svr.muxer.Listen(1, 1, func(data []byte) bool {
+	svr.tlsListener = svr.muxer.Listen(1, 1, func(data []byte) bool {
 		return int(data[0]) == frpNet.FRP_TLS_HEAD_BYTE
 	})
-	svr.tlsListener = frpNet.WrapLogListener(tlsListener)
 
 	// Create nat hole controller.
 	if cfg.BindUdpPort > 0 {
@@ -224,6 +225,13 @@ func NewService() (svr *Service, err error) {
 	var statsEnable bool
 	// Create dashboard web server.
 	if cfg.DashboardPort > 0 {
+		// Init dashboard assets
+		err = assets.Load(cfg.AssetsDir)
+		if err != nil {
+			err = fmt.Errorf("Load assets error: %v", err)
+			return
+		}
+
 		err = svr.RunDashboardServer(cfg.DashboardAddr, cfg.DashboardPort)
 		if err != nil {
 			err = fmt.Errorf("Create dashboard web server error, %v", err)
@@ -241,7 +249,7 @@ func (svr *Service) Run() {
 	if svr.rc.NatHoleController != nil {
 		go svr.rc.NatHoleController.Run()
 	}
-	if g.GlbServerCfg.KcpBindPort > 0 {
+	if svr.cfg.KcpBindPort > 0 {
 		go svr.HandleListener(svr.kcpListener)
 	}
 
@@ -251,7 +259,7 @@ func (svr *Service) Run() {
 	svr.HandleListener(svr.listener)
 }
 
-func (svr *Service) HandleListener(l frpNet.Listener) {
+func (svr *Service) HandleListener(l net.Listener) {
 	// Listen for incoming connections from client.
 	for {
 		c, err := l.Accept()
@@ -259,11 +267,23 @@ func (svr *Service) HandleListener(l frpNet.Listener) {
 			log.Warn("Listener for incoming connections from client closed")
 			return
 		}
-		c = frpNet.CheckAndEnableTLSServerConn(c, svr.tlsConfig)
+		// inject xlog object into net.Conn context
+		xl := xlog.New()
+		c = frpNet.NewContextConn(c, xlog.NewContext(context.Background(), xl))
+
+		log.Trace("start check TLS connection...")
+		originConn := c
+		c, err = frpNet.CheckAndEnableTLSServerConnWithTimeout(c, svr.tlsConfig, connReadTimeout)
+		if err != nil {
+			log.Warn("CheckAndEnableTLSServerConnWithTimeout error: %v", err)
+			originConn.Close()
+			continue
+		}
+		log.Trace("success check TLS connection")
 
 		// Start a new goroutine for dealing connections.
-		go func(frpConn frpNet.Conn) {
-			dealFn := func(conn frpNet.Conn) {
+		go func(frpConn net.Conn) {
+			dealFn := func(conn net.Conn) {
 				var rawMsg msg.Message
 				conn.SetReadDeadline(time.Now().Add(connReadTimeout))
 				if rawMsg, err = msg.ReadMsg(conn); err != nil {
@@ -279,7 +299,7 @@ func (svr *Service) HandleListener(l frpNet.Listener) {
 					// If login failed, send error message there.
 					// Otherwise send success message in control's work goroutine.
 					if err != nil {
-						conn.Warn("%v", err)
+						xl.Warn("register control error: %v", err)
 						msg.WriteMsg(conn, &msg.LoginResp{
 							Version: version.Full(),
 							Error:   err.Error(),
@@ -290,7 +310,7 @@ func (svr *Service) HandleListener(l frpNet.Listener) {
 					svr.RegisterWorkConn(conn, m)
 				case *msg.NewVisitorConn:
 					if err = svr.RegisterVisitorConn(conn, m); err != nil {
-						conn.Warn("%v", err)
+						xl.Warn("register visitor conn error: %v", err)
 						msg.WriteMsg(conn, &msg.NewVisitorConnResp{
 							ProxyName: m.ProxyName,
 							Error:     err.Error(),
@@ -308,7 +328,7 @@ func (svr *Service) HandleListener(l frpNet.Listener) {
 				}
 			}
 
-			if g.GlbServerCfg.TcpMux {
+			if svr.cfg.TcpMux {
 				fmuxCfg := fmux.DefaultConfig()
 				fmuxCfg.KeepAliveInterval = 20 * time.Second
 				fmuxCfg.LogOutput = ioutil.Discard
@@ -326,8 +346,7 @@ func (svr *Service) HandleListener(l frpNet.Listener) {
 						session.Close()
 						return
 					}
-					wrapConn := frpNet.WrapConn(stream)
-					go dealFn(wrapConn)
+					go dealFn(stream)
 				}
 			} else {
 				dealFn(frpConn)
@@ -336,22 +355,7 @@ func (svr *Service) HandleListener(l frpNet.Listener) {
 	}
 }
 
-func (svr *Service) RegisterControl(ctlConn frpNet.Conn, loginMsg *msg.Login) (err error) {
-	ctlConn.Info("client login info: ip [%s] version [%s] hostname [%s] os [%s] arch [%s]",
-		ctlConn.RemoteAddr().String(), loginMsg.Version, loginMsg.Hostname, loginMsg.Os, loginMsg.Arch)
-
-	// Check client version.
-	if ok, msg := version.Compat(loginMsg.Version); !ok {
-		err = fmt.Errorf("%s", msg)
-		return
-	}
-
-	// Check auth.
-	if util.GetAuthKey(g.GlbServerCfg.Token, loginMsg.Timestamp) != loginMsg.PrivilegeKey {
-		err = fmt.Errorf("authorization failed")
-		return
-	}
-
+func (svr *Service) RegisterControl(ctlConn net.Conn, loginMsg *msg.Login) (err error) {
 	// If client's RunId is empty, it's a new client, we just create a new controller.
 	// Otherwise, we check if there is one controller has the same run id. If so, we release previous controller and start new one.
 	if loginMsg.RunId == "" {
@@ -361,13 +365,31 @@ func (svr *Service) RegisterControl(ctlConn frpNet.Conn, loginMsg *msg.Login) (e
 		}
 	}
 
-	ctl := NewControl(svr.rc, svr.pxyManager, svr.statsCollector, ctlConn, loginMsg)
+	ctx := frpNet.NewContextFromConn(ctlConn)
+	xl := xlog.FromContextSafe(ctx)
+	xl.AppendPrefix(loginMsg.RunId)
+	ctx = xlog.NewContext(ctx, xl)
+	xl.Info("client login info: ip [%s] version [%s] hostname [%s] os [%s] arch [%s]",
+		ctlConn.RemoteAddr().String(), loginMsg.Version, loginMsg.Hostname, loginMsg.Os, loginMsg.Arch)
+
+	// Check client version.
+	if ok, msg := version.Compat(loginMsg.Version); !ok {
+		err = fmt.Errorf("%s", msg)
+		return
+	}
+
+	// Check auth.
+	if util.GetAuthKey(svr.cfg.Token, loginMsg.Timestamp) != loginMsg.PrivilegeKey {
+		err = fmt.Errorf("authorization failed")
+		return
+	}
+
+	ctl := NewControl(ctx, svr.rc, svr.pxyManager, svr.statsCollector, ctlConn, loginMsg, svr.cfg)
 
 	if oldCtl := svr.ctlManager.Add(loginMsg.RunId, ctl); oldCtl != nil {
 		oldCtl.allShutdown.WaitDone()
 	}
 
-	ctlConn.AddLogPrefix(loginMsg.RunId)
 	ctl.Start()
 
 	// for statistics
@@ -382,17 +404,18 @@ func (svr *Service) RegisterControl(ctlConn frpNet.Conn, loginMsg *msg.Login) (e
 }
 
 // RegisterWorkConn register a new work connection to control and proxies need it.
-func (svr *Service) RegisterWorkConn(workConn frpNet.Conn, newMsg *msg.NewWorkConn) {
+func (svr *Service) RegisterWorkConn(workConn net.Conn, newMsg *msg.NewWorkConn) {
+	xl := frpNet.NewLogFromConn(workConn)
 	ctl, exist := svr.ctlManager.GetById(newMsg.RunId)
 	if !exist {
-		workConn.Warn("No client control found for run id [%s]", newMsg.RunId)
+		xl.Warn("No client control found for run id [%s]", newMsg.RunId)
 		return
 	}
 	ctl.RegisterWorkConn(workConn)
 	return
 }
 
-func (svr *Service) RegisterVisitorConn(visitorConn frpNet.Conn, newMsg *msg.NewVisitorConn) error {
+func (svr *Service) RegisterVisitorConn(visitorConn net.Conn, newMsg *msg.NewVisitorConn) error {
 	return svr.rc.VisitorManager.NewConn(newMsg.ProxyName, visitorConn, newMsg.Timestamp, newMsg.SignKey,
 		newMsg.UseEncryption, newMsg.UseCompression)
 }

tests/ci/auto_test_frpc.ini

@@ -2,8 +2,7 @@
 server_addr = 127.0.0.1
 server_port = 10700
 log_file = console
-# debug, info, warn, error
-log_level = debug
+log_level = trace
 token = 123456
 admin_port = 10600
 admin_user = abc

tests/ci/auto_test_frps.ini

@@ -2,8 +2,7 @@
 bind_addr = 0.0.0.0
 bind_port = 10700
 vhost_http_port = 10804
-log_file = console
-log_level = debug
+log_level = trace
 token = 123456
 allow_ports = 10000-20000,20002,30000-50000
 subdomain_host = sub.com

tests/ci/health/health_test.go

@@ -67,12 +67,26 @@ custom_domains = test2.com
 health_check_type = http
 health_check_interval_s = 1
 health_check_url = /health
+
+[http3]
+type = http
+local_port = 15005
+custom_domains = test.balancing.com
+group = test-balancing
+group_key = 123
+
+[http4]
+type = http
+local_port = 15006
+custom_domains = test.balancing.com
+group = test-balancing
+group_key = 123
 `
 
 func TestHealthCheck(t *testing.T) {
 	assert := assert.New(t)
 
-	// ****** start backgroud services ******
+	// ****** start background services ******
 	echoSvc1 := mock.NewEchoServer(15001, 1, "echo1")
 	err := echoSvc1.Start()
 	if assert.NoError(err) {
@@ -124,6 +138,22 @@ func TestHealthCheck(t *testing.T) {
 		defer httpSvc2.Stop()
 	}
 
+	httpSvc3 := mock.NewHttpServer(15005, func(w http.ResponseWriter, r *http.Request) {
+		w.Write([]byte("http3"))
+	})
+	err = httpSvc3.Start()
+	if assert.NoError(err) {
+		defer httpSvc3.Stop()
+	}
+
+	httpSvc4 := mock.NewHttpServer(15006, func(w http.ResponseWriter, r *http.Request) {
+		w.Write([]byte("http4"))
+	})
+	err = httpSvc4.Start()
+	if assert.NoError(err) {
+		defer httpSvc4.Stop()
+	}
+
 	time.Sleep(200 * time.Millisecond)
 
 	// ****** start frps and frpc ******
@@ -244,4 +274,20 @@ func TestHealthCheck(t *testing.T) {
 	assert.NoError(err)
 	assert.Equal(200, code)
 	assert.Equal("http2", body)
+
+	// ****** load balancing type http ******
+	result = make([]string, 0)
+
+	code, body, _, err = util.SendHttpMsg("GET", "http://127.0.0.1:14000/xxx", "test.balancing.com", nil, "")
+	assert.NoError(err)
+	assert.Equal(200, code)
+	result = append(result, body)
+
+	code, body, _, err = util.SendHttpMsg("GET", "http://127.0.0.1:14000/xxx", "test.balancing.com", nil, "")
+	assert.NoError(err)
+	assert.Equal(200, code)
+	result = append(result, body)
+
+	assert.Contains(result, "http3")
+	assert.Contains(result, "http4")
 }

tests/mock/echo_server.go

@@ -11,7 +11,7 @@ import (
 )
 
 type EchoServer struct {
-	l frpNet.Listener
+	l net.Listener
 
 	port        int
 	repeatedNum int
@@ -30,7 +30,7 @@ func NewEchoServer(port int, repeatedNum int, specifyStr string) *EchoServer {
 }
 
 func (es *EchoServer) Start() error {
-	l, err := frpNet.ListenTcp("127.0.0.1", es.port)
+	l, err := net.Listen("tcp", fmt.Sprintf("127.0.0.1:%d", es.port))
 	if err != nil {
 		fmt.Printf("echo server listen error: %v\n", err)
 		return err

tests/util/util.go

@@ -14,7 +14,6 @@ import (
 	"time"
 
 	"github.com/fatedier/frp/client"
-	frpNet "github.com/fatedier/frp/utils/net"
 )
 
 func GetProxyStatus(statusAddr string, user string, passwd string, name string) (status *client.ProxyStatusResp, err error) {
@@ -28,51 +27,51 @@ func GetProxyStatus(statusAddr string, user string, passwd string, name string)
 	resp, err := http.DefaultClient.Do(req)
 	if err != nil {
 		return status, err
-	} else {
-		if resp.StatusCode != 200 {
-			return status, fmt.Errorf("admin api status code [%d]", resp.StatusCode)
-		}
-		defer resp.Body.Close()
-		body, err := ioutil.ReadAll(resp.Body)
-		if err != nil {
-			return status, err
-		}
-		allStatus := &client.StatusResp{}
-		err = json.Unmarshal(body, &allStatus)
-		if err != nil {
-			return status, fmt.Errorf("unmarshal http response error: %s", strings.TrimSpace(string(body)))
-		}
-		for _, s := range allStatus.Tcp {
-			if s.Name == name {
-				return &s, nil
-			}
-		}
-		for _, s := range allStatus.Udp {
-			if s.Name == name {
-				return &s, nil
-			}
-		}
-		for _, s := range allStatus.Http {
-			if s.Name == name {
-				return &s, nil
-			}
-		}
-		for _, s := range allStatus.Https {
-			if s.Name == name {
-				return &s, nil
-			}
-		}
-		for _, s := range allStatus.Stcp {
-			if s.Name == name {
-				return &s, nil
-			}
-		}
-		for _, s := range allStatus.Xtcp {
-			if s.Name == name {
-				return &s, nil
-			}
+	}
+	defer resp.Body.Close()
+	if resp.StatusCode != 200 {
+		return status, fmt.Errorf("admin api status code [%d]", resp.StatusCode)
+	}
+	body, err := ioutil.ReadAll(resp.Body)
+	if err != nil {
+		return status, err
+	}
+	allStatus := &client.StatusResp{}
+	err = json.Unmarshal(body, &allStatus)
+	if err != nil {
+		return status, fmt.Errorf("unmarshal http response error: %s", strings.TrimSpace(string(body)))
+	}
+	for _, s := range allStatus.Tcp {
+		if s.Name == name {
+			return &s, nil
 		}
 	}
+	for _, s := range allStatus.Udp {
+		if s.Name == name {
+			return &s, nil
+		}
+	}
+	for _, s := range allStatus.Http {
+		if s.Name == name {
+			return &s, nil
+		}
+	}
+	for _, s := range allStatus.Https {
+		if s.Name == name {
+			return &s, nil
+		}
+	}
+	for _, s := range allStatus.Stcp {
+		if s.Name == name {
+			return &s, nil
+		}
+	}
+	for _, s := range allStatus.Xtcp {
+		if s.Name == name {
+			return &s, nil
+		}
+	}
+
 	return status, errors.New("no proxy status found")
 }
 
@@ -87,18 +86,18 @@ func ReloadConf(reloadAddr string, user string, passwd string) error {
 	resp, err := http.DefaultClient.Do(req)
 	if err != nil {
 		return err
-	} else {
-		if resp.StatusCode != 200 {
-			return fmt.Errorf("admin api status code [%d]", resp.StatusCode)
-		}
-		defer resp.Body.Close()
-		io.Copy(ioutil.Discard, resp.Body)
 	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != 200 {
+		return fmt.Errorf("admin api status code [%d]", resp.StatusCode)
+	}
+	io.Copy(ioutil.Discard, resp.Body)
 	return nil
 }
 
 func SendTcpMsg(addr string, msg string) (res string, err error) {
-	c, err := frpNet.ConnectTcpServer(addr)
+	c, err := net.Dial("tcp", addr)
 	if err != nil {
 		err = fmt.Errorf("connect to tcp server error: %v", err)
 		return

utils/limit/reader.go

@@ -0,0 +1,51 @@
+// Copyright 2019 fatedier, fatedier@gmail.com
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package limit
+
+import (
+	"context"
+	"io"
+
+	"golang.org/x/time/rate"
+)
+
+type Reader struct {
+	r       io.Reader
+	limiter *rate.Limiter
+}
+
+func NewReader(r io.Reader, limiter *rate.Limiter) *Reader {
+	return &Reader{
+		r:       r,
+		limiter: limiter,
+	}
+}
+
+func (r *Reader) Read(p []byte) (n int, err error) {
+	b := r.limiter.Burst()
+	if b < len(p) {
+		p = p[:b]
+	}
+	n, err = r.r.Read(p)
+	if err != nil {
+		return
+	}
+
+	err = r.limiter.WaitN(context.Background(), n)
+	if err != nil {
+		return
+	}
+	return
+}

utils/limit/writer.go

@@ -0,0 +1,60 @@
+// Copyright 2019 fatedier, fatedier@gmail.com
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package limit
+
+import (
+	"context"
+	"io"
+
+	"golang.org/x/time/rate"
+)
+
+type Writer struct {
+	w       io.Writer
+	limiter *rate.Limiter
+}
+
+func NewWriter(w io.Writer, limiter *rate.Limiter) *Writer {
+	return &Writer{
+		w:       w,
+		limiter: limiter,
+	}
+}
+
+func (w *Writer) Write(p []byte) (n int, err error) {
+	var nn int
+	b := w.limiter.Burst()
+	for {
+		end := len(p)
+		if end == 0 {
+			break
+		}
+		if b < len(p) {
+			end = b
+		}
+		err = w.limiter.WaitN(context.Background(), end)
+		if err != nil {
+			return
+		}
+
+		nn, err = w.w.Write(p[:end])
+		n += nn
+		if err != nil {
+			return
+		}
+		p = p[end:]
+	}
+	return
+}

utils/log/log.go

@@ -29,16 +29,20 @@ func init() {
 	Log.SetLogFuncCallDepth(Log.GetLogFuncCallDepth() + 1)
 }
 
-func InitLog(logWay string, logFile string, logLevel string, maxdays int64) {
-	SetLogFile(logWay, logFile, maxdays)
+func InitLog(logWay string, logFile string, logLevel string, maxdays int64, disableLogColor bool) {
+	SetLogFile(logWay, logFile, maxdays, disableLogColor)
 	SetLogLevel(logLevel)
 }
 
 // SetLogFile to configure log params
 // logWay: file or console
-func SetLogFile(logWay string, logFile string, maxdays int64) {
+func SetLogFile(logWay string, logFile string, maxdays int64, disableLogColor bool) {
 	if logWay == "console" {
-		Log.SetLogger("console", "")
+		params := ""
+		if disableLogColor {
+			params = fmt.Sprintf(`{"color": false}`)
+		}
+		Log.SetLogger("console", params)
 	} else {
 		params := fmt.Sprintf(`{"filename": "%s", "maxdays": %d}`, logFile, maxdays)
 		Log.SetLogger("file", params)
@@ -87,71 +91,3 @@ func Debug(format string, v ...interface{}) {
 func Trace(format string, v ...interface{}) {
 	Log.Trace(format, v...)
 }
-
-// Logger is the log interface
-type Logger interface {
-	AddLogPrefix(string)
-	GetPrefixStr() string
-	GetAllPrefix() []string
-	ClearLogPrefix()
-	Error(string, ...interface{})
-	Warn(string, ...interface{})
-	Info(string, ...interface{})
-	Debug(string, ...interface{})
-	Trace(string, ...interface{})
-}
-
-type PrefixLogger struct {
-	prefix    string
-	allPrefix []string
-}
-
-func NewPrefixLogger(prefix string) *PrefixLogger {
-	logger := &PrefixLogger{
-		allPrefix: make([]string, 0),
-	}
-	logger.AddLogPrefix(prefix)
-	return logger
-}
-
-func (pl *PrefixLogger) AddLogPrefix(prefix string) {
-	if len(prefix) == 0 {
-		return
-	}
-
-	pl.prefix += "[" + prefix + "] "
-	pl.allPrefix = append(pl.allPrefix, prefix)
-}
-
-func (pl *PrefixLogger) GetPrefixStr() string {
-	return pl.prefix
-}
-
-func (pl *PrefixLogger) GetAllPrefix() []string {
-	return pl.allPrefix
-}
-
-func (pl *PrefixLogger) ClearLogPrefix() {
-	pl.prefix = ""
-	pl.allPrefix = make([]string, 0)
-}
-
-func (pl *PrefixLogger) Error(format string, v ...interface{}) {
-	Log.Error(pl.prefix+format, v...)
-}
-
-func (pl *PrefixLogger) Warn(format string, v ...interface{}) {
-	Log.Warn(pl.prefix+format, v...)
-}
-
-func (pl *PrefixLogger) Info(format string, v ...interface{}) {
-	Log.Info(pl.prefix+format, v...)
-}
-
-func (pl *PrefixLogger) Debug(format string, v ...interface{}) {
-	Log.Debug(pl.prefix+format, v...)
-}
-
-func (pl *PrefixLogger) Trace(format string, v ...interface{}) {
-	Log.Trace(pl.prefix+format, v...)
-}

utils/net/conn.go

@@ -15,6 +15,7 @@
 package net
 
 import (
+	"context"
 	"crypto/tls"
 	"errors"
 	"fmt"
@@ -23,41 +24,64 @@ import (
 	"sync/atomic"
 	"time"
 
-	"github.com/fatedier/frp/utils/log"
-
+	"github.com/fatedier/frp/utils/xlog"
 	gnet "github.com/fatedier/golib/net"
 	kcp "github.com/fatedier/kcp-go"
 )
 
-// Conn is the interface of connections used in frp.
-type Conn interface {
-	net.Conn
-	log.Logger
+type ContextGetter interface {
+	Context() context.Context
 }
 
-type WrapLogConn struct {
-	net.Conn
-	log.Logger
+type ContextSetter interface {
+	WithContext(ctx context.Context)
 }
 
-func WrapConn(c net.Conn) Conn {
-	return &WrapLogConn{
-		Conn:   c,
-		Logger: log.NewPrefixLogger(""),
+func NewLogFromConn(conn net.Conn) *xlog.Logger {
+	if c, ok := conn.(ContextGetter); ok {
+		return xlog.FromContextSafe(c.Context())
 	}
+	return xlog.New()
+}
+
+func NewContextFromConn(conn net.Conn) context.Context {
+	if c, ok := conn.(ContextGetter); ok {
+		return c.Context()
+	}
+	return context.Background()
+}
+
+// ContextConn is the connection with context
+type ContextConn struct {
+	net.Conn
+
+	ctx context.Context
+}
+
+func NewContextConn(c net.Conn, ctx context.Context) *ContextConn {
+	return &ContextConn{
+		Conn: c,
+		ctx:  ctx,
+	}
+}
+
+func (c *ContextConn) WithContext(ctx context.Context) {
+	c.ctx = ctx
+}
+
+func (c *ContextConn) Context() context.Context {
+	return c.ctx
 }
 
 type WrapReadWriteCloserConn struct {
 	io.ReadWriteCloser
-	log.Logger
 
 	underConn net.Conn
 }
 
-func WrapReadWriteCloserToConn(rwc io.ReadWriteCloser, underConn net.Conn) Conn {
+func WrapReadWriteCloserToConn(rwc io.ReadWriteCloser, underConn net.Conn) net.Conn {
 	return &WrapReadWriteCloserConn{
 		ReadWriteCloser: rwc,
-		Logger:          log.NewPrefixLogger(""),
 		underConn:       underConn,
 	}
 }
@@ -99,7 +123,6 @@ func (conn *WrapReadWriteCloserConn) SetWriteDeadline(t time.Time) error {
 
 type CloseNotifyConn struct {
 	net.Conn
-	log.Logger
 
 	// 1 means closed
 	closeFlag int32
@@ -108,10 +131,9 @@ type CloseNotifyConn struct {
 }
 
 // closeFn will be only called once
-func WrapCloseNotifyConn(c net.Conn, closeFn func()) Conn {
+func WrapCloseNotifyConn(c net.Conn, closeFn func()) net.Conn {
 	return &CloseNotifyConn{
 		Conn:    c,
-		Logger:  log.NewPrefixLogger(""),
 		closeFn: closeFn,
 	}
 }
@@ -128,7 +150,7 @@ func (cc *CloseNotifyConn) Close() (err error) {
 }
 
 type StatsConn struct {
-	Conn
+	net.Conn
 
 	closed     int64 // 1 means closed
 	totalRead  int64
@@ -136,7 +158,7 @@ type StatsConn struct {
 	statsFunc  func(totalRead, totalWrite int64)
 }
 
-func WrapStatsConn(conn Conn, statsFunc func(total, totalWrite int64)) *StatsConn {
+func WrapStatsConn(conn net.Conn, statsFunc func(total, totalWrite int64)) *StatsConn {
 	return &StatsConn{
 		Conn:      conn,
 		statsFunc: statsFunc,
@@ -166,10 +188,10 @@ func (statsConn *StatsConn) Close() (err error) {
 	return
 }
 
-func ConnectServer(protocol string, addr string) (c Conn, err error) {
+func ConnectServer(protocol string, addr string) (c net.Conn, err error) {
 	switch protocol {
 	case "tcp":
-		return ConnectTcpServer(addr)
+		return net.Dial("tcp", addr)
 	case "kcp":
 		kcpConn, errRet := kcp.DialWithOptions(addr, nil, 10, 3)
 		if errRet != nil {
@@ -184,21 +206,17 @@ func ConnectServer(protocol string, addr string) (c Conn, err error) {
 		kcpConn.SetACKNoDelay(false)
 		kcpConn.SetReadBuffer(4194304)
 		kcpConn.SetWriteBuffer(4194304)
-		c = WrapConn(kcpConn)
+		c = kcpConn
 		return
 	default:
 		return nil, fmt.Errorf("unsupport protocol: %s", protocol)
 	}
 }
 
-func ConnectServerByProxy(proxyUrl string, protocol string, addr string) (c Conn, err error) {
+func ConnectServerByProxy(proxyURL string, protocol string, addr string) (c net.Conn, err error) {
 	switch protocol {
 	case "tcp":
-		var conn net.Conn
-		if conn, err = gnet.DialTcpByProxy(proxyUrl, addr); err != nil {
-			return
-		}
-		return WrapConn(conn), nil
+		return gnet.DialTcpByProxy(proxyURL, addr)
 	case "kcp":
 		// http proxy is not supported for kcp
 		return ConnectServer(protocol, addr)
@@ -209,7 +227,7 @@ func ConnectServerByProxy(proxyUrl string, protocol string, addr string) (c Conn
 	}
 }
 
-func ConnectServerByProxyWithTLS(proxyUrl string, protocol string, addr string, tlsConfig *tls.Config) (c Conn, err error) {
+func ConnectServerByProxyWithTLS(proxyUrl string, protocol string, addr string, tlsConfig *tls.Config) (c net.Conn, err error) {
 	c, err = ConnectServerByProxy(proxyUrl, protocol, addr)
 	if err != nil {
 		return

utils/net/kcp.go

@@ -18,17 +18,13 @@ import (
 	"fmt"
 	"net"
 
-	"github.com/fatedier/frp/utils/log"
-
 	kcp "github.com/fatedier/kcp-go"
 )
 
 type KcpListener struct {
-	net.Addr
 	listener  net.Listener
-	accept    chan Conn
+	acceptCh  chan net.Conn
 	closeFlag bool
-	log.Logger
 }
 
 func ListenKcp(bindAddr string, bindPort int) (l *KcpListener, err error) {
@@ -40,11 +36,9 @@ func ListenKcp(bindAddr string, bindPort int) (l *KcpListener, err error) {
 	listener.SetWriteBuffer(4194304)
 
 	l = &KcpListener{
-		Addr:      listener.Addr(),
 		listener:  listener,
-		accept:    make(chan Conn),
+		acceptCh:  make(chan net.Conn),
 		closeFlag: false,
-		Logger:    log.NewPrefixLogger(""),
 	}
 
 	go func() {
@@ -52,7 +46,7 @@ func ListenKcp(bindAddr string, bindPort int) (l *KcpListener, err error) {
 			conn, err := listener.AcceptKCP()
 			if err != nil {
 				if l.closeFlag {
-					close(l.accept)
+					close(l.acceptCh)
 					return
 				}
 				continue
@@ -64,14 +58,14 @@ func ListenKcp(bindAddr string, bindPort int) (l *KcpListener, err error) {
 			conn.SetWindowSize(1024, 1024)
 			conn.SetACKNoDelay(false)
 
-			l.accept <- WrapConn(conn)
+			l.acceptCh <- conn
 		}
 	}()
 	return l, err
 }
 
-func (l *KcpListener) Accept() (Conn, error) {
-	conn, ok := <-l.accept
+func (l *KcpListener) Accept() (net.Conn, error) {
+	conn, ok := <-l.acceptCh
 	if !ok {
 		return conn, fmt.Errorf("channel for kcp listener closed")
 	}
@@ -86,6 +80,10 @@ func (l *KcpListener) Close() error {
 	return nil
 }
 
+func (l *KcpListener) Addr() net.Addr {
+	return l.listener.Addr()
+}
+
 func NewKcpConnFromUdp(conn *net.UDPConn, connected bool, raddr string) (net.Conn, error) {
 	kcpConn, err := kcp.NewConnEx(1, connected, raddr, nil, 10, 3, conn)
 	if err != nil {

utils/net/listener.go

@@ -19,65 +19,34 @@ import (
 	"net"
 	"sync"
 
-	"github.com/fatedier/frp/utils/log"
-
 	"github.com/fatedier/golib/errors"
 )
 
-type Listener interface {
-	Accept() (Conn, error)
-	Close() error
-	log.Logger
-}
-
-type LogListener struct {
-	l net.Listener
-	net.Listener
-	log.Logger
-}
-
-func WrapLogListener(l net.Listener) Listener {
-	return &LogListener{
-		l:        l,
-		Listener: l,
-		Logger:   log.NewPrefixLogger(""),
-	}
-}
-
-func (logL *LogListener) Accept() (Conn, error) {
-	c, err := logL.l.Accept()
-	return WrapConn(c), err
-}
-
 // Custom listener
 type CustomListener struct {
-	conns  chan Conn
-	closed bool
-	mu     sync.Mutex
-
-	log.Logger
+	acceptCh chan net.Conn
+	closed   bool
+	mu       sync.Mutex
 }
 
 func NewCustomListener() *CustomListener {
 	return &CustomListener{
-		conns:  make(chan Conn, 64),
-		Logger: log.NewPrefixLogger(""),
+		acceptCh: make(chan net.Conn, 64),
 	}
 }
 
-func (l *CustomListener) Accept() (Conn, error) {
-	conn, ok := <-l.conns
+func (l *CustomListener) Accept() (net.Conn, error) {
+	conn, ok := <-l.acceptCh
 	if !ok {
 		return nil, fmt.Errorf("listener closed")
 	}
-	conn.AddLogPrefix(l.GetPrefixStr())
 	return conn, nil
 }
 
-func (l *CustomListener) PutConn(conn Conn) error {
+func (l *CustomListener) PutConn(conn net.Conn) error {
 	err := errors.PanicToError(func() {
 		select {
-		case l.conns <- conn:
+		case l.acceptCh <- conn:
 		default:
 			conn.Close()
 		}
@@ -89,7 +58,7 @@ func (l *CustomListener) Close() error {
 	l.mu.Lock()
 	defer l.mu.Unlock()
 	if !l.closed {
-		close(l.conns)
+		close(l.acceptCh)
 		l.closed = true
 	}
 	return nil

utils/net/tcp.go

@@ -1,111 +0,0 @@
-// Copyright 2016 fatedier, fatedier@gmail.com
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package net
-
-import (
-	"fmt"
-	"net"
-
-	"github.com/fatedier/frp/utils/log"
-)
-
-type TcpListener struct {
-	net.Addr
-	listener  net.Listener
-	accept    chan Conn
-	closeFlag bool
-	log.Logger
-}
-
-func ListenTcp(bindAddr string, bindPort int) (l *TcpListener, err error) {
-	tcpAddr, err := net.ResolveTCPAddr("tcp", fmt.Sprintf("%s:%d", bindAddr, bindPort))
-	if err != nil {
-		return l, err
-	}
-	listener, err := net.ListenTCP("tcp", tcpAddr)
-	if err != nil {
-		return l, err
-	}
-
-	l = &TcpListener{
-		Addr:      listener.Addr(),
-		listener:  listener,
-		accept:    make(chan Conn),
-		closeFlag: false,
-		Logger:    log.NewPrefixLogger(""),
-	}
-
-	go func() {
-		for {
-			conn, err := listener.AcceptTCP()
-			if err != nil {
-				if l.closeFlag {
-					close(l.accept)
-					return
-				}
-				continue
-			}
-
-			c := NewTcpConn(conn)
-			l.accept <- c
-		}
-	}()
-	return l, err
-}
-
-// Wait util get one new connection or listener is closed
-// if listener is closed, err returned.
-func (l *TcpListener) Accept() (Conn, error) {
-	conn, ok := <-l.accept
-	if !ok {
-		return conn, fmt.Errorf("channel for tcp listener closed")
-	}
-	return conn, nil
-}
-
-func (l *TcpListener) Close() error {
-	if !l.closeFlag {
-		l.closeFlag = true
-		l.listener.Close()
-	}
-	return nil
-}
-
-// Wrap for TCPConn.
-type TcpConn struct {
-	net.Conn
-	log.Logger
-}
-
-func NewTcpConn(conn net.Conn) (c *TcpConn) {
-	c = &TcpConn{
-		Conn:   conn,
-		Logger: log.NewPrefixLogger(""),
-	}
-	return
-}
-
-func ConnectTcpServer(addr string) (c Conn, err error) {
-	servertAddr, err := net.ResolveTCPAddr("tcp", addr)
-	if err != nil {
-		return
-	}
-	conn, err := net.DialTCP("tcp", nil, servertAddr)
-	if err != nil {
-		return
-	}
-	c = NewTcpConn(conn)
-	return
-}

utils/net/tls.go

@@ -17,6 +17,7 @@ package net
 import (
 	"crypto/tls"
 	"net"
+	"time"
 
 	gnet "github.com/fatedier/golib/net"
 )
@@ -25,20 +26,27 @@ var (
 	FRP_TLS_HEAD_BYTE = 0x17
 )
 
-func WrapTLSClientConn(c net.Conn, tlsConfig *tls.Config) (out Conn) {
+func WrapTLSClientConn(c net.Conn, tlsConfig *tls.Config) (out net.Conn) {
 	c.Write([]byte{byte(FRP_TLS_HEAD_BYTE)})
-	out = WrapConn(tls.Client(c, tlsConfig))
+	out = tls.Client(c, tlsConfig)
 	return
 }
 
-func CheckAndEnableTLSServerConn(c net.Conn, tlsConfig *tls.Config) (out Conn) {
-	sc, r := gnet.NewSharedConnSize(c, 1)
+func CheckAndEnableTLSServerConnWithTimeout(c net.Conn, tlsConfig *tls.Config, timeout time.Duration) (out net.Conn, err error) {
+	sc, r := gnet.NewSharedConnSize(c, 2)
 	buf := make([]byte, 1)
-	n, _ := r.Read(buf)
+	var n int
+	c.SetReadDeadline(time.Now().Add(timeout))
+	n, err = r.Read(buf)
+	c.SetReadDeadline(time.Time{})
+	if err != nil {
+		return
+	}
+
 	if n == 1 && int(buf[0]) == FRP_TLS_HEAD_BYTE {
-		out = WrapConn(tls.Server(c, tlsConfig))
+		out = tls.Server(c, tlsConfig)
 	} else {
-		out = WrapConn(sc)
+		out = sc
 	}
 	return
 }

utils/net/udp.go

@@ -21,8 +21,6 @@ import (
 	"sync"
 	"time"
 
-	"github.com/fatedier/frp/utils/log"
-
 	"github.com/fatedier/golib/pool"
 )
 
@@ -33,7 +31,6 @@ type UdpPacket struct {
 }
 
 type FakeUdpConn struct {
-	log.Logger
 	l *UdpListener
 
 	localAddr  net.Addr
@@ -47,7 +44,6 @@ type FakeUdpConn struct {
 
 func NewFakeUdpConn(l *UdpListener, laddr, raddr net.Addr) *FakeUdpConn {
 	fc := &FakeUdpConn{
-		Logger:     log.NewPrefixLogger(""),
 		l:          l,
 		localAddr:  laddr,
 		remoteAddr: raddr,
@@ -157,15 +153,13 @@ func (c *FakeUdpConn) SetWriteDeadline(t time.Time) error {
 }
 
 type UdpListener struct {
-	net.Addr
-	accept    chan Conn
+	addr      net.Addr
+	acceptCh  chan net.Conn
 	writeCh   chan *UdpPacket
 	readConn  net.Conn
 	closeFlag bool
 
 	fakeConns map[string]*FakeUdpConn
-
-	log.Logger
 }
 
 func ListenUDP(bindAddr string, bindPort int) (l *UdpListener, err error) {
@@ -176,11 +170,10 @@ func ListenUDP(bindAddr string, bindPort int) (l *UdpListener, err error) {
 	readConn, err := net.ListenUDP("udp", udpAddr)
 
 	l = &UdpListener{
-		Addr:      udpAddr,
-		accept:    make(chan Conn),
+		addr:      udpAddr,
+		acceptCh:  make(chan net.Conn),
 		writeCh:   make(chan *UdpPacket, 1000),
 		fakeConns: make(map[string]*FakeUdpConn),
-		Logger:    log.NewPrefixLogger(""),
 	}
 
 	// for reading
@@ -189,19 +182,19 @@ func ListenUDP(bindAddr string, bindPort int) (l *UdpListener, err error) {
 			buf := pool.GetBuf(1450)
 			n, remoteAddr, err := readConn.ReadFromUDP(buf)
 			if err != nil {
-				close(l.accept)
+				close(l.acceptCh)
 				close(l.writeCh)
 				return
 			}
 
 			fakeConn, exist := l.fakeConns[remoteAddr.String()]
 			if !exist || fakeConn.IsClosed() {
-				fakeConn = NewFakeUdpConn(l, l.Addr, remoteAddr)
+				fakeConn = NewFakeUdpConn(l, l.Addr(), remoteAddr)
 				l.fakeConns[remoteAddr.String()] = fakeConn
 			}
 			fakeConn.putPacket(buf[:n])
 
-			l.accept <- fakeConn
+			l.acceptCh <- fakeConn
 		}
 	}()
 
@@ -226,7 +219,6 @@ func (l *UdpListener) writeUdpPacket(packet *UdpPacket) (err error) {
 	defer func() {
 		if errRet := recover(); errRet != nil {
 			err = fmt.Errorf("udp write closed listener")
-			l.Info("udp write closed listener")
 		}
 	}()
 	l.writeCh <- packet
@@ -243,8 +235,8 @@ func (l *UdpListener) WriteMsg(buf []byte, remoteAddr *net.UDPAddr) (err error)
 	return
 }
 
-func (l *UdpListener) Accept() (Conn, error) {
-	conn, ok := <-l.accept
+func (l *UdpListener) Accept() (net.Conn, error) {
+	conn, ok := <-l.acceptCh
 	if !ok {
 		return conn, fmt.Errorf("channel for udp listener closed")
 	}
@@ -258,3 +250,7 @@ func (l *UdpListener) Close() error {
 	}
 	return nil
 }
+
+func (l *UdpListener) Addr() net.Addr {
+	return l.addr
+}

utils/net/websocket.go

@@ -8,8 +8,6 @@ import (
 	"net/url"
 	"time"
 
-	"github.com/fatedier/frp/utils/log"
-
 	"golang.org/x/net/websocket"
 )
 
@@ -22,10 +20,8 @@ const (
 )
 
 type WebsocketListener struct {
-	net.Addr
-	ln     net.Listener
-	accept chan Conn
-	log.Logger
+	ln       net.Listener
+	acceptCh chan net.Conn
 
 	server    *http.Server
 	httpMutex *http.ServeMux
@@ -35,9 +31,7 @@ type WebsocketListener struct {
 // ln: tcp listener for websocket connections
 func NewWebsocketListener(ln net.Listener) (wl *WebsocketListener) {
 	wl = &WebsocketListener{
-		Addr:   ln.Addr(),
-		accept: make(chan Conn),
-		Logger: log.NewPrefixLogger(""),
+		acceptCh: make(chan net.Conn),
 	}
 
 	muxer := http.NewServeMux()
@@ -46,7 +40,7 @@ func NewWebsocketListener(ln net.Listener) (wl *WebsocketListener) {
 		conn := WrapCloseNotifyConn(c, func() {
 			close(notifyCh)
 		})
-		wl.accept <- conn
+		wl.acceptCh <- conn
 		<-notifyCh
 	}))
 
@@ -68,8 +62,8 @@ func ListenWebsocket(bindAddr string, bindPort int) (*WebsocketListener, error)
 	return l, nil
 }
 
-func (p *WebsocketListener) Accept() (Conn, error) {
-	c, ok := <-p.accept
+func (p *WebsocketListener) Accept() (net.Conn, error) {
+	c, ok := <-p.acceptCh
 	if !ok {
 		return nil, ErrWebsocketListenerClosed
 	}
@@ -80,8 +74,12 @@ func (p *WebsocketListener) Close() error {
 	return p.server.Close()
 }
 
+func (p *WebsocketListener) Addr() net.Addr {
+	return p.ln.Addr()
+}
+
 // addr: domain:port
-func ConnectWebsocketServer(addr string) (Conn, error) {
+func ConnectWebsocketServer(addr string) (net.Conn, error) {
 	addr = "ws://" + addr + FrpWebsocketPath
 	uri, err := url.Parse(addr)
 	if err != nil {
@@ -101,6 +99,5 @@ func ConnectWebsocketServer(addr string) (Conn, error) {
 	if err != nil {
 		return nil, err
 	}
-	c := WrapConn(conn)
-	return c, nil
+	return conn, nil
 }

utils/version/version.go

@@ -19,7 +19,7 @@ import (
 	"strings"
 )
 
-var version string = "0.27.0"
+var version string = "0.30.0"
 
 func Full() string {
 	return version

utils/vhost/http.go

@@ -23,7 +23,6 @@ import (
 	"net"
 	"net/http"
 	"strings"
-	"sync"
 	"time"
 
 	frpLog "github.com/fatedier/frp/utils/log"
@@ -32,8 +31,7 @@ import (
 )
 
 var (
-	ErrRouterConfigConflict = errors.New("router config conflict")
-	ErrNoDomain             = errors.New("no such domain")
+	ErrNoDomain = errors.New("no such domain")
 )
 
 func getHostFromAddr(addr string) (host string) {
@@ -51,21 +49,19 @@ type HttpReverseProxyOptions struct {
 }
 
 type HttpReverseProxy struct {
-	proxy *ReverseProxy
-
+	proxy       *ReverseProxy
 	vhostRouter *VhostRouters
 
 	responseHeaderTimeout time.Duration
-	cfgMu                 sync.RWMutex
 }
 
-func NewHttpReverseProxy(option HttpReverseProxyOptions) *HttpReverseProxy {
+func NewHttpReverseProxy(option HttpReverseProxyOptions, vhostRouter *VhostRouters) *HttpReverseProxy {
 	if option.ResponseHeaderTimeoutS <= 0 {
 		option.ResponseHeaderTimeoutS = 60
 	}
 	rp := &HttpReverseProxy{
 		responseHeaderTimeout: time.Duration(option.ResponseHeaderTimeoutS) * time.Second,
-		vhostRouter:           NewVhostRouters(),
+		vhostRouter:           vhostRouter,
 	}
 	proxy := &ReverseProxy{
 		Director: func(req *http.Request) {
@@ -93,34 +89,30 @@ func NewHttpReverseProxy(option HttpReverseProxyOptions) *HttpReverseProxy {
 				return rp.CreateConnection(host, url, remote)
 			},
 		},
-		WebSocketDialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
-			url := ctx.Value("url").(string)
-			host := getHostFromAddr(ctx.Value("host").(string))
-			remote := ctx.Value("remote").(string)
-			return rp.CreateConnection(host, url, remote)
-		},
 		BufferPool: newWrapPool(),
 		ErrorLog:   log.New(newWrapLogger(), "", 0),
+		ErrorHandler: func(rw http.ResponseWriter, req *http.Request, err error) {
+			frpLog.Warn("do http proxy request error: %v", err)
+			rw.WriteHeader(http.StatusNotFound)
+			rw.Write(getNotFoundPageContent())
+		},
 	}
 	rp.proxy = proxy
 	return rp
 }
 
+// Register register the route config to reverse proxy
+// reverse proxy will use CreateConnFn from routeCfg to create a connection to the remote service
 func (rp *HttpReverseProxy) Register(routeCfg VhostRouteConfig) error {
-	rp.cfgMu.Lock()
-	defer rp.cfgMu.Unlock()
-	_, ok := rp.vhostRouter.Exist(routeCfg.Domain, routeCfg.Location)
-	if ok {
-		return ErrRouterConfigConflict
-	} else {
-		rp.vhostRouter.Add(routeCfg.Domain, routeCfg.Location, &routeCfg)
+	err := rp.vhostRouter.Add(routeCfg.Domain, routeCfg.Location, &routeCfg)
+	if err != nil {
+		return err
 	}
 	return nil
 }
 
+// UnRegister unregister route config by domain and location
 func (rp *HttpReverseProxy) UnRegister(domain string, location string) {
-	rp.cfgMu.Lock()
-	defer rp.cfgMu.Unlock()
 	rp.vhostRouter.Del(domain, location)
 }
 
@@ -140,6 +132,7 @@ func (rp *HttpReverseProxy) GetHeaders(domain string, location string) (headers
 	return
 }
 
+// CreateConnection create a new connection by route config
 func (rp *HttpReverseProxy) CreateConnection(domain string, location string, remoteAddr string) (net.Conn, error) {
 	vr, ok := rp.getVhost(domain, location)
 	if ok {
@@ -163,10 +156,8 @@ func (rp *HttpReverseProxy) CheckAuth(domain, location, user, passwd string) boo
 	return true
 }
 
+// getVhost get vhost router by domain and location
 func (rp *HttpReverseProxy) getVhost(domain string, location string) (vr *VhostRouter, ok bool) {
-	rp.cfgMu.RLock()
-	defer rp.cfgMu.RUnlock()
-
 	// first we check the full hostname
 	// if not exist, then check the wildcard_domain such as *.example.com
 	vr, ok = rp.vhostRouter.Get(domain, location)

utils/vhost/https.go

@@ -17,11 +17,10 @@ package vhost
 import (
 	"fmt"
 	"io"
+	"net"
 	"strings"
 	"time"
 
-	frpNet "github.com/fatedier/frp/utils/net"
-
 	gnet "github.com/fatedier/golib/net"
 	"github.com/fatedier/golib/pool"
 )
@@ -48,7 +47,7 @@ type HttpsMuxer struct {
 	*VhostMuxer
 }
 
-func NewHttpsMuxer(listener frpNet.Listener, timeout time.Duration) (*HttpsMuxer, error) {
+func NewHttpsMuxer(listener net.Listener, timeout time.Duration) (*HttpsMuxer, error) {
 	mux, err := NewVhostMuxer(listener, GetHttpsHostname, nil, nil, timeout)
 	return &HttpsMuxer{mux}, err
 }
@@ -182,7 +181,7 @@ func readHandshake(rd io.Reader) (host string, err error) {
 	return
 }
 
-func GetHttpsHostname(c frpNet.Conn) (_ frpNet.Conn, _ map[string]string, err error) {
+func GetHttpsHostname(c net.Conn) (_ net.Conn, _ map[string]string, err error) {
 	reqInfoMap := make(map[string]string, 0)
 	sc, rd := gnet.NewSharedConn(c)
 	host, err := readHandshake(rd)
@@ -191,5 +190,5 @@ func GetHttpsHostname(c frpNet.Conn) (_ frpNet.Conn, _ map[string]string, err er
 	}
 	reqInfoMap["Host"] = host
 	reqInfoMap["Scheme"] = "https"
-	return frpNet.WrapConn(sc), reqInfoMap, nil
+	return sc, reqInfoMap, nil
 }

utils/vhost/reverseproxy.go

@@ -8,6 +8,7 @@ package vhost
 
 import (
 	"context"
+	"fmt"
 	"io"
 	"log"
 	"net"
@@ -17,13 +18,9 @@ import (
 	"sync"
 	"time"
 
-	frpIo "github.com/fatedier/golib/io"
+	"golang.org/x/net/http/httpguts"
 )
 
-// onExitFlushLoop is a callback set by tests to detect the state of the
-// flushLoop() goroutine.
-var onExitFlushLoop func()
-
 // ReverseProxy is an HTTP Handler that takes an incoming request and
 // sends it to another server, proxying the response back to the
 // client.
@@ -44,12 +41,17 @@ type ReverseProxy struct {
 	// to flush to the client while copying the
 	// response body.
 	// If zero, no periodic flushing is done.
+	// A negative value means to flush immediately
+	// after each write to the client.
+	// The FlushInterval is ignored when ReverseProxy
+	// recognizes a response as a streaming response;
+	// for such responses, writes are flushed to the client
+	// immediately.
 	FlushInterval time.Duration
 
 	// ErrorLog specifies an optional logger for errors
 	// that occur when attempting to proxy the request.
-	// If nil, logging goes to os.Stderr via the log package's
-	// standard logger.
+	// If nil, logging is done via the log package's standard logger.
 	ErrorLog *log.Logger
 
 	// BufferPool optionally specifies a buffer pool to
@@ -57,12 +59,23 @@ type ReverseProxy struct {
 	// copying HTTP response bodies.
 	BufferPool BufferPool
 
-	// ModifyResponse is an optional function that
-	// modifies the Response from the backend.
-	// If it returns an error, the proxy returns a StatusBadGateway error.
+	// ModifyResponse is an optional function that modifies the
+	// Response from the backend. It is called if the backend
+	// returns a response at all, with any HTTP status code.
+	// If the backend is unreachable, the optional ErrorHandler is
+	// called without any call to ModifyResponse.
+	//
+	// If ModifyResponse returns an error, ErrorHandler is called
+	// with its error value. If ErrorHandler is nil, its default
+	// implementation is used.
 	ModifyResponse func(*http.Response) error
 
-	WebSocketDialContext func(ctx context.Context, network, addr string) (net.Conn, error)
+	// ErrorHandler is an optional function that handles errors
+	// reaching the backend or errors from ModifyResponse.
+	//
+	// If nil, the default is to log the provided error and return
+	// a 502 Status Bad Gateway response.
+	ErrorHandler func(http.ResponseWriter, *http.Request, error)
 }
 
 // A BufferPool is an interface for getting and returning temporary
@@ -118,18 +131,11 @@ func copyHeader(dst, src http.Header) {
 	}
 }
 
-func cloneHeader(h http.Header) http.Header {
-	h2 := make(http.Header, len(h))
-	for k, vv := range h {
-		vv2 := make([]string, len(vv))
-		copy(vv2, vv)
-		h2[k] = vv2
-	}
-	return h2
-}
-
 // Hop-by-hop headers. These are removed when sent to the backend.
-// http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html
+// As of RFC 7230, hop-by-hop headers are required to appear in the
+// Connection header field. These are the headers defined by the
+// obsoleted RFC 2616 (section 13.5.1) and are used for backward
+// compatibility.
 var hopHeaders = []string{
 	"Connection",
 	"Proxy-Connection", // non-standard but still sent by libcurl and rejected by e.g. google
@@ -137,55 +143,38 @@ var hopHeaders = []string{
 	"Proxy-Authenticate",
 	"Proxy-Authorization",
 	"Te",      // canonicalized version of "TE"
-	"Trailer", // not Trailers per URL above; http://www.rfc-editor.org/errata_search.php?eid=4522
+	"Trailer", // not Trailers per URL above; https://www.rfc-editor.org/errata_search.php?eid=4522
 	"Transfer-Encoding",
 	"Upgrade",
 }
 
+func (p *ReverseProxy) defaultErrorHandler(rw http.ResponseWriter, req *http.Request, err error) {
+	p.logf("http: proxy error: %v", err)
+	rw.WriteHeader(http.StatusBadGateway)
+}
+
+func (p *ReverseProxy) getErrorHandler() func(http.ResponseWriter, *http.Request, error) {
+	if p.ErrorHandler != nil {
+		return p.ErrorHandler
+	}
+	return p.defaultErrorHandler
+}
+
+// modifyResponse conditionally runs the optional ModifyResponse hook
+// and reports whether the request should proceed.
+func (p *ReverseProxy) modifyResponse(rw http.ResponseWriter, res *http.Response, req *http.Request) bool {
+	if p.ModifyResponse == nil {
+		return true
+	}
+	if err := p.ModifyResponse(res); err != nil {
+		res.Body.Close()
+		p.getErrorHandler()(rw, req, err)
+		return false
+	}
+	return true
+}
+
 func (p *ReverseProxy) ServeHTTP(rw http.ResponseWriter, req *http.Request) {
-	if IsWebsocketRequest(req) {
-		p.serveWebSocket(rw, req)
-	} else {
-		p.serveHTTP(rw, req)
-	}
-}
-
-func (p *ReverseProxy) serveWebSocket(rw http.ResponseWriter, req *http.Request) {
-	if p.WebSocketDialContext == nil {
-		rw.WriteHeader(500)
-		return
-	}
-
-	req = req.WithContext(context.WithValue(req.Context(), "url", req.URL.Path))
-	req = req.WithContext(context.WithValue(req.Context(), "host", req.Host))
-	req = req.WithContext(context.WithValue(req.Context(), "remote", req.RemoteAddr))
-
-	targetConn, err := p.WebSocketDialContext(req.Context(), "tcp", "")
-	if err != nil {
-		rw.WriteHeader(501)
-		return
-	}
-	defer targetConn.Close()
-
-	p.Director(req)
-
-	hijacker, ok := rw.(http.Hijacker)
-	if !ok {
-		rw.WriteHeader(500)
-		return
-	}
-	conn, _, errHijack := hijacker.Hijack()
-	if errHijack != nil {
-		rw.WriteHeader(500)
-		return
-	}
-	defer conn.Close()
-
-	req.Write(targetConn)
-	frpIo.Join(conn, targetConn)
-}
-
-func (p *ReverseProxy) serveHTTP(rw http.ResponseWriter, req *http.Request) {
 	transport := p.Transport
 	if transport == nil {
 		transport = http.DefaultTransport
@@ -206,38 +195,49 @@ func (p *ReverseProxy) serveHTTP(rw http.ResponseWriter, req *http.Request) {
 		}()
 	}
 
-	outreq := req.WithContext(ctx) // includes shallow copies of maps, but okay
+	outreq := req.WithContext(ctx)
 	if req.ContentLength == 0 {
 		outreq.Body = nil // Issue 16036: nil Body for http.Transport retries
 	}
 
-	outreq.Header = cloneHeader(req.Header)
-
-	// Modify for frp
+	// =============================
+	// Modified for frp
 	outreq = outreq.WithContext(context.WithValue(outreq.Context(), "url", req.URL.Path))
 	outreq = outreq.WithContext(context.WithValue(outreq.Context(), "host", req.Host))
 	outreq = outreq.WithContext(context.WithValue(outreq.Context(), "remote", req.RemoteAddr))
+	// =============================
 
 	p.Director(outreq)
 	outreq.Close = false
 
-	// Remove hop-by-hop headers listed in the "Connection" header.
-	// See RFC 2616, section 14.10.
-	if c := outreq.Header.Get("Connection"); c != "" {
-		for _, f := range strings.Split(c, ",") {
-			if f = strings.TrimSpace(f); f != "" {
-				outreq.Header.Del(f)
-			}
-		}
-	}
+	reqUpType := upgradeType(outreq.Header)
+	removeConnectionHeaders(outreq.Header)
 
 	// Remove hop-by-hop headers to the backend. Especially
 	// important is "Connection" because we want a persistent
 	// connection, regardless of what the client sent to us.
 	for _, h := range hopHeaders {
-		if outreq.Header.Get(h) != "" {
-			outreq.Header.Del(h)
+		hv := outreq.Header.Get(h)
+		if hv == "" {
+			continue
 		}
+		if h == "Te" && hv == "trailers" {
+			// Issue 21096: tell backend applications that
+			// care about trailer support that we support
+			// trailers. (We do, but we don't go out of
+			// our way to advertise that unless the
+			// incoming client request thought it was
+			// worth mentioning)
+			continue
+		}
+		outreq.Header.Del(h)
+	}
+
+	// After stripping all the hop-by-hop connection headers above, add back any
+	// necessary for protocol upgrades, such as for websockets.
+	if reqUpType != "" {
+		outreq.Header.Set("Connection", "Upgrade")
+		outreq.Header.Set("Upgrade", reqUpType)
 	}
 
 	if clientIP, _, err := net.SplitHostPort(req.RemoteAddr); err == nil {
@@ -252,33 +252,27 @@ func (p *ReverseProxy) serveHTTP(rw http.ResponseWriter, req *http.Request) {
 
 	res, err := transport.RoundTrip(outreq)
 	if err != nil {
-		p.logf("http: proxy error: %v", err)
-		rw.WriteHeader(http.StatusNotFound)
-
-		rw.Write(getNotFoundPageContent())
+		p.getErrorHandler()(rw, outreq, err)
 		return
 	}
 
-	// Remove hop-by-hop headers listed in the
-	// "Connection" header of the response.
-	if c := res.Header.Get("Connection"); c != "" {
-		for _, f := range strings.Split(c, ",") {
-			if f = strings.TrimSpace(f); f != "" {
-				res.Header.Del(f)
-			}
+	// Deal with 101 Switching Protocols responses: (WebSocket, h2c, etc)
+	if res.StatusCode == http.StatusSwitchingProtocols {
+		if !p.modifyResponse(rw, res, outreq) {
+			return
 		}
+		p.handleUpgradeResponse(rw, outreq, res)
+		return
 	}
 
+	removeConnectionHeaders(res.Header)
+
 	for _, h := range hopHeaders {
 		res.Header.Del(h)
 	}
 
-	if p.ModifyResponse != nil {
-		if err := p.ModifyResponse(res); err != nil {
-			p.logf("http: proxy error: %v", err)
-			rw.WriteHeader(http.StatusBadGateway)
-			return
-		}
+	if !p.modifyResponse(rw, res, outreq) {
+		return
 	}
 
 	copyHeader(rw.Header(), res.Header)
@@ -295,6 +289,21 @@ func (p *ReverseProxy) serveHTTP(rw http.ResponseWriter, req *http.Request) {
 	}
 
 	rw.WriteHeader(res.StatusCode)
+
+	err = p.copyResponse(rw, res.Body, p.flushInterval(req, res))
+	if err != nil {
+		defer res.Body.Close()
+		// Since we're streaming the response, if we run into an error all we can do
+		// is abort the request. Issue 23643: ReverseProxy should use ErrAbortHandler
+		// on read error while copying body.
+		if !shouldPanicOnCopyError(req) {
+			p.logf("suppressing panic for copyResponse error in test; copy error: %v", err)
+			return
+		}
+		panic(http.ErrAbortHandler)
+	}
+	res.Body.Close() // close now, instead of defer, to populate res.Trailer
+
 	if len(res.Trailer) > 0 {
 		// Force chunking if we saw a response trailer.
 		// This prevents net/http from calculating the length for short
@@ -303,8 +312,6 @@ func (p *ReverseProxy) serveHTTP(rw http.ResponseWriter, req *http.Request) {
 			fl.Flush()
 		}
 	}
-	p.copyResponse(rw, res.Body)
-	res.Body.Close() // close now, instead of defer, to populate res.Trailer
 
 	if len(res.Trailer) == announcedTrailers {
 		copyHeader(rw.Header(), res.Trailer)
@@ -319,16 +326,68 @@ func (p *ReverseProxy) serveHTTP(rw http.ResponseWriter, req *http.Request) {
 	}
 }
 
-func (p *ReverseProxy) copyResponse(dst io.Writer, src io.Reader) {
-	if p.FlushInterval != 0 {
+var inOurTests bool // whether we're in our own tests
+
+// shouldPanicOnCopyError reports whether the reverse proxy should
+// panic with http.ErrAbortHandler. This is the right thing to do by
+// default, but Go 1.10 and earlier did not, so existing unit tests
+// weren't expecting panics. Only panic in our own tests, or when
+// running under the HTTP server.
+func shouldPanicOnCopyError(req *http.Request) bool {
+	if inOurTests {
+		// Our tests know to handle this panic.
+		return true
+	}
+	if req.Context().Value(http.ServerContextKey) != nil {
+		// We seem to be running under an HTTP server, so
+		// it'll recover the panic.
+		return true
+	}
+	// Otherwise act like Go 1.10 and earlier to not break
+	// existing tests.
+	return false
+}
+
+// removeConnectionHeaders removes hop-by-hop headers listed in the "Connection" header of h.
+// See RFC 7230, section 6.1
+func removeConnectionHeaders(h http.Header) {
+	for _, f := range h["Connection"] {
+		for _, sf := range strings.Split(f, ",") {
+			if sf = strings.TrimSpace(sf); sf != "" {
+				h.Del(sf)
+			}
+		}
+	}
+}
+
+// flushInterval returns the p.FlushInterval value, conditionally
+// overriding its value for a specific request/response.
+func (p *ReverseProxy) flushInterval(req *http.Request, res *http.Response) time.Duration {
+	resCT := res.Header.Get("Content-Type")
+
+	// For Server-Sent Events responses, flush immediately.
+	// The MIME type is defined in https://www.w3.org/TR/eventsource/#text-event-stream
+	if resCT == "text/event-stream" {
+		return -1 // negative means immediately
+	}
+
+	// TODO: more specific cases? e.g. res.ContentLength == -1?
+	return p.FlushInterval
+}
+
+func (p *ReverseProxy) copyResponse(dst io.Writer, src io.Reader, flushInterval time.Duration) error {
+	if flushInterval != 0 {
 		if wf, ok := dst.(writeFlusher); ok {
 			mlw := &maxLatencyWriter{
 				dst:     wf,
-				latency: p.FlushInterval,
-				done:    make(chan bool),
+				latency: flushInterval,
 			}
-			go mlw.flushLoop()
 			defer mlw.stop()
+
+			// set up initial timer so headers get flushed even if body writes are delayed
+			mlw.flushPending = true
+			mlw.t = time.AfterFunc(flushInterval, mlw.delayedFlush)
+
 			dst = mlw
 		}
 	}
@@ -336,13 +395,14 @@ func (p *ReverseProxy) copyResponse(dst io.Writer, src io.Reader) {
 	var buf []byte
 	if p.BufferPool != nil {
 		buf = p.BufferPool.Get()
+		defer p.BufferPool.Put(buf)
 	}
-	p.copyBuffer(dst, src, buf)
-	if p.BufferPool != nil {
-		p.BufferPool.Put(buf)
-	}
+	_, err := p.copyBuffer(dst, src, buf)
+	return err
 }
 
+// copyBuffer returns any write errors or non-EOF read errors, and the amount
+// of bytes written.
 func (p *ReverseProxy) copyBuffer(dst io.Writer, src io.Reader, buf []byte) (int64, error) {
 	if len(buf) == 0 {
 		buf = make([]byte, 32*1024)
@@ -366,6 +426,9 @@ func (p *ReverseProxy) copyBuffer(dst io.Writer, src io.Reader, buf []byte) (int
 			}
 		}
 		if rerr != nil {
+			if rerr == io.EOF {
+				rerr = nil
+			}
 			return written, rerr
 		}
 	}
@@ -386,47 +449,115 @@ type writeFlusher interface {
 
 type maxLatencyWriter struct {
 	dst     writeFlusher
-	latency time.Duration
+	latency time.Duration // non-zero; negative means to flush immediately
 
-	mu   sync.Mutex // protects Write + Flush
-	done chan bool
+	mu           sync.Mutex // protects t, flushPending, and dst.Flush
+	t            *time.Timer
+	flushPending bool
 }
 
-func (m *maxLatencyWriter) Write(p []byte) (int, error) {
+func (m *maxLatencyWriter) Write(p []byte) (n int, err error) {
 	m.mu.Lock()
 	defer m.mu.Unlock()
-	return m.dst.Write(p)
+	n, err = m.dst.Write(p)
+	if m.latency < 0 {
+		m.dst.Flush()
+		return
+	}
+	if m.flushPending {
+		return
+	}
+	if m.t == nil {
+		m.t = time.AfterFunc(m.latency, m.delayedFlush)
+	} else {
+		m.t.Reset(m.latency)
+	}
+	m.flushPending = true
+	return
 }
 
-func (m *maxLatencyWriter) flushLoop() {
-	t := time.NewTicker(m.latency)
-	defer t.Stop()
-	for {
-		select {
-		case <-m.done:
-			if onExitFlushLoop != nil {
-				onExitFlushLoop()
-			}
-			return
-		case <-t.C:
-			m.mu.Lock()
-			m.dst.Flush()
-			m.mu.Unlock()
-		}
+func (m *maxLatencyWriter) delayedFlush() {
+	m.mu.Lock()
+	defer m.mu.Unlock()
+	if !m.flushPending { // if stop was called but AfterFunc already started this goroutine
+		return
+	}
+	m.dst.Flush()
+	m.flushPending = false
+}
+
+func (m *maxLatencyWriter) stop() {
+	m.mu.Lock()
+	defer m.mu.Unlock()
+	m.flushPending = false
+	if m.t != nil {
+		m.t.Stop()
 	}
 }
 
-func (m *maxLatencyWriter) stop() { m.done <- true }
-
-func IsWebsocketRequest(req *http.Request) bool {
-	containsHeader := func(name, value string) bool {
-		items := strings.Split(req.Header.Get(name), ",")
-		for _, item := range items {
-			if value == strings.ToLower(strings.TrimSpace(item)) {
-				return true
-			}
-		}
-		return false
+func upgradeType(h http.Header) string {
+	if !httpguts.HeaderValuesContainsToken(h["Connection"], "Upgrade") {
+		return ""
 	}
-	return containsHeader("Connection", "upgrade") && containsHeader("Upgrade", "websocket")
+	return strings.ToLower(h.Get("Upgrade"))
+}
+
+func (p *ReverseProxy) handleUpgradeResponse(rw http.ResponseWriter, req *http.Request, res *http.Response) {
+	reqUpType := upgradeType(req.Header)
+	resUpType := upgradeType(res.Header)
+	if reqUpType != resUpType {
+		p.getErrorHandler()(rw, req, fmt.Errorf("backend tried to switch protocol %q when %q was requested", resUpType, reqUpType))
+		return
+	}
+
+	copyHeader(res.Header, rw.Header())
+
+	hj, ok := rw.(http.Hijacker)
+	if !ok {
+		p.getErrorHandler()(rw, req, fmt.Errorf("can't switch protocols using non-Hijacker ResponseWriter type %T", rw))
+		return
+	}
+	backConn, ok := res.Body.(io.ReadWriteCloser)
+	if !ok {
+		p.getErrorHandler()(rw, req, fmt.Errorf("internal error: 101 switching protocols response with non-writable body"))
+		return
+	}
+	defer backConn.Close()
+	conn, brw, err := hj.Hijack()
+	if err != nil {
+		p.getErrorHandler()(rw, req, fmt.Errorf("Hijack failed on protocol switch: %v", err))
+		return
+	}
+	defer conn.Close()
+	res.Body = nil // so res.Write only writes the headers; we have res.Body in backConn above
+	if err := res.Write(brw); err != nil {
+		p.getErrorHandler()(rw, req, fmt.Errorf("response write: %v", err))
+		return
+	}
+	if err := brw.Flush(); err != nil {
+		p.getErrorHandler()(rw, req, fmt.Errorf("response flush: %v", err))
+		return
+	}
+	errc := make(chan error, 1)
+	spc := switchProtocolCopier{user: conn, backend: backConn}
+	go spc.copyToBackend(errc)
+	go spc.copyFromBackend(errc)
+	<-errc
+	return
+}
+
+// switchProtocolCopier exists so goroutines proxying data back and
+// forth have nice names in stacks.
+type switchProtocolCopier struct {
+	user, backend io.ReadWriter
+}
+
+func (c switchProtocolCopier) copyFromBackend(errc chan<- error) {
+	_, err := io.Copy(c.user, c.backend)
+	errc <- err
+}
+
+func (c switchProtocolCopier) copyToBackend(errc chan<- error) {
+	_, err := io.Copy(c.backend, c.user)
+	errc <- err
 }

utils/vhost/router.go

@@ -1,11 +1,16 @@
 package vhost
 
 import (
+	"errors"
 	"sort"
 	"strings"
 	"sync"
 )
 
+var (
+	ErrRouterConfigConflict = errors.New("router config conflict")
+)
+
 type VhostRouters struct {
 	RouterByDomain map[string][]*VhostRouter
 	mutex          sync.RWMutex
@@ -24,10 +29,14 @@ func NewVhostRouters() *VhostRouters {
 	}
 }
 
-func (r *VhostRouters) Add(domain, location string, payload interface{}) {
+func (r *VhostRouters) Add(domain, location string, payload interface{}) error {
 	r.mutex.Lock()
 	defer r.mutex.Unlock()
 
+	if _, exist := r.exist(domain, location); exist {
+		return ErrRouterConfigConflict
+	}
+
 	vrs, found := r.RouterByDomain[domain]
 	if !found {
 		vrs = make([]*VhostRouter, 0, 1)
@@ -42,6 +51,7 @@ func (r *VhostRouters) Add(domain, location string, payload interface{}) {
 
 	sort.Sort(sort.Reverse(ByLocation(vrs)))
 	r.RouterByDomain[domain] = vrs
+	return nil
 }
 
 func (r *VhostRouters) Del(domain, location string) {
@@ -80,10 +90,7 @@ func (r *VhostRouters) Get(host, path string) (vr *VhostRouter, exist bool) {
 	return
 }
 
-func (r *VhostRouters) Exist(host, path string) (vr *VhostRouter, exist bool) {
-	r.mutex.RLock()
-	defer r.mutex.RUnlock()
-
+func (r *VhostRouters) exist(host, path string) (vr *VhostRouter, exist bool) {
 	vrs, found := r.RouterByDomain[host]
 	if !found {
 		return

utils/vhost/vhost.go

@@ -13,32 +13,33 @@
 package vhost
 
 import (
+	"context"
 	"fmt"
+	"net"
 	"strings"
-	"sync"
 	"time"
 
 	"github.com/fatedier/frp/utils/log"
 	frpNet "github.com/fatedier/frp/utils/net"
+	"github.com/fatedier/frp/utils/xlog"
 
 	"github.com/fatedier/golib/errors"
 )
 
-type muxFunc func(frpNet.Conn) (frpNet.Conn, map[string]string, error)
-type httpAuthFunc func(frpNet.Conn, string, string, string) (bool, error)
-type hostRewriteFunc func(frpNet.Conn, string) (frpNet.Conn, error)
+type muxFunc func(net.Conn) (net.Conn, map[string]string, error)
+type httpAuthFunc func(net.Conn, string, string, string) (bool, error)
+type hostRewriteFunc func(net.Conn, string) (net.Conn, error)
 
 type VhostMuxer struct {
-	listener       frpNet.Listener
+	listener       net.Listener
 	timeout        time.Duration
 	vhostFunc      muxFunc
 	authFunc       httpAuthFunc
 	rewriteFunc    hostRewriteFunc
 	registryRouter *VhostRouters
-	mutex          sync.RWMutex
 }
 
-func NewVhostMuxer(listener frpNet.Listener, vhostFunc muxFunc, authFunc httpAuthFunc, rewriteFunc hostRewriteFunc, timeout time.Duration) (mux *VhostMuxer, err error) {
+func NewVhostMuxer(listener net.Listener, vhostFunc muxFunc, authFunc httpAuthFunc, rewriteFunc hostRewriteFunc, timeout time.Duration) (mux *VhostMuxer, err error) {
 	mux = &VhostMuxer{
 		listener:       listener,
 		timeout:        timeout,
@@ -51,8 +52,9 @@ func NewVhostMuxer(listener frpNet.Listener, vhostFunc muxFunc, authFunc httpAut
 	return mux, nil
 }
 
-type CreateConnFunc func(remoteAddr string) (frpNet.Conn, error)
+type CreateConnFunc func(remoteAddr string) (net.Conn, error)
 
+// VhostRouteConfig is the params used to match HTTP requests
 type VhostRouteConfig struct {
 	Domain      string
 	Location    string
@@ -66,15 +68,7 @@ type VhostRouteConfig struct {
 
 // listen for a new domain name, if rewriteHost is not empty  and rewriteFunc is not nil
 // then rewrite the host header to rewriteHost
-func (v *VhostMuxer) Listen(cfg *VhostRouteConfig) (l *Listener, err error) {
-	v.mutex.Lock()
-	defer v.mutex.Unlock()
-
-	_, ok := v.registryRouter.Exist(cfg.Domain, cfg.Location)
-	if ok {
-		return nil, fmt.Errorf("hostname [%s] location [%s] is already registered", cfg.Domain, cfg.Location)
-	}
-
+func (v *VhostMuxer) Listen(ctx context.Context, cfg *VhostRouteConfig) (l *Listener, err error) {
 	l = &Listener{
 		name:        cfg.Domain,
 		location:    cfg.Location,
@@ -82,17 +76,17 @@ func (v *VhostMuxer) Listen(cfg *VhostRouteConfig) (l *Listener, err error) {
 		userName:    cfg.Username,
 		passWord:    cfg.Password,
 		mux:         v,
-		accept:      make(chan frpNet.Conn),
-		Logger:      log.NewPrefixLogger(""),
+		accept:      make(chan net.Conn),
+		ctx:         ctx,
+	}
+	err = v.registryRouter.Add(cfg.Domain, cfg.Location, l)
+	if err != nil {
+		return
 	}
-	v.registryRouter.Add(cfg.Domain, cfg.Location, l)
 	return l, nil
 }
 
 func (v *VhostMuxer) getListener(name, path string) (l *Listener, exist bool) {
-	v.mutex.RLock()
-	defer v.mutex.RUnlock()
-
 	// first we check the full hostname
 	// if not exist, then check the wildcard_domain such as *.example.com
 	vr, found := v.registryRouter.Get(name, path)
@@ -132,7 +126,7 @@ func (v *VhostMuxer) run() {
 	}
 }
 
-func (v *VhostMuxer) handle(c frpNet.Conn) {
+func (v *VhostMuxer) handle(c net.Conn) {
 	if err := c.SetDeadline(time.Now().Add(v.timeout)); err != nil {
 		c.Close()
 		return
@@ -155,13 +149,14 @@ func (v *VhostMuxer) handle(c frpNet.Conn) {
 		c.Close()
 		return
 	}
+	xl := xlog.FromContextSafe(l.ctx)
 
 	// if authFunc is exist and userName/password is set
 	// then verify user access
 	if l.mux.authFunc != nil && l.userName != "" && l.passWord != "" {
 		bAccess, err := l.mux.authFunc(c, l.userName, l.passWord, reqInfoMap["Authorization"])
 		if bAccess == false || err != nil {
-			l.Debug("check http Authorization failed")
+			xl.Debug("check http Authorization failed")
 			res := noAuthResponse()
 			res.Write(c)
 			c.Close()
@@ -175,12 +170,12 @@ func (v *VhostMuxer) handle(c frpNet.Conn) {
 	}
 	c = sConn
 
-	l.Debug("get new http request host [%s] path [%s]", name, path)
+	xl.Debug("get new http request host [%s] path [%s]", name, path)
 	err = errors.PanicToError(func() {
 		l.accept <- c
 	})
 	if err != nil {
-		l.Warn("listener is already closed, ignore this request")
+		xl.Warn("listener is already closed, ignore this request")
 	}
 }
 
@@ -191,11 +186,12 @@ type Listener struct {
 	userName    string
 	passWord    string
 	mux         *VhostMuxer // for closing VhostMuxer
-	accept      chan frpNet.Conn
-	log.Logger
+	accept      chan net.Conn
+	ctx         context.Context
 }
 
-func (l *Listener) Accept() (frpNet.Conn, error) {
+func (l *Listener) Accept() (net.Conn, error) {
+	xl := xlog.FromContextSafe(l.ctx)
 	conn, ok := <-l.accept
 	if !ok {
 		return nil, fmt.Errorf("Listener closed")
@@ -207,17 +203,13 @@ func (l *Listener) Accept() (frpNet.Conn, error) {
 	if l.mux.rewriteFunc != nil {
 		sConn, err := l.mux.rewriteFunc(conn, l.rewriteHost)
 		if err != nil {
-			l.Warn("host header rewrite failed: %v", err)
+			xl.Warn("host header rewrite failed: %v", err)
 			return nil, fmt.Errorf("host header rewrite failed")
 		}
-		l.Debug("rewrite host to [%s] success", l.rewriteHost)
+		xl.Debug("rewrite host to [%s] success", l.rewriteHost)
 		conn = sConn
 	}
-
-	for _, prefix := range l.GetAllPrefix() {
-		conn.AddLogPrefix(prefix)
-	}
-	return conn, nil
+	return frpNet.NewContextConn(conn, l.ctx), nil
 }
 
 func (l *Listener) Close() error {
@@ -229,3 +221,7 @@ func (l *Listener) Close() error {
 func (l *Listener) Name() string {
 	return l.name
 }
+
+func (l *Listener) Addr() net.Addr {
+	return (*net.TCPAddr)(nil)
+}

utils/xlog/ctx.go

@@ -0,0 +1,42 @@
+// Copyright 2019 fatedier, fatedier@gmail.com
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package xlog
+
+import (
+	"context"
+)
+
+type key int
+
+const (
+	xlogKey key = 0
+)
+
+func NewContext(ctx context.Context, xl *Logger) context.Context {
+	return context.WithValue(ctx, xlogKey, xl)
+}
+
+func FromContext(ctx context.Context) (xl *Logger, ok bool) {
+	xl, ok = ctx.Value(xlogKey).(*Logger)
+	return
+}
+
+func FromContextSafe(ctx context.Context) *Logger {
+	xl, ok := ctx.Value(xlogKey).(*Logger)
+	if !ok {
+		xl = New()
+	}
+	return xl
+}

utils/xlog/xlog.go

@@ -0,0 +1,73 @@
+// Copyright 2019 fatedier, fatedier@gmail.com
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package xlog
+
+import (
+	"github.com/fatedier/frp/utils/log"
+)
+
+// Logger is not thread safety for operations on prefix
+type Logger struct {
+	prefixes []string
+
+	prefixString string
+}
+
+func New() *Logger {
+	return &Logger{
+		prefixes: make([]string, 0),
+	}
+}
+
+func (l *Logger) ResetPrefixes() (old []string) {
+	old = l.prefixes
+	l.prefixes = make([]string, 0)
+	l.prefixString = ""
+	return
+}
+
+func (l *Logger) AppendPrefix(prefix string) *Logger {
+	l.prefixes = append(l.prefixes, prefix)
+	l.prefixString += "[" + prefix + "] "
+	return l
+}
+
+func (l *Logger) Spawn() *Logger {
+	nl := New()
+	for _, v := range l.prefixes {
+		nl.AppendPrefix(v)
+	}
+	return nl
+}
+
+func (l *Logger) Error(format string, v ...interface{}) {
+	log.Log.Error(l.prefixString+format, v...)
+}
+
+func (l *Logger) Warn(format string, v ...interface{}) {
+	log.Log.Warn(l.prefixString+format, v...)
+}
+
+func (l *Logger) Info(format string, v ...interface{}) {
+	log.Log.Info(l.prefixString+format, v...)
+}
+
+func (l *Logger) Debug(format string, v ...interface{}) {
+	log.Log.Debug(l.prefixString+format, v...)
+}
+
+func (l *Logger) Trace(format string, v ...interface{}) {
+	log.Log.Trace(l.prefixString+format, v...)
+}

vendor/github.com/fatedier/kcp-go/README.md

@@ -34,6 +34,7 @@ This library intents to provide a **smooth, resilient, ordered, error-checked an
 1. Packet level encryption support with [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard), [TEA](https://en.wikipedia.org/wiki/Tiny_Encryption_Algorithm), [3DES](https://en.wikipedia.org/wiki/Triple_DES), [Blowfish](https://en.wikipedia.org/wiki/Blowfish_(cipher)), [Cast5](https://en.wikipedia.org/wiki/CAST-128), [Salsa20]( https://en.wikipedia.org/wiki/Salsa20), etc. in [CFB](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Cipher_Feedback_.28CFB.29) mode, which generates completely anonymous packet.
 1. Only **A fixed number of goroutines** will be created for the entire server application, costs in **context switch** between goroutines have been taken into consideration.
 1. Compatible with [skywind3000's](https://github.com/skywind3000) C version with various improvements.
+1. Platform-dependent optimizations: [sendmmsg](http://man7.org/linux/man-pages/man2/sendmmsg.2.html) and [recvmmsg](http://man7.org/linux/man-pages/man2/recvmmsg.2.html) were expoloited for linux.
 
 ## Documentation
 
@@ -43,6 +44,24 @@ For complete documentation, see the associated [Godoc](https://godoc.org/github.
 
 <img src="frame.png" alt="Frame Format" height="109px" />
 
+```
+NONCE:
+  16bytes cryptographically secure random number, nonce changes for every packet.
+  
+CRC32:
+  CRC-32 checksum of data using the IEEE polynomial
+ 
+FEC TYPE:
+  typeData = 0xF1
+  typeParity = 0xF2
+  
+FEC SEQID:
+  monotonically increasing in range: [0, (0xffffffff/shardSize) * shardSize - 1]
+  
+SIZE:
+  The size of KCP frame plus 2
+```
+
 ```
 +-----------------+
 | SESSION         |
@@ -65,16 +84,11 @@ For complete documentation, see the associated [Godoc](https://godoc.org/github.
 ```
 
 
-## Usage
+## Examples
 
-Client:   [full demo](https://github.com/xtaci/kcptun/blob/master/client/main.go)
-```go
-kcpconn, err := kcp.DialWithOptions("192.168.0.1:10000", nil, 10, 3)
-```
-Server:   [full demo](https://github.com/xtaci/kcptun/blob/master/server/main.go)
-```go
-lis, err := kcp.ListenWithOptions(":10000", nil, 10, 3)
-```
+1. [simple examples](https://github.com/xtaci/kcp-go/tree/master/examples)
+2. [kcptun client](https://github.com/xtaci/kcptun/blob/master/client/main.go)
+3. [kcptun server](https://github.com/xtaci/kcptun/blob/master/server/main.go)
 
 ## Benchmark
 ```
@@ -128,6 +142,10 @@ PASS
 ok  	github.com/xtaci/kcp-go	50.349s
 ```
 
+
+## Typical Flame Graph
+![Flame Graph in kcptun](flame.png)
+
 ## Key Design Considerations
 
 1. slice vs. container/list
@@ -159,6 +177,18 @@ BenchmarkNow-4         	100000000	        15.6 ns/op
 
 In kcp-go, after each `kcp.output()` function call, current clock time will be updated upon return, and for a single `kcp.flush()` operation, current time will be queried from system once. For most of the time, 5000 connections costs 5000 * 15.6ns = 78us(a fixed cost while no packet needs to be sent), as for 10MB/s data transfering with 1400 MTU, `kcp.output()` will be called around 7500 times and costs 117us for `time.Now()` in **every second**.
 
+3. Memory management
+
+Primary memory allocation are done from a global buffer pool xmit.Buf, in kcp-go, when we need to allocate some bytes, we can get from that pool, and a fixed-capacity 1500 bytes(mtuLimit) will be returned, the rx queue, tx queue and fec queue all receive bytes from there, and they will return the bytes to the pool after using to prevent unnecessary zer0ing of bytes. The pool mechanism maintained a high watermark for slice objects, these in-flight objects from the pool will survive from the perodical garbage collection, meanwhile the pool kept the ability to return the memory to runtime if in idle.
+
+4. Information security
+
+kcp-go is shipped with builtin packet encryption powered by various block encryption algorithms and works in [Cipher Feedback Mode](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Cipher_Feedback_(CFB)), for each packet to be sent, the encryption process will start from encrypting a [nonce](https://en.wikipedia.org/wiki/Cryptographic_nonce) from the [system entropy](https://en.wikipedia.org/wiki//dev/random), so encryption to same plaintexts never leads to a same ciphertexts thereafter.
+
+The contents of the packets are completely anonymous with encryption, including the headers(FEC,KCP), checksums and contents. Note that, no matter which encryption method you choose on you upper layer, if you disable encryption, the transmit will be insecure somehow, since the header is ***PLAINTEXT*** to everyone it would be susceptible to header tampering, such as jamming the *sliding window size*, *round-trip time*, *FEC property* and *checksums*. ```AES-128``` is suggested for minimal encryption since modern CPUs are shipped with [AES-NI](https://en.wikipedia.org/wiki/AES_instruction_set) instructions and performs even better than `salsa20`(check the table above).
+
+Other possible attacks to kcp-go includes: a) [traffic analysis](https://en.wikipedia.org/wiki/Traffic_analysis), dataflow on specific websites may have pattern while interchanging data, but this type of eavesdropping has been mitigated by adapting [smux](https://github.com/xtaci/smux) to mix data streams so as to introduce noises, perfect solution to this has not appeared yet, theroretically by shuffling/mixing messages on larger scale network may mitigate this problem.  b) [replay attack](https://en.wikipedia.org/wiki/Replay_attack), since the asymmetrical encryption has not been introduced into kcp-go for some reason, capturing the packets and replay them on a different machine is possible, (notice: hijacking the session and decrypting the contents is still *impossible*), so upper layers should contain a asymmetrical encryption system to guarantee the authenticity of each message(to process message exactly once), such as HTTPS/OpenSSL/LibreSSL, only by signing the requests with private keys can eliminate this type of attack. 
+
 ## Connection Termination
 
 Control messages like **SYN/FIN/RST** in TCP **are not defined** in KCP, you need some **keepalive/heartbeat mechanism** in the application-level. A real world example is to use some **multiplexing** protocol over session, such as [smux](https://github.com/xtaci/smux)(with embedded keepalive mechanism), see [kcptun](https://github.com/xtaci/kcptun) for example.
@@ -169,6 +199,14 @@ Q: I'm handling >5K connections on my server, the CPU utilization is so high.
 
 A: A standalone `agent` or `gate` server for running kcp-go is suggested, not only for CPU utilization, but also important to the **precision** of RTT measurements(timing) which indirectly affects retransmission. By increasing update `interval` with `SetNoDelay` like `conn.SetNoDelay(1, 40, 1, 1)` will dramatically reduce system load, but lower the performance.
 
+Q: When should I enable FEC?
+
+A: Forward error correction is critical to long-distance transmission, because a packet loss will lead to a huge penalty in time. And for the complicated packet routing network in modern world, round-trip time based loss check will not always be efficient, the big deviation of RTT samples in the long way usually leads to a larger RTO value in typical rtt estimator, which in other words, slows down the transmission.
+  
+Q: Should I enable encryption?
+
+A: Yes, for the safety of protocol, even if the upper layer has encrypted.
+
 ## Who is using this?
 
 1. https://github.com/xtaci/kcptun -- A Secure Tunnel Based On KCP over UDP.

vendor/github.com/fatedier/kcp-go/batchconn.go

@@ -0,0 +1,12 @@
+package kcp
+
+import "golang.org/x/net/ipv4"
+
+const (
+	batchSize = 16
+)
+
+type batchConn interface {
+	WriteBatch(ms []ipv4.Message, flags int) (int, error)
+	ReadBatch(ms []ipv4.Message, flags int) (int, error)
+}

vendor/github.com/fatedier/kcp-go/fec.go

@@ -11,36 +11,34 @@ const (
 	fecHeaderSize      = 6
 	fecHeaderSizePlus2 = fecHeaderSize + 2 // plus 2B data size
 	typeData           = 0xf1
-	typeFEC            = 0xf2
+	typeParity         = 0xf2
 )
 
-type (
-	// fecPacket is a decoded FEC packet
-	fecPacket struct {
-		seqid uint32
-		flag  uint16
-		data  []byte
-	}
+// fecPacket is a decoded FEC packet
+type fecPacket []byte
 
-	// fecDecoder for decoding incoming packets
-	fecDecoder struct {
-		rxlimit      int // queue size limit
-		dataShards   int
-		parityShards int
-		shardSize    int
-		rx           []fecPacket // ordered receive queue
+func (bts fecPacket) seqid() uint32 { return binary.LittleEndian.Uint32(bts) }
+func (bts fecPacket) flag() uint16  { return binary.LittleEndian.Uint16(bts[4:]) }
+func (bts fecPacket) data() []byte  { return bts[6:] }
 
-		// caches
-		decodeCache [][]byte
-		flagCache   []bool
+// fecDecoder for decoding incoming packets
+type fecDecoder struct {
+	rxlimit      int // queue size limit
+	dataShards   int
+	parityShards int
+	shardSize    int
+	rx           []fecPacket // ordered receive queue
 
-		// zeros
-		zeros []byte
+	// caches
+	decodeCache [][]byte
+	flagCache   []bool
 
-		// RS decoder
-		codec reedsolomon.Encoder
-	}
-)
+	// zeros
+	zeros []byte
+
+	// RS decoder
+	codec reedsolomon.Encoder
+}
 
 func newFECDecoder(rxlimit, dataShards, parityShards int) *fecDecoder {
 	if dataShards <= 0 || parityShards <= 0 {
@@ -66,33 +64,24 @@ func newFECDecoder(rxlimit, dataShards, parityShards int) *fecDecoder {
 	return dec
 }
 
-// decodeBytes a fec packet
-func (dec *fecDecoder) decodeBytes(data []byte) fecPacket {
-	var pkt fecPacket
-	pkt.seqid = binary.LittleEndian.Uint32(data)
-	pkt.flag = binary.LittleEndian.Uint16(data[4:])
-	// allocate memory & copy
-	buf := xmitBuf.Get().([]byte)[:len(data)-6]
-	copy(buf, data[6:])
-	pkt.data = buf
-	return pkt
-}
-
 // decode a fec packet
-func (dec *fecDecoder) decode(pkt fecPacket) (recovered [][]byte) {
+func (dec *fecDecoder) decode(in fecPacket) (recovered [][]byte) {
 	// insertion
 	n := len(dec.rx) - 1
 	insertIdx := 0
 	for i := n; i >= 0; i-- {
-		if pkt.seqid == dec.rx[i].seqid { // de-duplicate
-			xmitBuf.Put(pkt.data)
+		if in.seqid() == dec.rx[i].seqid() { // de-duplicate
 			return nil
-		} else if _itimediff(pkt.seqid, dec.rx[i].seqid) > 0 { // insertion
+		} else if _itimediff(in.seqid(), dec.rx[i].seqid()) > 0 { // insertion
 			insertIdx = i + 1
 			break
 		}
 	}
 
+	// make a copy
+	pkt := fecPacket(xmitBuf.Get().([]byte)[:len(in)])
+	copy(pkt, in)
+
 	// insert into ordered rx queue
 	if insertIdx == n+1 {
 		dec.rx = append(dec.rx, pkt)
@@ -103,11 +92,11 @@ func (dec *fecDecoder) decode(pkt fecPacket) (recovered [][]byte) {
 	}
 
 	// shard range for current packet
-	shardBegin := pkt.seqid - pkt.seqid%uint32(dec.shardSize)
+	shardBegin := pkt.seqid() - pkt.seqid()%uint32(dec.shardSize)
 	shardEnd := shardBegin + uint32(dec.shardSize) - 1
 
 	// max search range in ordered queue for current shard
-	searchBegin := insertIdx - int(pkt.seqid%uint32(dec.shardSize))
+	searchBegin := insertIdx - int(pkt.seqid()%uint32(dec.shardSize))
 	if searchBegin < 0 {
 		searchBegin = 0
 	}
@@ -130,21 +119,21 @@ func (dec *fecDecoder) decode(pkt fecPacket) (recovered [][]byte) {
 
 		// shard assembly
 		for i := searchBegin; i <= searchEnd; i++ {
-			seqid := dec.rx[i].seqid
+			seqid := dec.rx[i].seqid()
 			if _itimediff(seqid, shardEnd) > 0 {
 				break
 			} else if _itimediff(seqid, shardBegin) >= 0 {
-				shards[seqid%uint32(dec.shardSize)] = dec.rx[i].data
+				shards[seqid%uint32(dec.shardSize)] = dec.rx[i].data()
 				shardsflag[seqid%uint32(dec.shardSize)] = true
 				numshard++
-				if dec.rx[i].flag == typeData {
+				if dec.rx[i].flag() == typeData {
 					numDataShard++
 				}
 				if numshard == 1 {
 					first = i
 				}
-				if len(dec.rx[i].data) > maxlen {
-					maxlen = len(dec.rx[i].data)
+				if len(dec.rx[i].data()) > maxlen {
+					maxlen = len(dec.rx[i].data())
 				}
 			}
 		}
@@ -159,11 +148,14 @@ func (dec *fecDecoder) decode(pkt fecPacket) (recovered [][]byte) {
 					dlen := len(shards[k])
 					shards[k] = shards[k][:maxlen]
 					copy(shards[k][dlen:], dec.zeros)
+				} else {
+					shards[k] = xmitBuf.Get().([]byte)[:0]
 				}
 			}
 			if err := dec.codec.ReconstructData(shards); err == nil {
 				for k := range shards[:dec.dataShards] {
 					if !shardsflag[k] {
+						// recovered data should be recycled
 						recovered = append(recovered, shards[k])
 					}
 				}
@@ -174,7 +166,7 @@ func (dec *fecDecoder) decode(pkt fecPacket) (recovered [][]byte) {
 
 	// keep rxlimit
 	if len(dec.rx) > dec.rxlimit {
-		if dec.rx[0].flag == typeData { // track the unrecoverable data
+		if dec.rx[0].flag() == typeData { // track the unrecoverable data
 			atomic.AddUint64(&DefaultSnmp.FECShortShards, 1)
 		}
 		dec.rx = dec.freeRange(0, 1, dec.rx)
@@ -182,15 +174,16 @@ func (dec *fecDecoder) decode(pkt fecPacket) (recovered [][]byte) {
 	return
 }
 
-// free a range of fecPacket, and zero for GC recycling
+// free a range of fecPacket
 func (dec *fecDecoder) freeRange(first, n int, q []fecPacket) []fecPacket {
 	for i := first; i < first+n; i++ { // recycle buffer
-		xmitBuf.Put(q[i].data)
+		xmitBuf.Put([]byte(q[i]))
+	}
+
+	if first == 0 && n < cap(q)/2 {
+		return q[n:]
 	}
 	copy(q[first:], q[first+n:])
-	for i := 0; i < n; i++ { // dereference data
-		q[len(q)-1-i].data = nil
-	}
 	return q[:len(q)-n]
 }
 
@@ -229,7 +222,7 @@ func newFECEncoder(dataShards, parityShards, offset int) *fecEncoder {
 	enc.dataShards = dataShards
 	enc.parityShards = parityShards
 	enc.shardSize = dataShards + parityShards
-	enc.paws = (0xffffffff/uint32(enc.shardSize) - 1) * uint32(enc.shardSize)
+	enc.paws = 0xffffffff / uint32(enc.shardSize) * uint32(enc.shardSize)
 	enc.headerOffset = offset
 	enc.payloadOffset = enc.headerOffset + fecHeaderSize
 
@@ -252,13 +245,16 @@ func newFECEncoder(dataShards, parityShards, offset int) *fecEncoder {
 // encodes the packet, outputs parity shards if we have collected quorum datashards
 // notice: the contents of 'ps' will be re-written in successive calling
 func (enc *fecEncoder) encode(b []byte) (ps [][]byte) {
+	// The header format:
+	// | FEC SEQID(4B) | FEC TYPE(2B) | SIZE (2B) | PAYLOAD(SIZE-2) |
+	// |<-headerOffset                |<-payloadOffset
 	enc.markData(b[enc.headerOffset:])
 	binary.LittleEndian.PutUint16(b[enc.payloadOffset:], uint16(len(b[enc.payloadOffset:])))
 
-	// copy data to fec datashards
+	// copy data from payloadOffset to fec shard cache
 	sz := len(b)
 	enc.shardCache[enc.shardCount] = enc.shardCache[enc.shardCount][:sz]
-	copy(enc.shardCache[enc.shardCount], b)
+	copy(enc.shardCache[enc.shardCount][enc.payloadOffset:], b[enc.payloadOffset:])
 	enc.shardCount++
 
 	// track max datashard length
@@ -285,7 +281,7 @@ func (enc *fecEncoder) encode(b []byte) (ps [][]byte) {
 		if err := enc.codec.Encode(cache); err == nil {
 			ps = enc.shardCache[enc.dataShards:]
 			for k := range ps {
-				enc.markFEC(ps[k][enc.headerOffset:])
+				enc.markParity(ps[k][enc.headerOffset:])
 				ps[k] = ps[k][:enc.maxSize]
 			}
 		}
@@ -304,8 +300,9 @@ func (enc *fecEncoder) markData(data []byte) {
 	enc.next++
 }
 
-func (enc *fecEncoder) markFEC(data []byte) {
+func (enc *fecEncoder) markParity(data []byte) {
 	binary.LittleEndian.PutUint32(data, enc.next)
-	binary.LittleEndian.PutUint16(data[4:], typeFEC)
+	binary.LittleEndian.PutUint16(data[4:], typeParity)
+	// sequence wrap will only happen at parity shard
 	enc.next = (enc.next + 1) % enc.paws
 }

vendor/github.com/fatedier/kcp-go/kcp.go

@@ -1,9 +1,9 @@
-// Package kcp - A Fast and Reliable ARQ Protocol
 package kcp
 
 import (
 	"encoding/binary"
 	"sync/atomic"
+	"time"
 )
 
 const (
@@ -30,6 +30,12 @@ const (
 	IKCP_PROBE_LIMIT = 120000 // up to 120 secs to probe window
 )
 
+// monotonic reference time point
+var refTime time.Time = time.Now()
+
+// currentMs returns current elasped monotonic milliseconds since program startup
+func currentMs() uint32 { return uint32(time.Now().Sub(refTime) / time.Millisecond) }
+
 // output_callback is a prototype which ought capture conn and call conn.Write
 type output_callback func(buf []byte, size int)
 
@@ -145,8 +151,9 @@ type KCP struct {
 
 	acklist []ackItem
 
-	buffer []byte
-	output output_callback
+	buffer   []byte
+	reserved int
+	output   output_callback
 }
 
 type ackItem struct {
@@ -154,8 +161,11 @@ type ackItem struct {
 	ts uint32
 }
 
-// NewKCP create a new kcp control object, 'conv' must equal in two endpoint
-// from the same connection.
+// NewKCP create a new kcp state machine
+//
+// 'conv' must be equal in the connection peers, or else data will be silently rejected.
+//
+// 'output' function will be called whenever these is data to be sent on wire.
 func NewKCP(conv uint32, output output_callback) *KCP {
 	kcp := new(KCP)
 	kcp.conv = conv
@@ -164,7 +174,7 @@ func NewKCP(conv uint32, output output_callback) *KCP {
 	kcp.rmt_wnd = IKCP_WND_RCV
 	kcp.mtu = IKCP_MTU_DEF
 	kcp.mss = kcp.mtu - IKCP_OVERHEAD
-	kcp.buffer = make([]byte, (kcp.mtu+IKCP_OVERHEAD)*3)
+	kcp.buffer = make([]byte, kcp.mtu)
 	kcp.rx_rto = IKCP_RTO_DEF
 	kcp.rx_minrto = IKCP_RTO_MIN
 	kcp.interval = IKCP_INTERVAL
@@ -189,6 +199,19 @@ func (kcp *KCP) delSegment(seg *segment) {
 	}
 }
 
+// ReserveBytes keeps n bytes untouched from the beginning of the buffer,
+// the output_callback function should be aware of this.
+//
+// Return false if n >= mss
+func (kcp *KCP) ReserveBytes(n int) bool {
+	if n >= int(kcp.mtu-IKCP_OVERHEAD) || n < 0 {
+		return false
+	}
+	kcp.reserved = n
+	kcp.mss = kcp.mtu - IKCP_OVERHEAD - uint32(n)
+	return true
+}
+
 // PeekSize checks the size of next message in the recv queue
 func (kcp *KCP) PeekSize() (length int) {
 	if len(kcp.rcv_queue) == 0 {
@@ -214,19 +237,21 @@ func (kcp *KCP) PeekSize() (length int) {
 	return
 }
 
-// Recv is user/upper level recv: returns size, returns below zero for EAGAIN
+// Receive data from kcp state machine
+//
+// Return number of bytes read.
+//
+// Return -1 when there is no readable data.
+//
+// Return -2 if len(buffer) is smaller than kcp.PeekSize().
 func (kcp *KCP) Recv(buffer []byte) (n int) {
-	if len(kcp.rcv_queue) == 0 {
+	peeksize := kcp.PeekSize()
+	if peeksize < 0 {
 		return -1
 	}
 
-	peeksize := kcp.PeekSize()
-	if peeksize < 0 {
-		return -2
-	}
-
 	if peeksize > len(buffer) {
-		return -3
+		return -2
 	}
 
 	var fast_recover bool
@@ -255,7 +280,7 @@ func (kcp *KCP) Recv(buffer []byte) (n int) {
 	count = 0
 	for k := range kcp.rcv_buf {
 		seg := &kcp.rcv_buf[k]
-		if seg.sn == kcp.rcv_nxt && len(kcp.rcv_queue) < int(kcp.rcv_wnd) {
+		if seg.sn == kcp.rcv_nxt && len(kcp.rcv_queue)+count < int(kcp.rcv_wnd) {
 			kcp.rcv_nxt++
 			count++
 		} else {
@@ -386,6 +411,10 @@ func (kcp *KCP) parse_ack(sn uint32) {
 	for k := range kcp.snd_buf {
 		seg := &kcp.snd_buf[k]
 		if sn == seg.sn {
+			// mark and free space, but leave the segment here,
+			// and wait until `una` to delete this, then we don't
+			// have to shift the segments behind forward,
+			// which is an expensive operation for large window
 			seg.acked = 1
 			kcp.delSegment(seg)
 			break
@@ -474,7 +503,7 @@ func (kcp *KCP) parse_data(newseg segment) bool {
 	count := 0
 	for k := range kcp.rcv_buf {
 		seg := &kcp.rcv_buf[k]
-		if seg.sn == kcp.rcv_nxt && len(kcp.rcv_queue) < int(kcp.rcv_wnd) {
+		if seg.sn == kcp.rcv_nxt && len(kcp.rcv_queue)+count < int(kcp.rcv_wnd) {
 			kcp.rcv_nxt++
 			count++
 		} else {
@@ -489,8 +518,12 @@ func (kcp *KCP) parse_data(newseg segment) bool {
 	return repeat
 }
 
-// Input when you received a low level packet (eg. UDP packet), call it
-// regular indicates a regular packet has received(not from FEC)
+// Input a packet into kcp state machine.
+//
+// 'regular' indicates it's a real data packet from remote, and it means it's not generated from ReedSolomon
+// codecs.
+//
+// 'ackNoDelay' will trigger immediate ACK, but surely it will not be efficient in bandwidth
 func (kcp *KCP) Input(data []byte, regular, ackNoDelay bool) int {
 	snd_una := kcp.snd_una
 	if len(data) < IKCP_OVERHEAD {
@@ -634,14 +667,28 @@ func (kcp *KCP) flush(ackOnly bool) uint32 {
 	seg.una = kcp.rcv_nxt
 
 	buffer := kcp.buffer
-	// flush acknowledges
-	ptr := buffer
-	for i, ack := range kcp.acklist {
+	ptr := buffer[kcp.reserved:] // keep n bytes untouched
+
+	// makeSpace makes room for writing
+	makeSpace := func(space int) {
 		size := len(buffer) - len(ptr)
-		if size+IKCP_OVERHEAD > int(kcp.mtu) {
+		if size+space > int(kcp.mtu) {
 			kcp.output(buffer, size)
-			ptr = buffer
+			ptr = buffer[kcp.reserved:]
 		}
+	}
+
+	// flush bytes in buffer if there is any
+	flushBuffer := func() {
+		size := len(buffer) - len(ptr)
+		if size > kcp.reserved {
+			kcp.output(buffer, size)
+		}
+	}
+
+	// flush acknowledges
+	for i, ack := range kcp.acklist {
+		makeSpace(IKCP_OVERHEAD)
 		// filter jitters caused by bufferbloat
 		if ack.sn >= kcp.rcv_nxt || len(kcp.acklist)-1 == i {
 			seg.sn, seg.ts = ack.sn, ack.ts
@@ -651,10 +698,7 @@ func (kcp *KCP) flush(ackOnly bool) uint32 {
 	kcp.acklist = kcp.acklist[0:0]
 
 	if ackOnly { // flash remain ack segments
-		size := len(buffer) - len(ptr)
-		if size > 0 {
-			kcp.output(buffer, size)
-		}
+		flushBuffer()
 		return kcp.interval
 	}
 
@@ -685,22 +729,14 @@ func (kcp *KCP) flush(ackOnly bool) uint32 {
 	// flush window probing commands
 	if (kcp.probe & IKCP_ASK_SEND) != 0 {
 		seg.cmd = IKCP_CMD_WASK
-		size := len(buffer) - len(ptr)
-		if size+IKCP_OVERHEAD > int(kcp.mtu) {
-			kcp.output(buffer, size)
-			ptr = buffer
-		}
+		makeSpace(IKCP_OVERHEAD)
 		ptr = seg.encode(ptr)
 	}
 
 	// flush window probing commands
 	if (kcp.probe & IKCP_ASK_TELL) != 0 {
 		seg.cmd = IKCP_CMD_WINS
-		size := len(buffer) - len(ptr)
-		if size+IKCP_OVERHEAD > int(kcp.mtu) {
-			kcp.output(buffer, size)
-			ptr = buffer
-		}
+		makeSpace(IKCP_OVERHEAD)
 		ptr = seg.encode(ptr)
 	}
 
@@ -779,20 +815,14 @@ func (kcp *KCP) flush(ackOnly bool) uint32 {
 		}
 
 		if needsend {
-			current = currentMs() // time update for a blocking call
+			current = currentMs()
 			segment.xmit++
 			segment.ts = current
 			segment.wnd = seg.wnd
 			segment.una = seg.una
 
-			size := len(buffer) - len(ptr)
 			need := IKCP_OVERHEAD + len(segment.data)
-
-			if size+need > int(kcp.mtu) {
-				kcp.output(buffer, size)
-				ptr = buffer
-			}
-
+			makeSpace(need)
 			ptr = segment.encode(ptr)
 			copy(ptr, segment.data)
 			ptr = ptr[len(segment.data):]
@@ -809,10 +839,7 @@ func (kcp *KCP) flush(ackOnly bool) uint32 {
 	}
 
 	// flash remain segments
-	size := len(buffer) - len(ptr)
-	if size > 0 {
-		kcp.output(buffer, size)
-	}
+	flushBuffer()
 
 	// counter updates
 	sum := lostSegs
@@ -864,6 +891,8 @@ func (kcp *KCP) flush(ackOnly bool) uint32 {
 	return uint32(minrto)
 }
 
+// (deprecated)
+//
 // Update updates state (call it repeatedly, every 10ms-100ms), or you can ask
 // ikcp_check when to call it again (without ikcp_input/_send calling).
 // 'current' - current timestamp in millisec.
@@ -892,6 +921,8 @@ func (kcp *KCP) Update() {
 	}
 }
 
+// (deprecated)
+//
 // Check determines when should you invoke ikcp_update:
 // returns when you should invoke ikcp_update in millisec, if there
 // is no ikcp_input/_send calling. you can call ikcp_update in that
@@ -947,12 +978,16 @@ func (kcp *KCP) SetMtu(mtu int) int {
 	if mtu < 50 || mtu < IKCP_OVERHEAD {
 		return -1
 	}
-	buffer := make([]byte, (mtu+IKCP_OVERHEAD)*3)
+	if kcp.reserved >= int(kcp.mtu-IKCP_OVERHEAD) || kcp.reserved < 0 {
+		return -1
+	}
+
+	buffer := make([]byte, mtu)
 	if buffer == nil {
 		return -2
 	}
 	kcp.mtu = uint32(mtu)
-	kcp.mss = kcp.mtu - IKCP_OVERHEAD
+	kcp.mss = kcp.mtu - IKCP_OVERHEAD - uint32(kcp.reserved)
 	kcp.buffer = buffer
 	return 0
 }
@@ -1006,7 +1041,13 @@ func (kcp *KCP) WaitSnd() int {
 }
 
 // remove front n elements from queue
+// if the number of elements to remove is more than half of the size.
+// just shift the rear elements to front, otherwise just reslice q to q[n:]
+// then the cost of runtime.growslice can always be less than n/2
 func (kcp *KCP) remove_front(q []segment, n int) []segment {
-	newn := copy(q, q[n:])
-	return q[:newn]
+	if n > cap(q)/2 {
+		newn := copy(q, q[n:])
+		return q[:newn]
+	}
+	return q[n:]
 }

vendor/github.com/fatedier/kcp-go/readloop.go

@@ -0,0 +1,48 @@
+package kcp
+
+import (
+	"sync/atomic"
+
+	"github.com/pkg/errors"
+)
+
+func (s *UDPSession) defaultReadLoop() {
+	buf := make([]byte, mtuLimit)
+	var src string
+	for {
+		if n, addr, err := s.conn.ReadFrom(buf); err == nil {
+			// make sure the packet is from the same source
+			if src == "" { // set source address
+				src = addr.String()
+			} else if addr.String() != src {
+				atomic.AddUint64(&DefaultSnmp.InErrs, 1)
+				continue
+			}
+
+			if n >= s.headerSize+IKCP_OVERHEAD {
+				s.packetInput(buf[:n])
+			} else {
+				atomic.AddUint64(&DefaultSnmp.InErrs, 1)
+			}
+		} else {
+			s.notifyReadError(errors.WithStack(err))
+			return
+		}
+	}
+}
+
+func (l *Listener) defaultMonitor() {
+	buf := make([]byte, mtuLimit)
+	for {
+		if n, from, err := l.conn.ReadFrom(buf); err == nil {
+			if n >= l.headerSize+IKCP_OVERHEAD {
+				l.packetInput(buf[:n], from)
+			} else {
+				atomic.AddUint64(&DefaultSnmp.InErrs, 1)
+			}
+		} else {
+			l.notifyReadError(errors.WithStack(err))
+			return
+		}
+	}
+}

vendor/github.com/fatedier/kcp-go/readloop_generic.go

@@ -0,0 +1,11 @@
+// +build !linux
+
+package kcp
+
+func (s *UDPSession) readLoop() {
+	s.defaultReadLoop()
+}
+
+func (l *Listener) monitor() {
+	l.defaultMonitor()
+}

vendor/github.com/fatedier/kcp-go/readloop_linux.go

@@ -0,0 +1,120 @@
+// +build linux
+
+package kcp
+
+import (
+	"net"
+	"os"
+	"sync/atomic"
+
+	"github.com/pkg/errors"
+	"golang.org/x/net/ipv4"
+	"golang.org/x/net/ipv6"
+)
+
+// the read loop for a client session
+func (s *UDPSession) readLoop() {
+	// default version
+	if s.xconn == nil {
+		s.defaultReadLoop()
+		return
+	}
+
+	// x/net version
+	var src string
+	msgs := make([]ipv4.Message, batchSize)
+	for k := range msgs {
+		msgs[k].Buffers = [][]byte{make([]byte, mtuLimit)}
+	}
+
+	for {
+		if count, err := s.xconn.ReadBatch(msgs, 0); err == nil {
+			for i := 0; i < count; i++ {
+				msg := &msgs[i]
+				// make sure the packet is from the same source
+				if src == "" { // set source address if nil
+					src = msg.Addr.String()
+				} else if msg.Addr.String() != src {
+					atomic.AddUint64(&DefaultSnmp.InErrs, 1)
+					continue
+				}
+
+				if msg.N < s.headerSize+IKCP_OVERHEAD {
+					atomic.AddUint64(&DefaultSnmp.InErrs, 1)
+					continue
+				}
+
+				// source and size has validated
+				s.packetInput(msg.Buffers[0][:msg.N])
+			}
+		} else {
+			// compatibility issue:
+			// for linux kernel<=2.6.32, support for sendmmsg is not available
+			// an error of type os.SyscallError will be returned
+			if operr, ok := err.(*net.OpError); ok {
+				if se, ok := operr.Err.(*os.SyscallError); ok {
+					if se.Syscall == "recvmmsg" {
+						s.defaultReadLoop()
+						return
+					}
+				}
+			}
+			s.notifyReadError(errors.WithStack(err))
+			return
+		}
+	}
+}
+
+// monitor incoming data for all connections of server
+func (l *Listener) monitor() {
+	var xconn batchConn
+	if _, ok := l.conn.(*net.UDPConn); ok {
+		addr, err := net.ResolveUDPAddr("udp", l.conn.LocalAddr().String())
+		if err == nil {
+			if addr.IP.To4() != nil {
+				xconn = ipv4.NewPacketConn(l.conn)
+			} else {
+				xconn = ipv6.NewPacketConn(l.conn)
+			}
+		}
+	}
+
+	// default version
+	if xconn == nil {
+		l.defaultMonitor()
+		return
+	}
+
+	// x/net version
+	msgs := make([]ipv4.Message, batchSize)
+	for k := range msgs {
+		msgs[k].Buffers = [][]byte{make([]byte, mtuLimit)}
+	}
+
+	for {
+		if count, err := xconn.ReadBatch(msgs, 0); err == nil {
+			for i := 0; i < count; i++ {
+				msg := &msgs[i]
+				if msg.N >= l.headerSize+IKCP_OVERHEAD {
+					l.packetInput(msg.Buffers[0][:msg.N], msg.Addr)
+				} else {
+					atomic.AddUint64(&DefaultSnmp.InErrs, 1)
+				}
+			}
+		} else {
+			// compatibility issue:
+			// for linux kernel<=2.6.32, support for sendmmsg is not available
+			// an error of type os.SyscallError will be returned
+			if operr, ok := err.(*net.OpError); ok {
+				if se, ok := operr.Err.(*os.SyscallError); ok {
+					if se.Syscall == "recvmmsg" {
+						l.defaultMonitor()
+						return
+					}
+				}
+			}
+			l.notifyReadError(errors.WithStack(err))
+			return
+		}
+	}
+}

vendor/github.com/fatedier/kcp-go/sess.go

@@ -1,9 +1,17 @@
+// Package kcp-go is a Reliable-UDP library for golang.
+//
+// This library intents to provide a smooth, resilient, ordered,
+// error-checked and anonymous delivery of streams over UDP packets.
+//
+// The interfaces of this package aims to be compatible with
+// net.Conn in standard library, but offers powerful features for advanced users.
 package kcp
 
 import (
 	"crypto/rand"
 	"encoding/binary"
 	"hash/crc32"
+	"io"
 	"net"
 	"sync"
 	"sync/atomic"
@@ -14,14 +22,6 @@ import (
 	"golang.org/x/net/ipv6"
 )
 
-type errTimeout struct {
-	error
-}
-
-func (errTimeout) Timeout() bool   { return true }
-func (errTimeout) Temporary() bool { return true }
-func (errTimeout) Error() string   { return "i/o timeout" }
-
 const (
 	// 16-bytes nonce for each packet
 	nonceSize = 16
@@ -42,9 +42,9 @@ const (
 	acceptBacklog = 128
 )
 
-const (
-	errBrokenPipe       = "broken pipe"
-	errInvalidOperation = "invalid operation"
+var (
+	errInvalidOperation = errors.New("invalid operation")
+	errTimeout          = errors.New("timeout")
 )
 
 var (
@@ -72,8 +72,6 @@ type (
 		// recvbuf turns packets into stream
 		recvbuf []byte
 		bufptr  []byte
-		// header extended output buffer, if has header
-		ext []byte
 
 		// FEC codec
 		fecDecoder *fecDecoder
@@ -90,16 +88,27 @@ type (
 
 		// notifications
 		die          chan struct{} // notify current session has Closed
+		dieOnce      sync.Once
 		chReadEvent  chan struct{} // notify Read() can be called without blocking
 		chWriteEvent chan struct{} // notify Write() can be called without blocking
-		chReadError  chan error    // notify PacketConn.Read() have an error
-		chWriteError chan error    // notify PacketConn.Write() have an error
+
+		// socket error handling
+		socketReadError      atomic.Value
+		socketWriteError     atomic.Value
+		chSocketReadError    chan struct{}
+		chSocketWriteError   chan struct{}
+		socketReadErrorOnce  sync.Once
+		socketWriteErrorOnce sync.Once
 
 		// nonce generator
 		nonce Entropy
 
-		isClosed bool // flag the session has Closed
-		mu       sync.Mutex
+		// packets waiting to be sent on wire
+		txqueue         []ipv4.Message
+		xconn           batchConn // for x/net
+		xconnWriteError error
+
+		mu sync.Mutex
 	}
 
 	setReadBuffer interface {
@@ -119,14 +128,26 @@ func newUDPSession(conv uint32, dataShards, parityShards int, l *Listener, conn
 	sess.nonce.Init()
 	sess.chReadEvent = make(chan struct{}, 1)
 	sess.chWriteEvent = make(chan struct{}, 1)
-	sess.chReadError = make(chan error, 1)
-	sess.chWriteError = make(chan error, 1)
+	sess.chSocketReadError = make(chan struct{})
+	sess.chSocketWriteError = make(chan struct{})
 	sess.remote = remote
 	sess.conn = conn
 	sess.l = l
 	sess.block = block
 	sess.recvbuf = make([]byte, mtuLimit)
 
+	// cast to writebatch conn
+	if _, ok := conn.(*net.UDPConn); ok {
+		addr, err := net.ResolveUDPAddr("udp", conn.LocalAddr().String())
+		if err == nil {
+			if addr.IP.To4() != nil {
+				sess.xconn = ipv4.NewPacketConn(conn)
+			} else {
+				sess.xconn = ipv6.NewPacketConn(conn)
+			}
+		}
+	}
+
 	// FEC codec initialization
 	sess.fecDecoder = newFECDecoder(rxFECMulti*(dataShards+parityShards), dataShards, parityShards)
 	if sess.block != nil {
@@ -143,17 +164,12 @@ func newUDPSession(conv uint32, dataShards, parityShards int, l *Listener, conn
 		sess.headerSize += fecHeaderSizePlus2
 	}
 
-	// we only need to allocate extended packet buffer if we have the additional header
-	if sess.headerSize > 0 {
-		sess.ext = make([]byte, mtuLimit)
-	}
-
 	sess.kcp = NewKCP(conv, func(buf []byte, size int) {
-		if size >= IKCP_OVERHEAD {
+		if size >= IKCP_OVERHEAD+sess.headerSize {
 			sess.output(buf[:size])
 		}
 	})
-	sess.kcp.SetMtu(IKCP_MTU_DEF - sess.headerSize)
+	sess.kcp.ReserveBytes(sess.headerSize)
 
 	// register current session to the global updater,
 	// which call sess.update() periodically.
@@ -165,6 +181,7 @@ func newUDPSession(conv uint32, dataShards, parityShards int, l *Listener, conn
 	} else {
 		atomic.AddUint64(&DefaultSnmp.PassiveOpens, 1)
 	}
+
 	currestab := atomic.AddUint64(&DefaultSnmp.CurrEstab, 1)
 	maxconn := atomic.LoadUint64(&DefaultSnmp.MaxConn)
 	if currestab > maxconn {
@@ -186,11 +203,6 @@ func (s *UDPSession) Read(b []byte) (n int, err error) {
 			return n, nil
 		}
 
-		if s.isClosed {
-			s.mu.Unlock()
-			return 0, errors.New(errBrokenPipe)
-		}
-
 		if size := s.kcp.PeekSize(); size > 0 { // peek data size from kcp
 			if len(b) >= size { // receive data into 'b' directly
 				s.kcp.Recv(b)
@@ -220,7 +232,7 @@ func (s *UDPSession) Read(b []byte) (n int, err error) {
 		if !s.rd.IsZero() {
 			if time.Now().After(s.rd) {
 				s.mu.Unlock()
-				return 0, errTimeout{}
+				return 0, errors.WithStack(errTimeout)
 			}
 
 			delay := s.rd.Sub(time.Now())
@@ -229,63 +241,66 @@ func (s *UDPSession) Read(b []byte) (n int, err error) {
 		}
 		s.mu.Unlock()
 
-		// wait for read event or timeout
+		// wait for read event or timeout or error
 		select {
 		case <-s.chReadEvent:
-		case <-c:
-		case <-s.die:
-		case err = <-s.chReadError:
 			if timeout != nil {
 				timeout.Stop()
 			}
-			return n, err
-		}
-
-		if timeout != nil {
-			timeout.Stop()
+		case <-c:
+			return 0, errors.WithStack(errTimeout)
+		case <-s.chSocketReadError:
+			return 0, s.socketReadError.Load().(error)
+		case <-s.die:
+			return 0, errors.WithStack(io.ErrClosedPipe)
 		}
 	}
 }
 
 // Write implements net.Conn
-func (s *UDPSession) Write(b []byte) (n int, err error) {
+func (s *UDPSession) Write(b []byte) (n int, err error) { return s.WriteBuffers([][]byte{b}) }
+
+// WriteBuffers write a vector of byte slices to the underlying connection
+func (s *UDPSession) WriteBuffers(v [][]byte) (n int, err error) {
 	for {
-		s.mu.Lock()
-		if s.isClosed {
-			s.mu.Unlock()
-			return 0, errors.New(errBrokenPipe)
+		select {
+		case <-s.chSocketWriteError:
+			return 0, s.socketWriteError.Load().(error)
+		case <-s.die:
+			return 0, errors.WithStack(io.ErrClosedPipe)
+		default:
 		}
 
-		// controls how much data will be sent to kcp core
-		// to prevent the memory from exhuasting
+		s.mu.Lock()
 		if s.kcp.WaitSnd() < int(s.kcp.snd_wnd) {
-			n = len(b)
-			for {
-				if len(b) <= int(s.kcp.mss) {
-					s.kcp.Send(b)
-					break
-				} else {
-					s.kcp.Send(b[:s.kcp.mss])
-					b = b[s.kcp.mss:]
+			for _, b := range v {
+				n += len(b)
+				for {
+					if len(b) <= int(s.kcp.mss) {
+						s.kcp.Send(b)
+						break
+					} else {
+						s.kcp.Send(b[:s.kcp.mss])
+						b = b[s.kcp.mss:]
+					}
 				}
 			}
 
-			// flush immediately if the queue is full
 			if s.kcp.WaitSnd() >= int(s.kcp.snd_wnd) || !s.writeDelay {
 				s.kcp.flush(false)
+				s.uncork()
 			}
 			s.mu.Unlock()
 			atomic.AddUint64(&DefaultSnmp.BytesSent, uint64(n))
 			return n, nil
 		}
 
-		// deadline for current writing operation
 		var timeout *time.Timer
 		var c <-chan time.Time
 		if !s.wd.IsZero() {
 			if time.Now().After(s.wd) {
 				s.mu.Unlock()
-				return 0, errTimeout{}
+				return 0, errors.WithStack(errTimeout)
 			}
 			delay := s.wd.Sub(time.Now())
 			timeout = time.NewTimer(delay)
@@ -293,44 +308,52 @@ func (s *UDPSession) Write(b []byte) (n int, err error) {
 		}
 		s.mu.Unlock()
 
-		// wait for write event or timeout
 		select {
 		case <-s.chWriteEvent:
-		case <-c:
-		case <-s.die:
-		case err = <-s.chWriteError:
 			if timeout != nil {
 				timeout.Stop()
 			}
-			return n, err
-		}
-
-		if timeout != nil {
-			timeout.Stop()
+		case <-c:
+			return 0, errors.WithStack(errTimeout)
+		case <-s.chSocketWriteError:
+			return 0, s.socketWriteError.Load().(error)
+		case <-s.die:
+			return 0, errors.WithStack(io.ErrClosedPipe)
 		}
 	}
 }
 
+// uncork sends data in txqueue if there is any
+func (s *UDPSession) uncork() {
+	if len(s.txqueue) > 0 {
+		s.tx(s.txqueue)
+		s.txqueue = s.txqueue[:0]
+	}
+	return
+}
+
 // Close closes the connection.
 func (s *UDPSession) Close() error {
-	// remove current session from updater & listener(if necessary)
-	updater.removeSession(s)
-	if s.l != nil { // notify listener
-		s.l.closeSession(s.remote)
-	}
+	var once bool
+	s.dieOnce.Do(func() {
+		close(s.die)
+		once = true
+	})
 
-	s.mu.Lock()
-	defer s.mu.Unlock()
-	if s.isClosed {
-		return errors.New(errBrokenPipe)
+	if once {
+		// remove from updater
+		updater.removeSession(s)
+		atomic.AddUint64(&DefaultSnmp.CurrEstab, ^uint64(0))
+
+		if s.l != nil { // belongs to listener
+			s.l.closeSession(s.remote)
+			return nil
+		} else { // client socket close
+			return s.conn.Close()
+		}
+	} else {
+		return errors.WithStack(io.ErrClosedPipe)
 	}
-	close(s.die)
-	s.isClosed = true
-	atomic.AddUint64(&DefaultSnmp.CurrEstab, ^uint64(0))
-	if s.l == nil { // client socket close
-		return s.conn.Close()
-	}
-	return nil
 }
 
 // LocalAddr returns the local network address. The Addr returned is shared by all invocations of LocalAddr, so do not modify it.
@@ -390,7 +413,7 @@ func (s *UDPSession) SetMtu(mtu int) bool {
 
 	s.mu.Lock()
 	defer s.mu.Unlock()
-	s.kcp.SetMtu(mtu - s.headerSize)
+	s.kcp.SetMtu(mtu)
 	return true
 }
 
@@ -412,7 +435,9 @@ func (s *UDPSession) SetACKNoDelay(nodelay bool) {
 	s.ackNoDelay = nodelay
 }
 
-// SetDUP duplicates udp packets for kcp output, for testing purpose only
+// (deprecated)
+//
+// SetDUP duplicates udp packets for kcp output.
 func (s *UDPSession) SetDUP(dup int) {
 	s.mu.Lock()
 	defer s.mu.Unlock()
@@ -427,19 +452,29 @@ func (s *UDPSession) SetNoDelay(nodelay, interval, resend, nc int) {
 	s.kcp.NoDelay(nodelay, interval, resend, nc)
 }
 
-// SetDSCP sets the 6bit DSCP field of IP header, no effect if it's accepted from Listener
+// SetDSCP sets the 6bit DSCP field in IPv4 header, or 8bit Traffic Class in IPv6 header.
+//
+// It has no effect if it's accepted from Listener.
 func (s *UDPSession) SetDSCP(dscp int) error {
 	s.mu.Lock()
 	defer s.mu.Unlock()
-	if s.l == nil {
-		if nc, ok := s.conn.(net.Conn); ok {
-			if err := ipv4.NewConn(nc).SetTOS(dscp << 2); err != nil {
-				return ipv6.NewConn(nc).SetTrafficClass(dscp)
-			}
+	if s.l != nil {
+		return errInvalidOperation
+	}
+	if nc, ok := s.conn.(net.Conn); ok {
+		var succeed bool
+		if err := ipv4.NewConn(nc).SetTOS(dscp << 2); err == nil {
+			succeed = true
+		}
+		if err := ipv6.NewConn(nc).SetTrafficClass(dscp); err == nil {
+			succeed = true
+		}
+
+		if succeed {
 			return nil
 		}
 	}
-	return errors.New(errInvalidOperation)
+	return errInvalidOperation
 }
 
 // SetReadBuffer sets the socket read buffer, no effect if it's accepted from Listener
@@ -451,7 +486,7 @@ func (s *UDPSession) SetReadBuffer(bytes int) error {
 			return nc.SetReadBuffer(bytes)
 		}
 	}
-	return errors.New(errInvalidOperation)
+	return errInvalidOperation
 }
 
 // SetWriteBuffer sets the socket write buffer, no effect if it's accepted from Listener
@@ -463,37 +498,29 @@ func (s *UDPSession) SetWriteBuffer(bytes int) error {
 			return nc.SetWriteBuffer(bytes)
 		}
 	}
-	return errors.New(errInvalidOperation)
+	return errInvalidOperation
 }
 
 // post-processing for sending a packet from kcp core
 // steps:
-// 0. Header extending
 // 1. FEC packet generation
 // 2. CRC32 integrity
 // 3. Encryption
-// 4. WriteTo kernel
+// 4. TxQueue
 func (s *UDPSession) output(buf []byte) {
 	var ecc [][]byte
 
-	// 0. extend buf's header space(if necessary)
-	ext := buf
-	if s.headerSize > 0 {
-		ext = s.ext[:s.headerSize+len(buf)]
-		copy(ext[s.headerSize:], buf)
-	}
-
 	// 1. FEC encoding
 	if s.fecEncoder != nil {
-		ecc = s.fecEncoder.encode(ext)
+		ecc = s.fecEncoder.encode(buf)
 	}
 
 	// 2&3. crc32 & encryption
 	if s.block != nil {
-		s.nonce.Fill(ext[:nonceSize])
-		checksum := crc32.ChecksumIEEE(ext[cryptHeaderSize:])
-		binary.LittleEndian.PutUint32(ext[nonceSize:], checksum)
-		s.block.Encrypt(ext, ext)
+		s.nonce.Fill(buf[:nonceSize])
+		checksum := crc32.ChecksumIEEE(buf[cryptHeaderSize:])
+		binary.LittleEndian.PutUint32(buf[nonceSize:], checksum)
+		s.block.Encrypt(buf, buf)
 
 		for k := range ecc {
 			s.nonce.Fill(ecc[k][:nonceSize])
@@ -503,28 +530,23 @@ func (s *UDPSession) output(buf []byte) {
 		}
 	}
 
-	// 4. WriteTo kernel
-	nbytes := 0
-	npkts := 0
+	// 4. TxQueue
+	var msg ipv4.Message
 	for i := 0; i < s.dup+1; i++ {
-		if n, err := s.conn.WriteTo(ext, s.remote); err == nil {
-			nbytes += n
-			npkts++
-		} else {
-			s.notifyWriteError(err)
-		}
+		bts := xmitBuf.Get().([]byte)[:len(buf)]
+		copy(bts, buf)
+		msg.Buffers = [][]byte{bts}
+		msg.Addr = s.remote
+		s.txqueue = append(s.txqueue, msg)
 	}
 
 	for k := range ecc {
-		if n, err := s.conn.WriteTo(ecc[k], s.remote); err == nil {
-			nbytes += n
-			npkts++
-		} else {
-			s.notifyWriteError(err)
-		}
+		bts := xmitBuf.Get().([]byte)[:len(ecc[k])]
+		copy(bts, ecc[k])
+		msg.Buffers = [][]byte{bts}
+		msg.Addr = s.remote
+		s.txqueue = append(s.txqueue, msg)
 	}
-	atomic.AddUint64(&DefaultSnmp.OutPkts, uint64(npkts))
-	atomic.AddUint64(&DefaultSnmp.OutBytes, uint64(nbytes))
 }
 
 // kcp update, returns interval for next calling
@@ -535,6 +557,7 @@ func (s *UDPSession) update() (interval time.Duration) {
 	if s.kcp.WaitSnd() < waitsnd {
 		s.notifyWriteEvent()
 	}
+	s.uncork()
 	s.mu.Unlock()
 	return
 }
@@ -556,10 +579,39 @@ func (s *UDPSession) notifyWriteEvent() {
 	}
 }
 
+func (s *UDPSession) notifyReadError(err error) {
+	s.socketReadErrorOnce.Do(func() {
+		s.socketReadError.Store(err)
+		close(s.chSocketReadError)
+	})
+}
+
 func (s *UDPSession) notifyWriteError(err error) {
-	select {
-	case s.chWriteError <- err:
-	default:
+	s.socketWriteErrorOnce.Do(func() {
+		s.socketWriteError.Store(err)
+		close(s.chSocketWriteError)
+	})
+}
+
+// packet input stage
+func (s *UDPSession) packetInput(data []byte) {
+	dataValid := false
+	if s.block != nil {
+		s.block.Decrypt(data, data)
+		data = data[nonceSize:]
+		checksum := crc32.ChecksumIEEE(data[crcSize:])
+		if checksum == binary.LittleEndian.Uint32(data) {
+			data = data[crcSize:]
+			dataValid = true
+		} else {
+			atomic.AddUint64(&DefaultSnmp.InCsumErrors, 1)
+		}
+	} else if s.block == nil {
+		dataValid = true
+	}
+
+	if dataValid {
+		s.kcpInput(data)
 	}
 }
 
@@ -568,16 +620,16 @@ func (s *UDPSession) kcpInput(data []byte) {
 
 	if s.fecDecoder != nil {
 		if len(data) > fecHeaderSize { // must be larger than fec header size
-			f := s.fecDecoder.decodeBytes(data)
-			if f.flag == typeData || f.flag == typeFEC { // header check
-				if f.flag == typeFEC {
+			f := fecPacket(data)
+			if f.flag() == typeData || f.flag() == typeParity { // header check
+				if f.flag() == typeParity {
 					fecParityShards++
 				}
 				recovers := s.fecDecoder.decode(f)
 
 				s.mu.Lock()
 				waitsnd := s.kcp.WaitSnd()
-				if f.flag == typeData {
+				if f.flag() == typeData {
 					if ret := s.kcp.Input(data[fecHeaderSizePlus2:], true, s.ackNoDelay); ret != 0 {
 						kcpInErrors++
 					}
@@ -598,6 +650,8 @@ func (s *UDPSession) kcpInput(data []byte) {
 					} else {
 						fecErrs++
 					}
+					// recycle the recovers
+					xmitBuf.Put(r)
 				}
 
 				// to notify the readers to receive the data
@@ -608,6 +662,7 @@ func (s *UDPSession) kcpInput(data []byte) {
 				if s.kcp.WaitSnd() < waitsnd {
 					s.notifyWriteEvent()
 				}
+				s.uncork()
 				s.mu.Unlock()
 			} else {
 				atomic.AddUint64(&DefaultSnmp.InErrs, 1)
@@ -627,6 +682,7 @@ func (s *UDPSession) kcpInput(data []byte) {
 		if s.kcp.WaitSnd() < waitsnd {
 			s.notifyWriteEvent()
 		}
+		s.uncork()
 		s.mu.Unlock()
 	}
 
@@ -644,50 +700,7 @@ func (s *UDPSession) kcpInput(data []byte) {
 	if fecRecovered > 0 {
 		atomic.AddUint64(&DefaultSnmp.FECRecovered, fecRecovered)
 	}
-}
 
-// the read loop for a client session
-func (s *UDPSession) readLoop() {
-	buf := make([]byte, mtuLimit)
-	var src string
-	for {
-		if n, addr, err := s.conn.ReadFrom(buf); err == nil {
-			// make sure the packet is from the same source
-			if src == "" { // set source address
-				src = addr.String()
-			} else if addr.String() != src {
-				atomic.AddUint64(&DefaultSnmp.InErrs, 1)
-				continue
-			}
-
-			if n >= s.headerSize+IKCP_OVERHEAD {
-				data := buf[:n]
-				dataValid := false
-				if s.block != nil {
-					s.block.Decrypt(data, data)
-					data = data[nonceSize:]
-					checksum := crc32.ChecksumIEEE(data[crcSize:])
-					if checksum == binary.LittleEndian.Uint32(data) {
-						data = data[crcSize:]
-						dataValid = true
-					} else {
-						atomic.AddUint64(&DefaultSnmp.InCsumErrors, 1)
-					}
-				} else if s.block == nil {
-					dataValid = true
-				}
-
-				if dataValid {
-					s.kcpInput(data)
-				}
-			} else {
-				atomic.AddUint64(&DefaultSnmp.InErrs, 1)
-			}
-		} else {
-			s.chReadError <- err
-			return
-		}
-	}
 }
 
 type (
@@ -704,98 +717,91 @@ type (
 		chAccepts       chan *UDPSession // Listen() backlog
 		chSessionClosed chan net.Addr    // session close queue
 		headerSize      int              // the additional header to a KCP frame
-		die             chan struct{}    // notify the listener has closed
-		rd              atomic.Value     // read deadline for Accept()
-		wd              atomic.Value
+
+		die     chan struct{} // notify the listener has closed
+		dieOnce sync.Once
+
+		// socket error handling
+		socketReadError     atomic.Value
+		chSocketReadError   chan struct{}
+		socketReadErrorOnce sync.Once
+
+		rd atomic.Value // read deadline for Accept()
 	}
 )
 
-// monitor incoming data for all connections of server
-func (l *Listener) monitor() {
-	// a cache for session object last used
-	var lastAddr string
-	var lastSession *UDPSession
-	buf := make([]byte, mtuLimit)
-	for {
-		if n, from, err := l.conn.ReadFrom(buf); err == nil {
-			if n >= l.headerSize+IKCP_OVERHEAD {
-				data := buf[:n]
-				dataValid := false
-				if l.block != nil {
-					l.block.Decrypt(data, data)
-					data = data[nonceSize:]
-					checksum := crc32.ChecksumIEEE(data[crcSize:])
-					if checksum == binary.LittleEndian.Uint32(data) {
-						data = data[crcSize:]
-						dataValid = true
-					} else {
-						atomic.AddUint64(&DefaultSnmp.InCsumErrors, 1)
+// packet input stage
+func (l *Listener) packetInput(data []byte, addr net.Addr) {
+	dataValid := false
+	if l.block != nil {
+		l.block.Decrypt(data, data)
+		data = data[nonceSize:]
+		checksum := crc32.ChecksumIEEE(data[crcSize:])
+		if checksum == binary.LittleEndian.Uint32(data) {
+			data = data[crcSize:]
+			dataValid = true
+		} else {
+			atomic.AddUint64(&DefaultSnmp.InCsumErrors, 1)
+		}
+	} else if l.block == nil {
+		dataValid = true
+	}
+
+	if dataValid {
+		l.sessionLock.Lock()
+		s, ok := l.sessions[addr.String()]
+		l.sessionLock.Unlock()
+
+		if !ok { // new address:port
+			if len(l.chAccepts) < cap(l.chAccepts) { // do not let the new sessions overwhelm accept queue
+				var conv uint32
+				convValid := false
+				if l.fecDecoder != nil {
+					isfec := binary.LittleEndian.Uint16(data[4:])
+					if isfec == typeData {
+						conv = binary.LittleEndian.Uint32(data[fecHeaderSizePlus2:])
+						convValid = true
 					}
-				} else if l.block == nil {
-					dataValid = true
+				} else {
+					conv = binary.LittleEndian.Uint32(data)
+					convValid = true
 				}
 
-				if dataValid {
-					addr := from.String()
-					var s *UDPSession
-					var ok bool
-
-					// the packets received from an address always come in batch,
-					// cache the session for next packet, without querying map.
-					if addr == lastAddr {
-						s, ok = lastSession, true
-					} else {
-						l.sessionLock.Lock()
-						if s, ok = l.sessions[addr]; ok {
-							lastSession = s
-							lastAddr = addr
-						}
-						l.sessionLock.Unlock()
-					}
-
-					if !ok { // new session
-						if len(l.chAccepts) < cap(l.chAccepts) { // do not let the new sessions overwhelm accept queue
-							var conv uint32
-							convValid := false
-							if l.fecDecoder != nil {
-								isfec := binary.LittleEndian.Uint16(data[4:])
-								if isfec == typeData {
-									conv = binary.LittleEndian.Uint32(data[fecHeaderSizePlus2:])
-									convValid = true
-								}
-							} else {
-								conv = binary.LittleEndian.Uint32(data)
-								convValid = true
-							}
-
-							if convValid { // creates a new session only if the 'conv' field in kcp is accessible
-								s := newUDPSession(conv, l.dataShards, l.parityShards, l, l.conn, from, l.block)
-								s.kcpInput(data)
-								l.sessionLock.Lock()
-								l.sessions[addr] = s
-								l.sessionLock.Unlock()
-								l.chAccepts <- s
-							}
-						}
-					} else {
-						s.kcpInput(data)
-					}
+				if convValid { // creates a new session only if the 'conv' field in kcp is accessible
+					s := newUDPSession(conv, l.dataShards, l.parityShards, l, l.conn, addr, l.block)
+					s.kcpInput(data)
+					l.sessionLock.Lock()
+					l.sessions[addr.String()] = s
+					l.sessionLock.Unlock()
+					l.chAccepts <- s
 				}
-			} else {
-				atomic.AddUint64(&DefaultSnmp.InErrs, 1)
 			}
 		} else {
-			return
+			s.kcpInput(data)
 		}
 	}
 }
 
+func (l *Listener) notifyReadError(err error) {
+	l.socketReadErrorOnce.Do(func() {
+		l.socketReadError.Store(err)
+		close(l.chSocketReadError)
+
+		// propagate read error to all sessions
+		l.sessionLock.Lock()
+		for _, s := range l.sessions {
+			s.notifyReadError(err)
+		}
+		l.sessionLock.Unlock()
+	})
+}
+
 // SetReadBuffer sets the socket read buffer for the Listener
 func (l *Listener) SetReadBuffer(bytes int) error {
 	if nc, ok := l.conn.(setReadBuffer); ok {
 		return nc.SetReadBuffer(bytes)
 	}
-	return errors.New(errInvalidOperation)
+	return errInvalidOperation
 }
 
 // SetWriteBuffer sets the socket write buffer for the Listener
@@ -803,18 +809,25 @@ func (l *Listener) SetWriteBuffer(bytes int) error {
 	if nc, ok := l.conn.(setWriteBuffer); ok {
 		return nc.SetWriteBuffer(bytes)
 	}
-	return errors.New(errInvalidOperation)
+	return errInvalidOperation
 }
 
-// SetDSCP sets the 6bit DSCP field of IP header
+// SetDSCP sets the 6bit DSCP field in IPv4 header, or 8bit Traffic Class in IPv6 header.
 func (l *Listener) SetDSCP(dscp int) error {
 	if nc, ok := l.conn.(net.Conn); ok {
-		if err := ipv4.NewConn(nc).SetTOS(dscp << 2); err != nil {
-			return ipv6.NewConn(nc).SetTrafficClass(dscp)
+		var succeed bool
+		if err := ipv4.NewConn(nc).SetTOS(dscp << 2); err == nil {
+			succeed = true
+		}
+		if err := ipv6.NewConn(nc).SetTrafficClass(dscp); err == nil {
+			succeed = true
+		}
+
+		if succeed {
+			return nil
 		}
-		return nil
 	}
-	return errors.New(errInvalidOperation)
+	return errInvalidOperation
 }
 
 // Accept implements the Accept method in the Listener interface; it waits for the next call and returns a generic Conn.
@@ -831,11 +844,13 @@ func (l *Listener) AcceptKCP() (*UDPSession, error) {
 
 	select {
 	case <-timeout:
-		return nil, &errTimeout{}
+		return nil, errors.WithStack(errTimeout)
 	case c := <-l.chAccepts:
 		return c, nil
+	case <-l.chSocketReadError:
+		return nil, l.socketReadError.Load().(error)
 	case <-l.die:
-		return nil, errors.New(errBrokenPipe)
+		return nil, errors.WithStack(io.ErrClosedPipe)
 	}
 }
 
@@ -853,15 +868,21 @@ func (l *Listener) SetReadDeadline(t time.Time) error {
 }
 
 // SetWriteDeadline implements the Conn SetWriteDeadline method.
-func (l *Listener) SetWriteDeadline(t time.Time) error {
-	l.wd.Store(t)
-	return nil
-}
+func (l *Listener) SetWriteDeadline(t time.Time) error { return errInvalidOperation }
 
-// Close stops listening on the UDP address. Already Accepted connections are not closed.
+// Close stops listening on the UDP address, and closes the socket
 func (l *Listener) Close() error {
-	close(l.die)
-	return l.conn.Close()
+	var once bool
+	l.dieOnce.Do(func() {
+		close(l.die)
+		once = true
+	})
+
+	if once {
+		return l.conn.Close()
+	} else {
+		return errors.WithStack(io.ErrClosedPipe)
+	}
 }
 
 // closeSession notify the listener that a session has closed
@@ -881,16 +902,21 @@ func (l *Listener) Addr() net.Addr { return l.conn.LocalAddr() }
 // Listen listens for incoming KCP packets addressed to the local address laddr on the network "udp",
 func Listen(laddr string) (net.Listener, error) { return ListenWithOptions(laddr, nil, 0, 0) }
 
-// ListenWithOptions listens for incoming KCP packets addressed to the local address laddr on the network "udp" with packet encryption,
-// dataShards, parityShards defines Reed-Solomon Erasure Coding parameters
+// ListenWithOptions listens for incoming KCP packets addressed to the local address laddr on the network "udp" with packet encryption.
+//
+// 'block' is the block encryption algorithm to encrypt packets.
+//
+// 'dataShards', 'parityShards' specifiy how many parity packets will be generated following the data packets.
+//
+// Check https://github.com/klauspost/reedsolomon for details
 func ListenWithOptions(laddr string, block BlockCrypt, dataShards, parityShards int) (*Listener, error) {
 	udpaddr, err := net.ResolveUDPAddr("udp", laddr)
 	if err != nil {
-		return nil, errors.Wrap(err, "net.ResolveUDPAddr")
+		return nil, errors.WithStack(err)
 	}
 	conn, err := net.ListenUDP("udp", udpaddr)
 	if err != nil {
-		return nil, errors.Wrap(err, "net.ListenUDP")
+		return nil, errors.WithStack(err)
 	}
 
 	return ServeConn(block, dataShards, parityShards, conn)
@@ -908,6 +934,7 @@ func ServeConn(block BlockCrypt, dataShards, parityShards int, conn net.PacketCo
 	l.parityShards = parityShards
 	l.block = block
 	l.fecDecoder = newFECDecoder(rxFECMulti*(dataShards+parityShards), dataShards, parityShards)
+	l.chSocketReadError = make(chan struct{})
 
 	// calculate header size
 	if l.block != nil {
@@ -921,15 +948,21 @@ func ServeConn(block BlockCrypt, dataShards, parityShards int, conn net.PacketCo
 	return l, nil
 }
 
-// Dial connects to the remote address "raddr" on the network "udp"
+// Dial connects to the remote address "raddr" on the network "udp" without encryption and FEC
 func Dial(raddr string) (net.Conn, error) { return DialWithOptions(raddr, nil, 0, 0) }
 
 // DialWithOptions connects to the remote address "raddr" on the network "udp" with packet encryption
+//
+// 'block' is the block encryption algorithm to encrypt packets.
+//
+// 'dataShards', 'parityShards' specifiy how many parity packets will be generated following the data packets.
+//
+// Check https://github.com/klauspost/reedsolomon for details
 func DialWithOptions(raddr string, block BlockCrypt, dataShards, parityShards int) (*UDPSession, error) {
 	// network type detection
 	udpaddr, err := net.ResolveUDPAddr("udp", raddr)
 	if err != nil {
-		return nil, errors.Wrap(err, "net.ResolveUDPAddr")
+		return nil, errors.WithStack(err)
 	}
 	network := "udp4"
 	if udpaddr.IP.To4() == nil {
@@ -938,30 +971,33 @@ func DialWithOptions(raddr string, block BlockCrypt, dataShards, parityShards in
 
 	conn, err := net.ListenUDP(network, nil)
 	if err != nil {
-		return nil, errors.Wrap(err, "net.DialUDP")
+		return nil, errors.WithStack(err)
 	}
 
 	return NewConn(raddr, block, dataShards, parityShards, conn)
 }
 
+// NewConn3 establishes a session and talks KCP protocol over a packet connection.
+func NewConn3(convid uint32, raddr net.Addr, block BlockCrypt, dataShards, parityShards int, conn net.PacketConn) (*UDPSession, error) {
+	return newUDPSession(convid, dataShards, parityShards, nil, conn, raddr, block), nil
+}
+
+// NewConn2 establishes a session and talks KCP protocol over a packet connection.
+func NewConn2(raddr net.Addr, block BlockCrypt, dataShards, parityShards int, conn net.PacketConn) (*UDPSession, error) {
+	var convid uint32
+	binary.Read(rand.Reader, binary.LittleEndian, &convid)
+	return NewConn3(convid, raddr, block, dataShards, parityShards, conn)
+}
+
 // NewConn establishes a session and talks KCP protocol over a packet connection.
 func NewConn(raddr string, block BlockCrypt, dataShards, parityShards int, conn net.PacketConn) (*UDPSession, error) {
 	udpaddr, err := net.ResolveUDPAddr("udp", raddr)
 	if err != nil {
-		return nil, errors.Wrap(err, "net.ResolveUDPAddr")
+		return nil, errors.WithStack(err)
 	}
-
-	var convid uint32
-	binary.Read(rand.Reader, binary.LittleEndian, &convid)
-	return newUDPSession(convid, dataShards, parityShards, nil, conn, udpaddr, block), nil
+	return NewConn2(udpaddr, block, dataShards, parityShards, conn)
 }
 
-// monotonic reference time point
-var refTime time.Time = time.Now()
-
-// currentMs returns current elasped monotonic milliseconds since program startup
-func currentMs() uint32 { return uint32(time.Now().Sub(refTime) / time.Millisecond) }
-
 func NewConnEx(convid uint32, connected bool, raddr string, block BlockCrypt, dataShards, parityShards int, conn *net.UDPConn) (*UDPSession, error) {
 	udpaddr, err := net.ResolveUDPAddr("udp", raddr)
 	if err != nil {

vendor/github.com/fatedier/kcp-go/tx.go

@@ -0,0 +1,25 @@
+package kcp
+
+import (
+	"sync/atomic"
+
+	"github.com/pkg/errors"
+	"golang.org/x/net/ipv4"
+)
+
+func (s *UDPSession) defaultTx(txqueue []ipv4.Message) {
+	nbytes := 0
+	npkts := 0
+	for k := range txqueue {
+		if n, err := s.conn.WriteTo(txqueue[k].Buffers[0], txqueue[k].Addr); err == nil {
+			nbytes += n
+			npkts++
+			xmitBuf.Put(txqueue[k].Buffers[0])
+		} else {
+			s.notifyWriteError(errors.WithStack(err))
+			break
+		}
+	}
+	atomic.AddUint64(&DefaultSnmp.OutPkts, uint64(npkts))
+	atomic.AddUint64(&DefaultSnmp.OutBytes, uint64(nbytes))
+}

vendor/github.com/fatedier/kcp-go/tx_generic.go

@@ -0,0 +1,11 @@
+// +build !linux
+
+package kcp
+
+import (
+	"golang.org/x/net/ipv4"
+)
+
+func (s *UDPSession) tx(txqueue []ipv4.Message) {
+	s.defaultTx(txqueue)
+}

vendor/github.com/fatedier/kcp-go/tx_linux.go

@@ -0,0 +1,52 @@
+// +build linux
+
+package kcp
+
+import (
+	"net"
+	"os"
+	"sync/atomic"
+
+	"github.com/pkg/errors"
+	"golang.org/x/net/ipv4"
+)
+
+func (s *UDPSession) tx(txqueue []ipv4.Message) {
+	// default version
+	if s.xconn == nil || s.xconnWriteError != nil {
+		s.defaultTx(txqueue)
+		return
+	}
+
+	// x/net version
+	nbytes := 0
+	npkts := 0
+	for len(txqueue) > 0 {
+		if n, err := s.xconn.WriteBatch(txqueue, 0); err == nil {
+			for k := range txqueue[:n] {
+				nbytes += len(txqueue[k].Buffers[0])
+				xmitBuf.Put(txqueue[k].Buffers[0])
+			}
+			npkts += n
+			txqueue = txqueue[n:]
+		} else {
+			// compatibility issue:
+			// for linux kernel<=2.6.32, support for sendmmsg is not available
+			// an error of type os.SyscallError will be returned
+			if operr, ok := err.(*net.OpError); ok {
+				if se, ok := operr.Err.(*os.SyscallError); ok {
+					if se.Syscall == "sendmmsg" {
+						s.xconnWriteError = se
+						s.defaultTx(txqueue)
+						return
+					}
+				}
+			}
+			s.notifyWriteError(errors.WithStack(err))
+			break
+		}
+	}
+
+	atomic.AddUint64(&DefaultSnmp.OutPkts, uint64(npkts))
+	atomic.AddUint64(&DefaultSnmp.OutBytes, uint64(nbytes))
+}

vendor/github.com/fatedier/kcp-go/updater.go

@@ -76,10 +76,10 @@ func (h *updateHeap) wakeup() {
 }
 
 func (h *updateHeap) updateTask() {
-	var timer <-chan time.Time
+	timer := time.NewTimer(0)
 	for {
 		select {
-		case <-timer:
+		case <-timer.C:
 		case <-h.chWakeUp:
 		}
 
@@ -87,7 +87,7 @@ func (h *updateHeap) updateTask() {
 		hlen := h.Len()
 		for i := 0; i < hlen; i++ {
 			entry := &h.entries[0]
-			if time.Now().After(entry.ts) {
+			if !time.Now().Before(entry.ts) {
 				interval := entry.s.update()
 				entry.ts = time.Now().Add(interval)
 				heap.Fix(h, 0)
@@ -97,7 +97,7 @@ func (h *updateHeap) updateTask() {
 		}
 
 		if hlen > 0 {
-			timer = time.After(h.entries[0].ts.Sub(time.Now()))
+			timer.Reset(h.entries[0].ts.Sub(time.Now()))
 		}
 		h.mu.Unlock()
 	}

vendor/github.com/gorilla/context/.travis.yml

@@ -1,19 +0,0 @@
-language: go
-sudo: false
-
-matrix:
-  include:
-    - go: 1.3
-    - go: 1.4
-    - go: 1.5
-    - go: 1.6
-    - go: 1.7
-    - go: tip
-  allow_failures:
-    - go: tip
-
-script:
-  - go get -t -v ./...
-  - diff -u <(echo -n) <(gofmt -d .)
-  - go vet $(go list ./... | grep -v /vendor/)
-  - go test -v -race ./...

vendor/github.com/gorilla/context/README.md

@@ -1,10 +0,0 @@
-context
-=======
-[![Build Status](https://travis-ci.org/gorilla/context.png?branch=master)](https://travis-ci.org/gorilla/context)
-
-gorilla/context is a general purpose registry for global request variables.
-
-> Note: gorilla/context, having been born well before `context.Context` existed, does not play well
-> with the shallow copying of the request that [`http.Request.WithContext`](https://golang.org/pkg/net/http/#Request.WithContext) (added to net/http Go 1.7 onwards) performs. You should either use *just* gorilla/context, or moving forward, the new `http.Request.Context()`.
-
-Read the full documentation here: http://www.gorillatoolkit.org/pkg/context