traefik/docs/benchmarks.md

214 lines
5.4 KiB
Markdown

# Benchmarks
## Configuration
I would like to thanks [vincentbernat](https://github.com/vincentbernat) from [exoscale.ch](https://www.exoscale.ch) who kindly provided the infrastructure needed for the benchmarks.
I used 4 VMs for the tests with the following configuration:
- 32 GB RAM
- 8 CPU Cores
- 10 GB SSD
- Ubuntu 14.04 LTS 64-bit
## Setup
1. One VM used to launch the benchmarking tool [wrk](https://github.com/wg/wrk)
2. One VM for Traefik (v1.0.0-beta.416) / nginx (v1.4.6)
3. Two VMs for 2 backend servers in go [whoami](https://github.com/containous/whoami/)
Each VM has been tuned using the following limits:
```bash
sysctl -w fs.file-max="9999999"
sysctl -w fs.nr_open="9999999"
sysctl -w net.core.netdev_max_backlog="4096"
sysctl -w net.core.rmem_max="16777216"
sysctl -w net.core.somaxconn="65535"
sysctl -w net.core.wmem_max="16777216"
sysctl -w net.ipv4.ip_local_port_range="1025 65535"
sysctl -w net.ipv4.tcp_fin_timeout="30"
sysctl -w net.ipv4.tcp_keepalive_time="30"
sysctl -w net.ipv4.tcp_max_syn_backlog="20480"
sysctl -w net.ipv4.tcp_max_tw_buckets="400000"
sysctl -w net.ipv4.tcp_no_metrics_save="1"
sysctl -w net.ipv4.tcp_syn_retries="2"
sysctl -w net.ipv4.tcp_synack_retries="2"
sysctl -w net.ipv4.tcp_tw_recycle="1"
sysctl -w net.ipv4.tcp_tw_reuse="1"
sysctl -w vm.min_free_kbytes="65536"
sysctl -w vm.overcommit_memory="1"
ulimit -n 9999999
```
### Nginx
Here is the config Nginx file use `/etc/nginx/nginx.conf`:
```
user www-data;
worker_processes auto;
worker_rlimit_nofile 200000;
pid /var/run/nginx.pid;
events {
worker_connections 10000;
use epoll;
multi_accept on;
}
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 300;
keepalive_requests 10000;
types_hash_max_size 2048;
open_file_cache max=200000 inactive=300s;
open_file_cache_valid 300s;
open_file_cache_min_uses 2;
open_file_cache_errors on;
server_tokens off;
dav_methods off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
access_log /var/log/nginx/access.log combined;
error_log /var/log/nginx/error.log warn;
gzip off;
gzip_vary off;
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*.conf;
}
```
Here is the Nginx vhost file used:
```
upstream whoami {
server IP-whoami1:80;
server IP-whoami2:80;
keepalive 300;
}
server {
listen 8001;
server_name test.traefik;
access_log off;
error_log /dev/null crit;
if ($host != "test.traefik") {
return 404;
}
location / {
proxy_pass http://whoami;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header X-Forwarded-Host $host;
}
}
```
### Traefik
Here is the `traefik.toml` file used:
```toml
maxIdleConnsPerHost = 100000
defaultEntryPoints = ["http"]
[entryPoints]
[entryPoints.http]
address = ":8000"
[file]
[backends]
[backends.backend1]
[backends.backend1.servers.server1]
url = "http://IP-whoami1:80"
weight = 1
[backends.backend1.servers.server2]
url = "http://IP-whoami2:80"
weight = 1
[frontends]
[frontends.frontend1]
backend = "backend1"
[frontends.frontend1.routes.test_1]
rule = "Host: test.traefik"
```
## Results
### whoami:
```shell
wrk -t20 -c1000 -d60s -H "Host: test.traefik" --latency http://IP-whoami:80/bench
Running 1m test @ http://IP-whoami:80/bench
20 threads and 1000 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 70.28ms 134.72ms 1.91s 89.94%
Req/Sec 2.92k 742.42 8.78k 68.80%
Latency Distribution
50% 10.63ms
75% 75.64ms
90% 205.65ms
99% 668.28ms
3476705 requests in 1.00m, 384.61MB read
Socket errors: connect 0, read 0, write 0, timeout 103
Requests/sec: 57894.35
Transfer/sec: 6.40MB
```
### nginx:
```shell
wrk -t20 -c1000 -d60s -H "Host: test.traefik" --latency http://IP-nginx:8001/bench
Running 1m test @ http://IP-nginx:8001/bench
20 threads and 1000 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 101.25ms 180.09ms 1.99s 89.34%
Req/Sec 1.69k 567.69 9.39k 72.62%
Latency Distribution
50% 15.46ms
75% 129.11ms
90% 302.44ms
99% 846.59ms
2018427 requests in 1.00m, 298.36MB read
Socket errors: connect 0, read 0, write 0, timeout 90
Requests/sec: 33591.67
Transfer/sec: 4.97MB
```
### Traefik:
```shell
wrk -t20 -c1000 -d60s -H "Host: test.traefik" --latency http://IP-traefik:8000/bench
Running 1m test @ http://IP-traefik:8000/bench
20 threads and 1000 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 91.72ms 150.43ms 2.00s 90.50%
Req/Sec 1.43k 266.37 2.97k 69.77%
Latency Distribution
50% 19.74ms
75% 121.98ms
90% 237.39ms
99% 687.49ms
1705073 requests in 1.00m, 188.63MB read
Socket errors: connect 0, read 0, write 0, timeout 7
Requests/sec: 28392.44
Transfer/sec: 3.14MB
```
## Conclusion
Traefik is obviously slower than Nginx, but not so much: Traefik can serve 28392 requests/sec and Nginx 33591 requests/sec which gives a ratio of 85%.
Not bad for young project :) !
Some areas of possible improvements:
- Use [GO_REUSEPORT](https://github.com/kavu/go_reuseport) listener
- Run a separate server instance per CPU core with `GOMAXPROCS=1` (it appears during benchmarks that there is a lot more context switches with Traefik than with nginx)