traefik/docs/benchmarks.md

# Benchmarks

## Configuration

I would like to thanks [vincentbernat](https://github.com/vincentbernat) from [exoscale.ch](https://www.exoscale.ch) who kindly provided the infrastructure needed for the benchmarks.

I used 4 VMs for the tests with the following configuration:

- 32 GB RAM
- 8 CPU Cores
- 10 GB SSD
- Ubuntu 14.04 LTS 64-bit

## Setup

1. One VM used to launch the benchmarking tool [wrk](https://github.com/wg/wrk)
2. One VM for Traefik (v1.0.0-beta.416) / nginx (v1.4.6)
3. Two VMs for 2 backend servers in go [whoami](https://github.com/containous/whoami/)

Each VM has been tuned using the following limits:

```bash
sysctl -w fs.file-max="9999999"
sysctl -w fs.nr_open="9999999"
sysctl -w net.core.netdev_max_backlog="4096"
sysctl -w net.core.rmem_max="16777216"
sysctl -w net.core.somaxconn="65535"
sysctl -w net.core.wmem_max="16777216"
sysctl -w net.ipv4.ip_local_port_range="1025       65535"
sysctl -w net.ipv4.tcp_fin_timeout="30"
sysctl -w net.ipv4.tcp_keepalive_time="30"
sysctl -w net.ipv4.tcp_max_syn_backlog="20480"
sysctl -w net.ipv4.tcp_max_tw_buckets="400000"
sysctl -w net.ipv4.tcp_no_metrics_save="1"
sysctl -w net.ipv4.tcp_syn_retries="2"
sysctl -w net.ipv4.tcp_synack_retries="2"
sysctl -w net.ipv4.tcp_tw_recycle="1"
sysctl -w net.ipv4.tcp_tw_reuse="1"
sysctl -w vm.min_free_kbytes="65536"
sysctl -w vm.overcommit_memory="1"
ulimit -n 9999999
```

### Nginx

Here is the config Nginx file use `/etc/nginx/nginx.conf`:

```
user www-data;
worker_processes auto;
worker_rlimit_nofile 200000;
pid /var/run/nginx.pid;

events {
    worker_connections 10000;
    use epoll;
    multi_accept on;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 300;
    keepalive_requests 10000;
    types_hash_max_size 2048;

    open_file_cache max=200000 inactive=300s;
    open_file_cache_valid 300s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;

    server_tokens off;
    dav_methods off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    access_log /var/log/nginx/access.log combined;
    error_log /var/log/nginx/error.log warn;

    gzip off;
    gzip_vary off;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*.conf;
}
```

Here is the Nginx vhost file used:

```
upstream whoami {
    server IP-whoami1:80;
    server IP-whoami2:80;
    keepalive 300;
}

server {
    listen 8001;
    server_name test.traefik;
    access_log off;
    error_log /dev/null crit;
    if ($host != "test.traefik") {
        return 404;
    }
    location / {
        proxy_pass http://whoami;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
	proxy_set_header  X-Forwarded-Host $host;
    }
}
```

### Traefik

Here is the `traefik.toml` file used:

```toml
maxIdleConnsPerHost = 100000
defaultEntryPoints = ["http"]

[entryPoints]
  [entryPoints.http]
  address = ":8000"

[file]
[backends]
  [backends.backend1]
    [backends.backend1.servers.server1]
    url = "http://IP-whoami1:80"
    weight = 1
    [backends.backend1.servers.server2]
    url = "http://IP-whoami2:80"
    weight = 1

[frontends]
  [frontends.frontend1]
  backend = "backend1"
    [frontends.frontend1.routes.test_1]
    rule = "Host: test.traefik"
```

## Results

### whoami:
```shell
wrk -t20 -c1000 -d60s -H "Host: test.traefik" --latency  http://IP-whoami:80/bench
Running 1m test @ http://IP-whoami:80/bench
  20 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    70.28ms  134.72ms   1.91s    89.94%
    Req/Sec     2.92k   742.42     8.78k    68.80%
  Latency Distribution
     50%   10.63ms
     75%   75.64ms
     90%  205.65ms
     99%  668.28ms
  3476705 requests in 1.00m, 384.61MB read
  Socket errors: connect 0, read 0, write 0, timeout 103
Requests/sec:  57894.35
Transfer/sec:      6.40MB
```

### nginx:
```shell
wrk -t20 -c1000 -d60s -H "Host: test.traefik" --latency  http://IP-nginx:8001/bench
Running 1m test @ http://IP-nginx:8001/bench
  20 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   101.25ms  180.09ms   1.99s    89.34%
    Req/Sec     1.69k   567.69     9.39k    72.62%
  Latency Distribution
     50%   15.46ms
     75%  129.11ms
     90%  302.44ms
     99%  846.59ms
  2018427 requests in 1.00m, 298.36MB read
  Socket errors: connect 0, read 0, write 0, timeout 90
Requests/sec:  33591.67
Transfer/sec:      4.97MB
```

### Traefik:

```shell
wrk -t20 -c1000 -d60s -H "Host: test.traefik" --latency  http://IP-traefik:8000/bench
Running 1m test @ http://IP-traefik:8000/bench
  20 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    91.72ms  150.43ms   2.00s    90.50%
    Req/Sec     1.43k   266.37     2.97k    69.77%
  Latency Distribution
     50%   19.74ms
     75%  121.98ms
     90%  237.39ms
     99%  687.49ms
  1705073 requests in 1.00m, 188.63MB read
  Socket errors: connect 0, read 0, write 0, timeout 7
Requests/sec:  28392.44
Transfer/sec:      3.14MB
```

## Conclusion

Traefik is obviously slower than Nginx, but not so much: Traefik can serve 28392 requests/sec and Nginx 33591 requests/sec which gives a ratio of 85%.
Not bad for young project :) !

Some areas of possible improvements:

- Use [GO_REUSEPORT](https://github.com/kavu/go_reuseport) listener
- Run a separate server instance per CPU core with `GOMAXPROCS=1` (it appears during benchmarks that there is a lot more context switches with Traefik than with nginx)