First attempt at using newer Caddy v2 server so thought I’d do some quick HTTP/2 & HTTP/3 benchmarks against my Nginx HTTP/2 & HTTP/3 Cloudflare Quiche patched servers to see where performance is at. The full write up and system/config details are at GitHub - centminmod/centminmod-caddy-v2
Test Parameters
Using h2load tester
- h2load HTTP/2 HTTPS load tests at 150, 500 and 1,000 user concurrency at different number of requests and max concurrent stream parameters
- h2load HTTP/3 HTTPS load tests at 150, 500 and 1,000 user concurrency at different number of requests and max concurrent stream parameters
Caddy v2 keeled over at 1000 user concurrency mark for both h2load HTTP/2 and HTTP/3 load tests while Nginx handled them fine both on the same Virtualbox CentOS 7.8 guest OS environment.
Just the tabulated results are below:
HTTP/2 HTTPS Benchmarks
server | h2load HTTP/2 | requests/s | ttfb min | ttfb avg | ttfb max | cipher | protocol | successful req | failed req |
---|---|---|---|---|---|---|---|---|---|
caddy v2 | t1 c150 n1000 m50 | 959.57 | 213.30ms | 696.74ms | 1.03s | ECDHE-ECDSA-AES256-GCM-SHA384 | h2 TLSv1.2 | 100% | 0% |
caddy v2 | t1 c500 n2000 m100 | 990.03 | 711.60ms | 1.36s | 1.98s | ECDHE-ECDSA-AES256-GCM-SHA384 | h2 TLSv1.2 | 100% | 0% |
caddy v2 | t1 c1000 n10000 m100 | 1049.00 | 965.65ms | 3.34s | 6.53s | ECDHE-ECDSA-AES256-GCM-SHA384 | h2 TLSv1.2 | 68.89% | 31.11% |
nginx 1.17.10 | t1 c150 n1000 m50 | 2224.74 | 158.04ms | 300.22ms | 440.22ms | ECDHE-ECDSA-AES128-GCM-SHA256 | h2 TLSv1.2 | 100% | 0% |
nginx 1.17.10 | t1 c500 n2000 m100 | 1600.52 | 583.80ms | 861.70ms | 1.23s | ECDHE-ECDSA-AES128-GCM-SHA256 | h2 TLSv1.2 | 100% | 0% |
nginx 1.17.10 | t1 c1000 n10000 m100 | 1912.05 | 949.61ms | 2.98s | 5.16s | ECDHE-ECDSA-AES128-GCM-SHA256 | h2 TLSv1.2 | 100% | 0% |
HTTP/3 HTTPS Benchmarks
server | h2load HTTP/3 | requests/s | ttfb min | ttfb avg | ttfb max | cipher | protocol | successful req | failed req |
---|---|---|---|---|---|---|---|---|---|
caddy v2 | t1 c150 n1000 m50 | 594.60 | 230.02ms | 590.87ms | 1.65s | TLS_AES_128-GCM_SHA256 | h3-27 TLSv1.3 | 100% | 0% |
caddy v2 | t1 c500 n2000 m100 | 333.49 | 353.84ms | 1.89s | 5.98s | TLS_AES_128-GCM_SHA256 | h3-27 TLSv1.3 | 100% | 0% |
caddy v2 | t1 c1000 n10000 m100 | failed | failed | failed | failed | - | - | 0% | 100% |
nginx 1.16.1 patch | t1 c150 n1000 m50 | 1856.67 | 213.84ms | 333.80ms | 510.07ms | TLS_AES_128-GCM_SHA256 | h3-27 TLSv1.3 | 100% | 0% |
nginx 1.16.1 patch | t1 c500 n2000 m100 | 842.13 | 325.78ms | 834.50ms | 1.59s | TLS_AES_128-GCM_SHA256 | h3-27 TLSv1.3 | 100% | 0% |
nginx 1.16.1 patch | t1 c1000 n10000 m100 | 847.89 | 657.03ms | 3.58s | 9.09s | TLS_AES_128-GCM_SHA256 | h3-27 TLSv1.3 | 100% | 0% |
Test ngx.domain.com
site test over curl with HTTP/3 support built using Cloudflare Quiche library.
curl-http3 --http3 -skD - -H "Accept-Encoding: gzip" https://ngx.domain.com/caddy-index.html -o /dev/null
HTTP/3 200
date: Sat, 09 May 2020 15:01:22 GMT
content-type: text/html; charset=utf-8
last-modified: Wed, 06 May 2020 18:44:09 GMT
vary: Accept-Encoding
etag: W/"5eb30579-2fc2"
server: nginx centminmod
x-powered-by: centminmod
alt-svc: h3-27=":443"; ma=86400
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
content-encoding: gzip
Caddy v2 HTTP/3 with experimental_http3
enabled
curl-http3 --http3 -skD - -H "Accept-Encoding: gzip" https://caddy.domain.com:4444/caddy-index.html -o /dev/null
HTTP/3 200
x-xss-protection: 1; mode=block
etag: "q9xapl9fm"
content-type: text/html; charset=utf-8
last-modified: Wed, 06 May 2020 18:44:09 GMT
content-encoding: gzip
x-powered-by: caddy centminmod
alt-svc: h3-27=":4444"; ma=2592000
x-content-type-options: nosniff
vary: Accept-Encoding
server: Caddy
For h2load HTTP/3 tests
h2load-http3 --version
h2load nghttp2/1.41.0-DEV
For curl
curl-http3 -V
curl 7.71.0-DEV (x86_64-pc-linux-gnu) libcurl/7.71.0-DEV BoringSSL zlib/1.2.11 brotli/1.0.7 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5) libssh2/1.8.0 nghttp2/1.36.0 quiche/0.3.0
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Largefile libz NTLM NTLM_WB PSL SSL UnixSockets