1. Caddy version (caddy version):
2.4.6
2. How I run Caddy:
I run Caddy in my docker-compose.yml using a caddybuilder, so that I can use the caddy-dns/route53 and caddy-tlsredis plugins.
Here’s my docker-compose.yml caddy section:
caddy:
build:
context: ./caddy
dockerfile: ./Dockerfile
restart: unless-stopped
command: caddy run --config /dockerapp/caddy/Caddyfile
ports:
- 80:80
- 443:443
volumes_from:
- web
volumes:
- ./caddy/Caddyfile:/etc/caddy/Caddyfile
- caddy_data:/data
- caddy_config:/config
networks:
- web-net
environment:
- AWS_ACCESS_KEY_ID=${AWS_SECRET_ACCESS_KEY}
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
and here’s the Dockerfile used to build Caddy in docker-compose.yml:
FROM caddy:builder AS builder
RUN xcaddy build \
--with github.com/caddy-dns/route53 \
--with github.com/gamalan/caddy-tlsredis@cb6e2c46bea8a6cafdcba76770be2a6db1200396
FROM caddy:latest
COPY --from=builder /usr/bin/caddy /usr/bin/caddy
a. System environment:
docker-compose running on AWS Elastic Beanstalk.
b. Command:
caddy run --config /dockerapp/caddy/Caddyfile
c. Service/unit/compose file:
docker-compose.yml caddy section posted above.
d. My complete Caddyfile or JSON config:
{
on_demand_tls {
ask 0.0.0.0:3000/caddy-domain-check
interval 2m
burst 5
}
debug
}
(SecurityHeaders) {
header_up X-Real-IP {remote_host}
header_up X-Forwarded-Proto {scheme}
}
my-example.com, *.my-example.com {
reverse_proxy 0.0.0.0:3000 {
header_up X-Forwarded-For {remote_host}
header_up X-Forwarded-Port {server_port}
header_up X-Forwarded-Proto {scheme}
}
file_server
log {
format single_field common_log
output file /data/logs/caddy.log
}
tls me@my-example.com{
dns route53
}
}
:443, :80 {
tls {
on_demand
}
tls internal {
on_demand
}
}
3. The problem I’m having:
curl -i https://my-example.com responds:
HTTP/1.1 400 Bad Request
transfer-encoding: chunked
Connection: keep-alive
Client sent an HTTP request to an HTTPS server.
Because my migration involves a switch to Caddy, docker-compose, and a new AWS Elastic Beanstalk environment, I’m having trouble tracking down what is to blame for the above issue. A couple things I’ve noticed:
-
If I remove the listener on 443 from my Elastic Beanstalk load balancer, the https request times out instead of returning “Client sent an HTTP request to an HTTPS server.”
-
The ssl certificate on this page is valid, but it’s issued by Amazon (via the certificate attached to my load balancer) rather than Caddy.
4. Error messages and/or full log output:
Caddy logs from startup until it finishes loading. Every line logged after this is a variation of the last line’s TLS handshake error
caddy_1 | {"level":"info","ts":1639782873.55904,"msg":"using provided configuration","config_file":"/dockerapp/caddy/Caddyfile","config_adapter":""}
caddy_1 | {"level":"warn","ts":1639782873.560051,"msg":"input is not formatted with 'caddy fmt'","adapter":"caddyfile","file":"/dockerapp/caddy/Caddyfile","line":11}
caddy_1 | {"level":"info","ts":1639782873.5652137,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["localhost:2019","[::1]:2019","127.0.0.1:2019"]}
caddy_1 | {"level":"info","ts":1639782873.5655,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000465f10"}
caddy_1 | {"level":"info","ts":1639782873.5658686,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
caddy_1 | {"level":"info","ts":1639782873.5658894,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
caddy_1 | {"level":"info","ts":1639782873.565914,"logger":"http","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv1","http_port":80}
caddy_1 | {"level":"warn","ts":1639782873.627143,"logger":"pki.ca.local","msg":"installing root certificate (you might be prompted for password)","path":"storage:pki/authorities/local/root.crt"}
caddy_1 | 2021/12/17 23:14:33 define JAVA_HOME environment variable to use the Java trust
caddy_1 | 2021/12/17 23:14:33 Warning: "certutil" is not available, install "certutil" with "apt install libnss3-tools" or "yum install nss-tools" and try again
caddy_1 | 2021/12/17 23:14:33 certificate installed properly in linux trusts
caddy_1 | {"level":"info","ts":1639782873.7765002,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
caddy_1 | {"level":"info","ts":1639782873.777416,"logger":"tls","msg":"finished cleaning storage units"}
caddy_1 | {"level":"debug","ts":1639782873.7806373,"logger":"http","msg":"starting server loop","address":"[::]:443","http3":false,"tls":true}
caddy_1 | {"level":"debug","ts":1639782873.7807045,"logger":"http","msg":"starting server loop","address":"[::]:80","http3":false,"tls":false}
caddy_1 | {"level":"info","ts":1639782873.780712,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["my-example.com","*.my-example.com"]}
caddy_1 | {"level":"debug","ts":1639782873.7811437,"logger":"tls","msg":"loading managed certificate","domain":"my-example.com","expiration":1647561599,"issuer_key":"acme.zerossl.com-v2-DV90","storage":"FileStorage:/data/caddy"}
caddy_1 | {"level":"debug","ts":1639782873.7816758,"logger":"tls.cache","msg":"added certificate to cache","subjects":["my-example.com"],"expiration":1647561599,"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"16f3f4feea021d4cb16fe91d99c8b34fa65b6c5bb59b206838360b4335523d9b","cache_size":1,"cache_capacity":10000}
caddy_1 | {"level":"debug","ts":1639782873.7819345,"logger":"tls","msg":"loading managed certificate","domain":"*.my-example.com","expiration":1647542310,"issuer_key":"acme-v02.api.letsencrypt.org-directory","storage":"FileStorage:/data/caddy"}
caddy_1 | {"level":"debug","ts":1639782873.7821574,"logger":"tls.cache","msg":"added certificate to cache","subjects":["*.my-example.com"],"expiration":1647542310,"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"54ef9bff38b059e94bbfdd3a2805f6c38fbcb015ce96ddc7fd3940f3b691b4ae","cache_size":2,"cache_capacity":10000}
caddy_1 | {"level":"info","ts":1639782873.782314,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
caddy_1 | {"level":"info","ts":1639782873.7823212,"msg":"serving initial configuration"}
caddy_1 | {"level":"debug","ts":1639782933.7712696,"logger":"http.stdlib","msg":"http: TLS handshake error from 172.31.17.51:21498: EOF"}
5. What I already tried:
In order to enable simple custom domains w/ SSL on my Rails SaaS app, I’m migrating to Caddy from nginx. I’m also having to migrate to a new docker-compose production environment on AWS Elastic Beanstalk at the same time.
I’ve tried countless iterations of the above Caddyfile and docker-compose.yml to arrive at what I have now. I suspect the problem has something to do with the docker-compose + Caddy configuration, but I feel like I’m at a dead end in terms of trying to understand this issue better by googling.