Hi @matt,
Thank you for this article. I have been struggling with setting up the AWS Load balancer with Caddy server for some days now. But thanks to your article, I now understand what I was doing wrong.
I would either like to set up “database for storage” or “forward to port 80”.
The thing is, I can’t seem to find documentation on how to setup with database.
On “forward to port 80” I did not quite understand. Are you saying I change the port 80 to like 580 like this: {http_port 580 }
To use a database for storage, configure the storage module of your choice: JSON Config Structure - Caddy Documentation
If you are forwarding port 80 to port 580, then yes, you will need to tell Caddy to use port 580 for HTTP.
Thank you.
From the docs this is what I have configured, could you please help:
{
storage dynamodb caddy{
table cad****
aws_endpoint arn:aws:dynamodb:****
aws_region us-east-1
}
}
k**.com:80, k**.com {
root * /var/www/k**
encode gzip
php_fastcgi unix//run/php/php7.4-fpm.sock
file_server
log {
output file /var/log/caddy/k**.log
format console
}
}
zo**.com:80, zo**.com {
root * /var/www/zo**
encode gzip
php_fastcgi unix//run/php/php7.4-fpm.sock
file_server
log {
output file /var/log/caddy/zo**.log
format console
}
}
Please use ``` on the lines before and after your config to use code formatting.
What do you need help with? What’s the problem? Please be specific.
I have an EC2 instance which has two production domains k**.com and Zo**.com and set up caddy web server. Run caddy config and worked perfectly, issued cert (https) and it was available for the domains.
Now I created an Application Load Balancer for it (AWS ELB) that has HTTP and HTTPS listener. On the security settings, I have to set up certificates for it so I created certificates for k**.com and Zo**.com which is implemented on Route 53 dns settings. At first the heath checks for port 443 and 80 did not work, the config was this:
{
email john@k**.com
}
(common) {
encode gzip
php_fastcgi unix//run/php/php7.4-fpm.sock
file_server
}
www.k***.com {
redir https://k***.com{uri}
}
k***.com {
root * /var/www/k***
import common
log {
output file /var/log/caddy/k**.log
format console
}
}
www.zo**.com {
redir https://zo**.com{uri}
}
zo**.com {
root * /var/www/zo**
import common
log {
output file /var/log/caddy/zo**.log
format console
}
}
Thus after checking caddy docs I adjusted the config to this:
(common) {
encode gzip
php_fastcgi unix//run/php/php7.4-fpm.sock
file_server
}
k***.com:80, k***.com {
root * /var/www/k***
import common
log {
output file /var/log/caddy/k**.log
format console
}
}
zo**.com:80, zo**.com {
root * /var/www/zo**
import common
log {
output file /var/log/caddy/zo**.log
format console
}
}
After the change the port 80 health check passed and could access the sites with http but port 443 did not, got "502 Bad Gateway " error when I access the sites with https.
So reading caddy docs again I saw I needed to set the configs on a shared device thus tried the dynamodb (after installing with xcaddy) then later to change to AWS EFS.
I have set up the EFS on the ubuntu instance and it works, now changed the config file to this:
{
storage file_system {
root /efs/caddy
}
}
k***.com:80, k***.com {
root * /var/www/k***
encode gzip
php_fastcgi unix//run/php/php7.4-fpm.sock
file_server
log {
output file /var/log/caddy/k**.log
format console
}
}
zo**.com:80, zo**.com {
root * /var/www/zo**
encode gzip
php_fastcgi unix//run/php/php7.4-fpm.sock
file_server
log {
output file /var/log/caddy/zo**.log
format console
}
}
The health check for 443 is still “unhealthy” and get 502 Bad Gateway when I go to the site. Port 80 is healthy and the sites are loading on http. Wondering what I am doing wrong? Please help.
This might be really basic, but are you ensuring your port 443 health check is HTTPS (rather than HTTP)?
Log output from Caddy when those failing health checks get issued might help.
● caddy.service - Caddy
Loaded: loaded (/lib/systemd/system/caddy.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2020-11-30 10:01:25 UTC; 4min 28s ago
Docs: https://caddyserver.com/docs/
Process: 28583 ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile (code=exited, status=0/SUCCESS)
Main PID: 28506 (caddy)
Tasks: 8 (limit: 1164)
Memory: 12.2M
CGroup: /system.slice/caddy.service
└─28506 /usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
Nov 30 10:05:08 ip-172-31-24-130 caddy[28506]: {"level":"info","ts":1606730708.8184826,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv1"}
Nov 30 10:05:08 ip-172-31-24-130 caddy[28506]: {"level":"warn","ts":1606730708.8186934,"logger":"http","msg":"user server is listening on same interface as automatic HTTP->HTTPS redirects; user-configured routes might override these redirects","server_name":"srv0","interface":"tcp/:80"}
Nov 30 10:05:08 ip-172-31-24-130 caddy[28506]: {"level":"info","ts":1606730708.8202882,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["b**.c**","k**.c**"]}
Nov 30 10:05:09 ip-172-31-24-130 caddy[28506]: {"level":"info","ts":1606730709.8511438,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc0001a4700"}
Nov 30 10:05:09 ip-172-31-24-130 caddy[28506]: {"level":"info","ts":1606730709.8520076,"msg":"autosaved config","file":"/var/lib/caddy/.config/caddy/autosave.json"}
Nov 30 10:05:09 ip-172-31-24-130 caddy[28506]: {"level":"info","ts":1606730709.852184,"logger":"admin.api","msg":"load complete"}
Nov 30 10:05:09 ip-172-31-24-130 systemd[1]: Reloaded Caddy.
Nov 30 10:05:10 ip-172-31-24-130 caddy[28506]: {"level":"info","ts":1606730710.3107305,"logger":"admin","msg":"stopped previous server"}
Nov 30 10:05:33 ip-172-31-24-130 caddy[28506]: {"level":"info","ts":1606730733.0954893,"logger":"http.log.access","msg":"handled request","request":{"remote_addr":"172.31.23.228:10384","proto":"HTTP/1.1","method":"GET","host":"172.31.24.130","uri":"/","headers":{"Connection":["close"],"User-Agent":["ELB-HealthChecker/2.0"],"Accept-Encoding":["gzip, compressed"]}},"common_log":"172.31.23.228 - - [30/Nov/2020:10:05:33 +0000] \"GET / HTTP/1.1\" 308 0","duration":0.000087235,"size":0,"status":308,"resp_headers":{"Connection":["close"],"Content-Type":[],"Server":["Caddy"],"Location":["https://172.31.24.130/"]}}
Nov 30 10:05:38 ip-172-31-24-130 caddy[28506]: {"level":"info","ts":1606730738.002086,"logger":"http.log.access","msg":"handled request","request":{"remote_addr":"172.31.35.110:49422","proto":"HTTP/1.1","method":"GET","host":"172.31.24.130","uri":"/","headers":{"Connection":["close"],"User-Agent":["ELB-HealthChecker/2.0"],"Accept-Encoding":["gzip, compressed"]}},"common_log":"172.31.35.110 - - [30/Nov/2020:10:05:38 +0000] \"GET / HTTP/1.1\" 308 0","duration":0.000085497,"size":0,"status":308,"resp_headers":{"Content-Type":[],"Server":["Caddy"],"Location":["https://172.31.24.130/"],"Connection":["close"]}}
This is from sudo systemctl status caddy
And this is from journalctl
Nov 30 10:09:03 ip-172-31-24-130 caddy[28506]: {"level":"info","ts":1606730943.2410867,"logger":"http.log.access","msg":"handled request","request":{"remote_addr":"172.31.23.228:10512","proto":"HTTP/1.1","method":"GET","host":"172.31.24.130","uri":"/","headers":{"Connection":["close"],"User-Agent":["ELB-HealthChecker/2.0"],"Accept-Encoding":["gzip, compressed"]}},"common_log":"172.31.23.228 - - [30/Nov/2020:10:09:03 +0000] \"GET / HTTP/1.1\" 308 0","duration":0.000060158,"size":0,"status":308,"resp_headers":{"Content-Type":[],"Server":["Caddy"],"Location":["https://172.31.24.130/"],"Connection":["close"]}}
Nov 30 10:09:08 ip-172-31-24-130 caddy[28506]: {"level":"info","ts":1606730948.149816,"logger":"http.log.access","msg":"handled request","request":{"remote_addr":"172.31.35.110:49546","proto":"HTTP/1.1","method":"GET","host":"172.31.24.130","uri":"/","headers":{"Connection":["close"],"User-Agent":["ELB-HealthChecker/2.0"],"Accept-Encoding":["gzip, compressed"]}},"common_log":"172.31.35.110 - - [30/Nov/2020:10:09:08 +0000] \"GET / HTTP/1.1\" 308 0","duration":0.000066261,"size":0,"status":308,"resp_headers":{"Server":["Caddy"],"Location":["https://172.31.24.130/"],"Connection":["close"],"Content-Type":[]}}
Nov 30 10:09:33 ip-172-31-24-130 caddy[28506]: {"level":"info","ts":1606730973.2578404,"logger":"http.log.access","msg":"handled request","request":{"remote_addr":"172.31.23.228:10534","proto":"HTTP/1.1","method":"GET","host":"172.31.24.130","uri":"/","headers":{"Accept-Encoding":["gzip, compressed"],"Connection":["close"],"User-Agent":["ELB-HealthChecker/2.0"]}},"common_log":"172.31.23.228 - - [30/Nov/2020:10:09:33 +0000] \"GET / HTTP/1.1\" 308 0","duration":0.000081305,"size":0,"status":308,"resp_headers":{"Location":["https://172.31.24.130/"],"Connection":["close"],"Content-Type":[],"Server":["Caddy"]}}
Nov 30 10:09:38 ip-172-31-24-130 caddy[28506]: {"level":"info","ts":1606730978.163548,"logger":"http.log.access","msg":"handled request","request":{"remote_addr":"172.31.35.110:49586","proto":"HTTP/1.1","method":"GET","host":"172.31.24.130","uri":"/","headers":{"Accept-Encoding":["gzip, compressed"],"Connection":["close"],"User-Agent":["ELB-HealthChecker/2.0"]}},"common_log":"172.31.35.110 - - [30/Nov/2020:10:09:38 +0000] \"GET / HTTP/1.1\" 308 0","duration":0.000094273,"size":0,"status":308,"resp_headers":{"Server":["Caddy"],"Location":["https://172.31.24.130/"],"Connection":["close"],"Content-Type":[]}}
And another log:
{"level":"info","ts":1606730782.2287745,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"172.31.23.228:10416","proto":"HTTP/1.1","method":"HEAD","host":"kaya.work","uri":"/","headers":{"X-Amzn-Trace-Id":["Root=1-5fc4c41e-1565fa112e1b5196106eb8ac"],"Content-Length":["0"],"User-Agent":["Mozilla/5.0+(compatible; UptimeRobot/2.0; http://www.uptimerobot.com/)"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"],"X-Forwarded-For":["63.143.42.252"],"X-Forwarded-Proto":["http"],"X-Forwarded-Port":["80"],"Accept-Language":["en-US,en;q=0.8"],"Accept-Charset":["ISO-8859-1,UTF-8;q=0.7,*;q=0.7"],"Accept-Encoding":["gzip, deflate"],"Cache-Control":["no-cache"],"Referer":["http://kaya.work"]}},"common_log":"172.31.23.228 - - [30/Nov/2020:10:06:22 +0000] \"HEAD / HTTP/1.1\" 200 0","duration":0.02907687,"size":0,"status":200,"resp_headers":{"Server":["Caddy"],"Content-Type":["text/html; charset=UTF-8"]}}
{"level":"info","ts":1606730894.0447547,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"172.31.23.228:10482","proto":"HTTP/1.1","method":"GET","host":"kaya.work","uri":"/","headers":{"Accept-Encoding":["gzip,deflate"],"User-Agent":["Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2226.0 Safari/537.36"],"Cookie":["beget=begetok"],"X-Forwarded-For":["5.188.62.214"],"X-Forwarded-Proto":["http"],"X-Forwarded-Port":["80"],"X-Amzn-Trace-Id":["Root=1-5fc4c48e-2635508c433811f16eb4c8ba"]}},"common_log":"172.31.23.228 - - [30/Nov/2020:10:08:14 +0000] \"GET / HTTP/1.1\" 200 22","duration":0.02444112,"size":22,"status":200,"resp_headers":{"Server":["Caddy"],"Content-Type":["text/html; charset=UTF-8"]}}
{"level":"info","ts":1606730898.6103437,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"172.31.23.228:10482","proto":"HTTP/1.1","method":"GET","host":"kaya.work","uri":"/robots.txt","headers":{"User-Agent":["Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2226.0 Safari/537.36"],"Cookie":["beget=begetok"],"X-Forwarded-For":["5.188.62.214"],"X-Forwarded-Proto":["http"],"X-Forwarded-Port":["80"],"X-Amzn-Trace-Id":["Root=1-5fc4c492-58962007666819400b953784"],"Accept-Encoding":["gzip,deflate"]}},"common_log":"172.31.23.228 - - [30/Nov/2020:10:08:18 +0000] \"GET /robots.txt HTTP/1.1\" 200 22","duration":0.030732852,"size":22,"status":200,"resp_headers":{"Server":["Caddy"],"Content-Type":["text/html; charset=UTF-8"]}}
{"level":"info","ts":1606731082.2401185,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"172.31.23.228:10590","proto":"HTTP/1.1","method":"HEAD","host":"kaya.work","uri":"/","headers":{"Accept-Language":["en-US,en;q=0.8"],"X-Forwarded-For":["63.143.42.252"],"X-Forwarded-Proto":["http"],"User-Agent":["Mozilla/5.0+(compatible; UptimeRobot/2.0; http://www.uptimerobot.com/)"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"],"Accept-Charset":["ISO-8859-1,UTF-8;q=0.7,*;q=0.7"],"Cache-Control":["no-cache"],"Referer":["http://kaya.work"],"Accept-Encoding":["gzip, deflate"],"X-Forwarded-Port":["80"],"X-Amzn-Trace-Id":["Root=1-5fc4c54a-48f4101c1fe55d4d0bcfde61"],"Content-Length":["0"]}},"common_log":"172.31.23.228 - - [30/Nov/2020:10:11:22 +0000] \"HEAD / HTTP/1.1\" 200 0","duration":0.035288389,"size":0,"status":200,"resp_headers":{"Server":["Caddy"],"Content-Type":["text/html; charset=UTF-8"]}}
And yes, port 443 health check is HTTPS, Protocol version HTTP/1.1. I also tried protocol HTTP/2 but didn’t pass.
Ok. I am almost giving up.
@matt I have really tried to understand this “Forward port 80 (HTTP).
This allows Caddy to obtain new certificates using the ACME HTTP challenge.” and this “Forward port 443 (TLS / HTTPS) – do not terminate TLS at the load balancer.” but I seem not to.
On the load balancer, the health check for 443 fails “unhealthy”. So what I have tried a couple of things. Do note I am setting up the load balancer, the domains DNS is still pointing to the instance IP address thus domain is not being served by load balancer:
As you can see I changed the https port to 5000. On the load balancer I put the listener of port HTTPS 443 to forward to port 5000 to target. Port443 listener to port 443 on target. See pic below.
Health check has failed and the site cannot be accessed using https but can be accessed using http.
So I change the caddy config file to remove the https 5000:
The health check about with port 5000 results to “unhealthy” but the site using https can now be accessed. Do note the DNS is still pointing to the server directly using IP and not pointing to the load balancer. Thus load balancer is not serving the domains.
THE only time the health check passes for port 443 is when I forward Listener 443 to port 80. And when I do that, styling does not load:
Though http:// does load properly.
Please, any help?
@Moe after reading your solution with network load balancer, I was wondering, did you manage to set up with Application load balancer?
This topic was automatically closed after 30 days. New replies are no longer allowed.
