Reverse proxy works and then fails

[tknx (TN)]

1. Output of caddy version:

/srv # caddy version
v2.5.2 h1:eCJdLyEyAGzuQTa5Mh3gETnYWDClo1LjtQm2q9RNZrs=

2. How I run Caddy:

1. Unraid docker - IP address set to local 10.0.0.102
2. Unifi router port forwards for 80 and 443 to same

a. System environment:

Unraid 6.10.3

b. Command:

root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='caddy' --net='br0' --ip='10.0.0.102' -e TZ="America/Los_Angeles" -e HOST_OS="Unraid" -e HOST_HOSTNAME="mnemosyne" -e HOST_CONTAINERNAME="caddy" -e 'TCP_PORT_80'='80' -e 'TCP_PORT_443'='443' -l net.unraid.docker.managed=dockerman -l net.unraid.docker.icon='https://d1q6f0aelx0por.cloudfront.net/product-logos/library-caddy-logo.png?' -v '/mnt/user/appdata/caddy/data':'/data':'rw' -v '/mnt/user/appdata/caddy/config':'/config':'rw' -v '/mnt/user/appdata/caddy/Caddyfile':'/etc/caddy/Caddyfile':'rw' 'caddy'

d. My complete Caddy config:

cweb.nagpal.house {
        reverse_proxy 10.0.0.152:8083
}

calibre.nagpal.house {
        reverse_proxy 10.0.0.151:8180
}

jellyfin.nagpal.house {
        reverse_proxy 10.0.0.141:8096
}

3. The problem I’m having:

When I start the docker the reverse proxy initially works but then stops working.

4. Error messages and/or full log output:

{"level":"info","ts":1659399599.8335507,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"info","ts":1659399599.8349602,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]}
{"level":"info","ts":1659399599.8350244,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1659399599.8350356,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1659399599.8351111,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00032b810"}
{"level":"info","ts":1659399599.8352401,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["jellyfin.nagpal.house","calibre.nagpal.house","cweb.nagpal.house","bitwarden.nagpal.house"]}
{"level":"info","ts":1659399599.835304,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
{"level":"info","ts":1659399599.8394647,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1659399599.8394806,"msg":"serving initial configuration"}
{"level":"info","ts":1659399599.8427417,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1659399665.3865333,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
{"level":"warn","ts":1659399665.386552,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
{"level":"info","ts":1659399665.3887699,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc00032b810"}
{"level":"info","ts":1659399665.3899367,"logger":"admin","msg":"stopped previous server","address":"tcp/localhost:2019"}
{"level":"info","ts":1659399665.389951,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
{"level":"info","ts":1659399665.63673,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"info","ts":1659399665.6377904,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1659399665.6378646,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1659399665.6378748,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1659399665.6379573,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000288c40"}
{"level":"debug","ts":1659399665.6395097,"logger":"http","msg":"starting server loop","address":"[::]:443","http3":false,"tls":true}
{"level":"debug","ts":1659399665.6395314,"logger":"http","msg":"starting server loop","address":"[::]:80","http3":false,"tls":false}
{"level":"info","ts":1659399665.6395345,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["bitwarden.nagpal.house","jellyfin.nagpal.house","calibre.nagpal.house","cweb.nagpal.house"]}
{"level":"info","ts":1659399665.639533,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
{"level":"debug","ts":1659399665.6404212,"logger":"tls","msg":"loading managed certificate","domain":"bitwarden.nagpal.house","expiration":1666895385,"issuer_key":"acme-v02.api.letsencrypt.org-directory","storage":"FileStorage:/data/caddy"}
{"level":"debug","ts":1659399665.6407056,"logger":"tls.cache","msg":"added certificate to cache","subjects":["bitwarden.nagpal.house"],"expiration":1666895385,"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"8d5d64c2ffa72e21a7af7266b4ef57d9bf2851595f4d2df6d9d09204b01515a6","cache_size":1,"cache_capacity":10000}
{"level":"debug","ts":1659399665.641461,"logger":"tls","msg":"loading managed certificate","domain":"jellyfin.nagpal.house","expiration":1666895385,"issuer_key":"acme-v02.api.letsencrypt.org-directory","storage":"FileStorage:/data/caddy"}
{"level":"debug","ts":1659399665.6417131,"logger":"tls.cache","msg":"added certificate to cache","subjects":["jellyfin.nagpal.house"],"expiration":1666895385,"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"f3bbde3e1f07ccd5b378309e9d2bdb38881387e4e031360a144348ad4817a1a5","cache_size":2,"cache_capacity":10000}
{"level":"debug","ts":1659399665.6424277,"logger":"tls","msg":"loading managed certificate","domain":"calibre.nagpal.house","expiration":1666895376,"issuer_key":"acme-v02.api.letsencrypt.org-directory","storage":"FileStorage:/data/caddy"}
{"level":"debug","ts":1659399665.6426458,"logger":"tls.cache","msg":"added certificate to cache","subjects":["calibre.nagpal.house"],"expiration":1666895376,"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"806bc99450c948fab393bf6367014f1e31a82274ee556351a814c52cdf6a6e6a","cache_size":3,"cache_capacity":10000}
{"level":"debug","ts":1659399665.6433017,"logger":"tls","msg":"loading managed certificate","domain":"cweb.nagpal.house","expiration":1666895377,"issuer_key":"acme-v02.api.letsencrypt.org-directory","storage":"FileStorage:/data/caddy"}
{"level":"debug","ts":1659399665.6435282,"logger":"tls.cache","msg":"added certificate to cache","subjects":["cweb.nagpal.house"],"expiration":1666895377,"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"3339e31a9760b122ee33d7417b4a883fa86577c7119139ea0a3b59a03fe90b6d","cache_size":4,"cache_capacity":10000}
{"level":"info","ts":1659399665.643714,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1659399665.643719,"msg":"serving initial configuration"}
{"level":"info","ts":1659399665.6467283,"logger":"tls","msg":"finished cleaning storage units"}

5. What I already tried:

Deleted all config files and restarted, checked ddns and port forwarding

6. Links to relevant resources:

[tknx (TN)]

OK, so let’s say I configure the docker and start it up. I can go to jellyfin.nagpal.house and it will work.

Then at some point, I leave my computer and come back, now it doesn’t work and gives me a Problem Loading Page on Firefox.

> curl -V jellyfin.nagpal.house

curl 7.83.1 (Windows) libcurl/7.83.1 Schannel
Release-Date: 2022-05-13
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp
Features: AsynchDNS HSTS IPv6 Kerberos Largefile NTLM SPNEGO SSL SSPI UnixSockets
PS C:\Users\tarun\OneDrive\Projects\ttk.hugo>

[francislavoie]

What do you mean by “stops working”? Please be specific. What behaviour are you seeing? What do you see when you make a request with curl -v?

I don’t see any errors in your logs, it doesn’t seem like a problem with Caddy to me.

[tknx (TN)]

@francislavoie see above, for some reason my comment appeared above yours

[francislavoie]

Use a lowercase -v not uppercase -V. They’re not the same thing. Uppercase is “version”, lowercase is “verbose”.

[tknx (TN)]

curl -v jellyfin.nagpal.house
*   Trying 142.254.87.196:80...
* connect to 142.254.87.196 port 80 failed: Connection refused
* Failed to connect to jellyfin.nagpal.house port 80 after 2150 ms: Connection refused
* Closing connection 0
curl: (7) Failed to connect to jellyfin.nagpal.house port 80 after 2150 ms: Connection refused

[francislavoie]

Looks like you have a networking issue, because it fails to even reach Caddy. So it’s not a problem with Caddy.

Are you sure that’s the right IP address? Does the domain resolve to the right thing? Do you have ports forwarded/opened/unblocked on your firewall?

[tknx (TN)]

OK - my caddy and a stupid other device had somehow gotten the same IP address - so once I switched it off it works. That is also why it would work for a moment and then fail as they fought over the address!