Caddy issues with some sites but not others

1. The problem I’m having:

I have setup caddy using xcaddy to also build in the cloudflare plugin to allow me to use ssl with local only addresses. I use cloudflare to direct *.lan.my-domain.com to the local ip of the caddy server.

This has worked but only in half of the sites I have tried. For instance test.lan.my-domain.com pointing to ip:85 of server correctly brings back page and with secure connection. However, the exact same setup but with homepage.lan.my-domain.com reverse proxy to same address brings back errors.

I also have problems with some sites, such as Truenas but I am not sure why those wont work? Do they have nginx on the server itself and causing conflicts??

On some of these I have also found that they seem to go through but a blank page is returned.

2. Error messages and/or full log output:

I have setup debug in the config but the below is the only logs outputted.

xen@caddy:~$ sudo journalctl -u caddy --no-pager | less +G
:["application/json"],"Date":["Mon, 19 Aug 2024 23:04:54 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Replay-Nonce":["Xt09NNgJ0NZ1rfEO0fSuWL6f8oe-5O9KTzSSa_DGGeVjF87crNU"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}
Aug 20 00:04:54 caddy caddy[518]: {"level":"info","ts":1724108694.7433498,"logger":"tls.issuance.acme.acme_client","msg":"authorization finalized","identifier":"homepage.lan.my-domain.com","authz_status":"valid"}
Aug 20 00:04:54 caddy caddy[518]: {"level":"info","ts":1724108694.743933,"logger":"tls.issuance.acme.acme_client","msg":"validations succeeded; finalizing order","order":"https://acme-v02.api.letsencrypt.org/acme/order/1899756326/297730241606"}
Aug 20 00:04:55 caddy caddy[518]: {"level":"debug","ts":1724108695.2626834,"logger":"tls.issuance.acme.acme_client","msg":"http request","method":"POST","url":"https://acme-v02.api.letsencrypt.org/acme/finalize/1899756326/297730241606","headers":{"Content-Type":["application/jose+json"],"User-Agent":["Caddy/2.8.4 CertMagic acmez (linux; amd64)"]},"response_headers":{"Boulder-Requester":["1899756326"],"Cache-Control":["public, max-age=0, no-cache"],"Content-Length":["450"],"Content-Type":["application/json"],"Date":["Mon, 19 Aug 2024 23:04:55 GMT"],"Link":["<https://acme-v02.api.letsencrypt.org/directory>;rel=\"index\""],"Location":["https://acme-v02.api.letsencrypt.org/acme/order/1899756326/297730241606"],"Replay-Nonce":["lpv3ejQgppAiElaVjoqDftw641HLG1y_Orp782jcbfnunjR4jZk"],"Server":["nginx"],"Strict-Transport-Security":["max-age=604800"],"X-Frame-Options":["DENY"]},"status_code":200}

Working:

url -vL test.lan.my-domain.com
* Host test.lan.my-domain.com:80 was resolved.
* IPv6: (none)
* IPv4: 192.168.10.15
*   Trying 192.168.10.15...
* Connected to test.lan.my-domain.com (192.168.10.15) port 80
> GET / HTTP/1.1
> Host: test.lan.my-domain.com
> User-Agent: curl/8.7.1
> Accept: */*
>
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://test.lan.my-domain.com/
< Server: Caddy
< Date: Mon, 19 Aug 2024 23:20:58 GMT
< Content-Length: 0
<
* Request completely sent off
* Closing connection
* Clear auth, redirects to port from 80 to 443
* Issue another request to this URL: 'https://test.lan.my-domain.com/'
* Host test.lan.my-domain.com:443 was resolved.
* IPv6: (none)
* IPv4: 192.168.10.15
*   Trying 192.168.10.15:443...
* Connected to test.lan.my-domain.com (192.168.10.15) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* ALPN: server accepted http/1.1
* using HTTP/1.x
> GET / HTTP/1.1
> Host: test.lan.my-domain.com
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 200 OK
< Alt-Svc: h3=":443"; ma=2592000
< Cache-Control: s-maxage=31536000, stale-while-revalidate
< Content-Type: text/html; charset=utf-8
< Date: Mon, 19 Aug 2024 23:20:58 GMT
< Etag: "3bzu09pv0ov84"
< Server: Caddy
< Vary: Accept-Encoding
< X-Nextjs-Cache: HIT
< X-Powered-By: Next.js
< Transfer-Encoding: chunked

Not working:

curl -vL home.lan.dacnet.me
* Host home.lan.my-domain.com:80 was resolved.
* IPv6: (none)
* IPv4: 192.168.10.15
*   Trying 192.168.10.15:80...
* Connected to home.lan.my-domain.com (192.168.10.15) port 80
> GET / HTTP/1.1
> Host: home.lan.my-domain.com
> User-Agent: curl/8.7.1
> Accept: */*
>
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://home.lan.my-domain.com/
< Server: Caddy
< Date: Mon, 19 Aug 2024 23:25:06 GMT
< Content-Length: 0
<
* Request completely sent off
* Closing connection
* Clear auth, redirects to port from 80 to 443
* Issue another request to this URL: 'https://home.lan.my-domain.com/'
* Host home.lan.my-domain.com:443 was resolved.
* IPv6: (none)
* IPv4: 192.168.10.15
*   Trying 192.168.10.15:443...
* Connected to home.lan.my-domain.com (192.168.10.15) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* schannel: next InitializeSecurityContext failed: SEC_E_ILLEGAL_MESSAGE (0x80090326) - This error usually occurs when a fatal SSL/TLS alert is received (e.g. handshake failed). More detail may be available in the Windows System event log.
* Closing connection
* schannel: shutting down SSL/TLS connection with home.lan.my-domain.com port 443
curl: (35) schannel: next InitializeSecurityContext failed: SEC_E_ILLEGAL_MESSAGE (0x80090326) - This error usually occurs when a fatal SSL/TLS alert is received (e.g. handshake failed). More detail may be available in the Windows System event log.
< Transfer-Encoding: chunked

3. Caddy version:

v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=

4. How I installed and ran Caddy:

a. System environment:

Caddy built with xcaddy following the below guides.

b. Command:

sudo systemctl caddy start
sudo nano /etc/caddy/Caddyfile
sudo systemctl reload caddy

c. Service/unit/compose file:

d. My complete Caddy config:

{
        debug
        acme_dns cloudflare CREDENTIAL
}

test.lan.my-domain.com {
        reverse_proxy 192.168.10.20:85
}

homepage.lan.my-domain.com {
        reverse_proxy 192.168.10.20:85
}

photo.lan.my-domain.com {
        reverse_proxy 192.168.10.20:2283
}

5. Links to relevant resources:

Well, you wrote homepage.lan in your Caddyfile, but you did home.lan in your curl. Did you forget you called it one way and not the other?

Yes that explains it, all working now! I guess thats what I get for trying to do this in the middle of the night.

Thank you