Caddy not responding to external requests

Ello,

1. The problem I’m having:

I use Caddy as a reverse proxy, but it isn’t responding to any of my requests anymore. It connects, but times out on http, but won’t even connect to https. This is a recent issue, and I’m not quite sure what is happening

C:\Users\lawso>curl -vL thatrandompi.xyz
* Host thatrandompi.xyz:80 was resolved.
* IPv6: (none)
* IPv4: 144.6.98.26
*   Trying 144.6.98.26:80...
* Connected to thatrandompi.xyz (144.6.98.26) port 80
> GET / HTTP/1.1
> Host: thatrandompi.xyz
> User-Agent: curl/8.9.1
> Accept: */*
>
* Request completely sent off
* Recv failure: Connection was reset
* closing connection #0
curl: (56) Recv failure: Connection was reset

C:\Users\lawso>curl -vL https://thatrandompi.xyz
* Host thatrandompi.xyz:443 was resolved.
* IPv6: (none)
* IPv4: 144.6.98.26
*   Trying 144.6.98.26:443...
* connect to 144.6.98.26 port 443 from 0.0.0.0 port 18210 failed: Timed out
* Failed to connect to thatrandompi.xyz port 443 after 21078 ms: Could not connect to server
* closing connection #0
curl: (28) Failed to connect to thatrandompi.xyz port 443 after 21078 ms: Could not connect to server

2. Error messages and/or full log output:

caddy  | {"level":"info","ts":1729639571.4709463,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
caddy  | {"level":"warn","ts":1729639571.4714034,"logger":"caddyfile","msg":"Unnecessary header_up X-Forwarded-For: the reverse proxy's default behavior is to pass headers to the upstream"}
caddy  | {"level":"warn","ts":1729639571.471412,"logger":"caddyfile","msg":"Unnecessary header_up X-Forwarded-Proto: the reverse proxy's default behavior is to pass headers to the upstream"}
caddy  | {"level":"info","ts":1729639571.4730887,"msg":"adapted config to JSON","adapter":"caddyfile"}
caddy  | {"level":"warn","ts":1729639571.4731326,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
caddy  | {"level":"info","ts":1729639571.474466,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]}
caddy  | {"level":"info","ts":1729639571.4748147,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0004b2c80"}
caddy  | {"level":"info","ts":1729639571.4749014,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
caddy  | {"level":"info","ts":1729639571.4749603,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
caddy  | {"level":"warn","ts":1729639571.475005,"logger":"http.auto_https","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv1","http_port":80}
caddy  | {"level":"info","ts":1729639571.476195,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
caddy  | {"level":"info","ts":1729639571.4766748,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
caddy  | {"level":"info","ts":1729639571.4769409,"logger":"http.log","msg":"server running","name":"srv1","protocols":["h1","h2","h3"]}
caddy  | {"level":"info","ts":1729639571.4769833,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["thatrandompi.xyz","premiers.thatrandompi.xyz","mail.thatrandompi.xyz","hass.thatrandompi.xyz","ssh.thatrandompi.xyz","testing.thatrandompi.xyz","docker.thatrandompi.xyz","flame.thatrandompi.xyz","notlawson.thatrandompi.xyz","absentee.thatrandompi.xyz"]}
caddy  | {"level":"info","ts":1729639571.4810016,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
caddy  | {"level":"info","ts":1729639571.4812045,"msg":"serving initial configuration"}
caddy  | {"level":"info","ts":1729639571.4811532,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"620455d3-e964-4394-af8a-3335d7c65628","try_again":1729725971.4811513,"try_again_in":86399.999999577}
caddy  | {"level":"info","ts":1729639571.4813263,"logger":"tls","msg":"finished cleaning storage units"}

3. Caddy version:

2.8.4 is the version in my Dockerfile

4. How I installed and ran Caddy:

a. System environment:

Custom Docker image, in Docker compose. Ubuntu.

FROM caddy:2.8.4-builder AS builder

RUN xcaddy build \
    --with github.com/aksdb/caddy-cgi/v2

FROM caddy:2.8.4

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

We use a Google WiFi Mesh

b. Command:

docker compose up -d

c. Service/unit/compose file:

services:
  caddy:
    build: .
    network_mode: host
    privileged: true
    restart: always
    container_name: caddy
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - caddy_data:/data
      - caddy_config:/config
      - /home/server_files/notlawson/server_files/homepage:/usr/homepage
      - /home/server_files/notlawson/sever_files/sites:/usr/sites
volumes:
  caddy_data:
  caddy_config:

d. My complete Caddy config:

{
        debug
}
{
    order cgi before respond
}
notlawson.thatrandompi.xyz {
        root * /usr/homepage
        file_server
}
thatrandompi.xyz {
        respond "Caddy is online"
}
hass.thatrandompi.xyz {
        reverse_proxy localhost:8123 {
                header_up Host {host}
                header_up X-Real-IP {remote_host}
                header_up X-Forwarded-For {remote_host}
                header_up X-Forwarded-Proto {scheme}
        }
}
ssh.thatrandompi.xyz {
        reverse_proxy localhost:8182
}
testing.thatrandompi.xyz {
        reverse_proxy localhost:5000
}
mail.thatrandompi.xyz {
        redir https://thatrandompi.xyz
}
docker.thatrandompi.xyz {
        reverse_proxy https://localhost:9443 {
                transport http {
                        tls_insecure_skip_verify
                }
        }
}
absentee.thatrandompi.xyz {
        reverse_proxy localhost:5052
}
http://watchtower.thatrandompi.xyz {
        reverse_proxy localhost:8900
}
premiers.thatrandompi.xyz {
        reverse_proxy localhost:5000
        cgi /update /home/notlawson/server_files/premiers/update.sh
}
flame.thatrandompi.xyz {
        reverse_proxy localhost:5005
}

5. Links to relevant resources:

Not really sure, sorry.
I can connect to my server while on my home network just fine, just externally it fails.
My router is port forwarding the respective ports (80, 443), but an open port checker shows that there is only the SSH port open.

Thanks in advance,
Lawson

Howdy @NotLawson, welcome to the Caddy community.

Did you change your ISP or router recently? I know that the provider your IP is associated with use CGNAT for residential connections, although port 22 still being open implies that might not be an issue.

Worth double checking, though, and worth double checking port forwards etc.

2 Likes

Remove all this from your config. Look at your logs, it warns about these headers, Caddy already sets those correctly by default.

2 Likes

Ello,

I haven’t recently changed anything with my ISP or router.
However, we use Google Nest WiFi, which has been really buggy as of late. I think this may be related, but am not sure, hence why I’m here.
I also had to specifically call my ISP (Aussie Broadband) when we first switched to unblocks ports such as 80 and 443, however I’m pretty sure the block extended to 22, so :person_shrugging:. That didn’t seem to do anything to fix the issue though. Also double checked the Google Home app, deleted and readded the port configuration, but that didn’t work.

I removed the config, which I got off an old tutorial about Caddy and HomeAssistant, and have never bothered to fix up because if it ain’t broke don’t fix it.

I kind of need my server to work because it is running my Digital Solutions Assesment/Premiers Coding Challenge Submission. It’s not critical, but would be greatly appreciated. Thanks for the speedy replies.

Many thanks, Lawson :smile:

If it’s working on your LAN but not from the internet, I’m not sure there’s anything Caddy can do to help, I’m afraid.

Something, somewhere in the middle isn’t passing packets through. It could be the ISP (probably not, Aussie Broadband is great), it could be your router. The only thing you know for sure is it’s not your server since you can get your pages over the local network.

2 Likes

Sorry about the late response, I think I need to give Aussie BB a call to unblock the ports again. Port 80 and 443 are usually blocked according to Aussie’s website, but not port 22 or even the MC port (22565). I’ll do this when I get a chance. Will update if it worked.
Cheers, Lawson

1 Like

All ports would be non-functional for incoming requests if CGNAT is still enabled.

1 Like