HTTP-01 timeout

1. The problem I’m having:

Setting up a new server, pointed a subdomain to the public IP and verified using traefik/whoami that port 80 is indeed accessible externally by the URL.
When running caddy, it fails solving the http-01 challenge, saying it has timed out and this may be a firewall error, but seeing as whoami was easily accessible, I tend to think that might not be the case.
curl seems to hang and timeout as well when trying to access via the domain address or the public IP; only when trying to access via the LAN IP does caddy log the attempt.

2. Error messages and/or full log output:

{
    "level": "error",
    "ts": 1764277431.7495704,
    "msg": "challenge failed",
    "identifier": "meshi.yeda-water.com",
    "challenge_type": "http-01",
    "problem": {
        "type": "urn:ietf:params:acme:error:connection",
        "title": "",
        "detail": "62.90.48.190: Fetching http://meshi.yeda-water.com/.well-known/acme-challenge/Ba7EhYw5F47FTzGBi2qD7scCbqa-_toT7rrNRVzmGIU: Timeout during connect (likely firewall problem)",
        "instance": "",
        "subproblems": null
    },
    "stacktrace": "github.com/mholt/acmez/v3.(*Client).pollAuthorization\n\tgithub.com/mholt/acmez/v3@v3.1.2/client.go:557\ngithub.com/mholt/acmez/v3.(*Client).solveChallenges\n\tgithub.com/mholt/acmez/v3@v3.1.2/client.go:378\ngithub.com/mholt/acmez/v3.(*Client).ObtainCertificate\n\tgithub.com/mholt/acmez/v3@v3.1.2/client.go:136\ngithub.com/caddyserver/certmagic.(*ACMEIssuer).doIssue\n\tgithub.com/caddyserver/certmagic@v0.24.0/acmeissuer.go:489\ngithub.com/caddyserver/certmagic.(*ACMEIssuer).Issue\n\tgithub.com/caddyserver/certmagic@v0.24.0/acmeissuer.go:382\ngithub.com/caddyserver/caddy/v2/modules/caddytls.(*ACMEIssuer).Issue\n\tgithub.com/caddyserver/caddy/v2@v2.10.2/modules/caddytls/acmeissuer.go:288\ngithub.com/caddyserver/certmagic.(*Config).obtainCert.func2\n\tgithub.com/caddyserver/certmagic@v0.24.0/config.go:626\ngithub.com/caddyserver/certmagic.doWithRetry\n\tgithub.com/caddyserver/certmagic@v0.24.0/async.go:104\ngithub.com/caddyserver/certmagic.(*Config).obtainCert\n\tgithub.com/caddyserver/certmagic@v0.24.0/config.go:700\ngithub.com/caddyserver/certmagic.(*Config).ObtainCertAsync\n\tgithub.com/caddyserver/certmagic@v0.24.0/config.go:505\ngithub.com/caddyserver/certmagic.(*Config).manageOne.func1\n\tgithub.com/caddyserver/certmagic@v0.24.0/config.go:415\ngithub.com/caddyserver/certmagic.(*jobManager).worker\n\tgithub.com/caddyserver/certmagic@v0.24.0/async.go:73"
}

3. Caddy version:

v2.10.2 h1:g/gTYjGMD0dec+UgMw8SnfmJ3I9+M2TdvoRL/Ovu6U8=

4. How I installed and ran Caddy:

I’m using the Caddy Docker container

a. System environment:

Ubuntu Server 24 running Caddy within Docker

b. Command:

docker compose up -d caddy

c. Service/unit/compose file:

services:
  caddy:
    image: caddy
    container_name: caddy
    hostname: caddy
    restart: unless-stopped
    ports:
      - 80:80
      - 443:443
      - 443:443/udp
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - ./site:/srv:ro
      - ./log:/var/log
      - data:/data/caddy
      - config:/config/caddy
    env_file:
      - .env

volumes:
  data:
    name: caddy-data
  config:
    name: caddy-config

.env:

EMAIL=dev@deanayalon.com

d. My complete Caddy config:

{
        email {$EMAIL}
}

(tls-insecure) {
        reverse_proxy {args[0]}:443 {
                transport http {
                        tls
                        tls_insecure_skip_verify
                }
        }
}
(log) {
        log {
                output file /var/log/{args[0]}.log
                format json
        }
}


meshi.yeda-water.com:443 {
        import log filemaker
        import tls-insecure fms
}

I know the template abstractions are useless with only one service, this used to have more :sweat_smile:

Can be summed up to this:

{
    email {$EMAIL}
}

meshi.yeda-water.com:443 {
    log {
       output file /var/log/filemaker.log
       format json
    }
    reverse_proxy fms:443 {
       transport http {
           tls
           tls_insecure_skip_verify
       }
    }
}

I should note, this server is going to replace an older one, I already have one running perfectly, set up with almost the exact same configurations, except the subdomain itself.

Any ideas where the problem might lie?

curl seems to hang and timeout as well when trying to access via the domain address or the public IP; only when trying to access via the LAN IP does caddy log the attempt.

If you have a NAT in front of caddy, make sure it handles NAT loopback correctly

My apologies, I’ll be completely honest - I do not know what NAT or NAT loopback are, as well as how to check for its existence and configuration.

A quick google says it allows multiple devices to use a single public IP - about the same way a router may operate? so is this something my host provider may have set up and know about?

Since my requests to the domain do make it all the way to the traefik/whoami container, wouldn’t that mean NAT, if enabled, is directing traffic correctly?

NAT is a feature commonly used with IPv4. It changes addresses to other addresses, with the most common reason reducing the total amount of addresses used on the internet, as IPv4 addresses have ran out.

It looks like your router does not handle loopback correctly. Some routers get this wrong as the firewall thinks a packet from internal to external has the wrong interface, so it gets blocked.

I don’t think this is a NAT hairpinning issue, based on the fact you’re saying “setting up a server”, I assume this is a VPS and not a home server setup.

Either way, this isn’t about you trying to connect to something within your own network (which is when NAT hairpinning triggers, i.e. you try to connect to your WAN IP while the target server is inside your LAN, your router doesn’t correctly route the packets back into your network).

The problem is that Let’s Encrypt can’t connect to your server:

Are you sure that IP is correct? Trying to connect hangs from my machine.

$ curl -v 62.90.48.190
*   Trying 62.90.48.190:80...
^C

$ ping 62.90.48.190                                                            
PING 62.90.48.190 (62.90.48.190) 56(84) bytes of data.
^C
--- 62.90.48.190 ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 5148ms

$ curl -v http://meshi.yeda-water.com
* Host meshi.yeda-water.com:80 was resolved.
* IPv6: (none)
* IPv4: 62.90.48.190
*   Trying 62.90.48.190:80...
^C

You have somekind of firewall or port forwarding issue.

1 Like

Yes, the IP is correct:

  • Your curl does nothing because currently the service is not up, i didn’t want to just leave caddy running trying to issue certificates in an endless failing loop. If caddy was up, you’d be timed out, if whoami was up, you’d correctly get the response whoami sends
  • Your pings wouldn’t work either way because the protocol is not exposed
  • As described earlier, if instead of caddy, I create a traefik/whoami container mapped to port 80, then access through either IP or the domain itself works flawlessly, so the firewall is not preventing access.

Well, Caddy isn’t doing anything special. Try running it with a config like this for testing HTTP:

:80 {
    respond "hello"
}

If that still doesn’t work then you have something wrong with some layer of networking (possibly Docker’s network stack).

1 Like

My apologies for the delayed response, I did not have the chance to test your suggestion, but I appreciate your input

Turns out the error was pretty simply my host provider blocking all requests not originating in our country.

Fernando and Francis, thank you both for your assistance, and I hope you have a great end to 2025 :slight_smile:

4 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.