Lets Encrypt, Error 400 - connection refused

[Kened33r (Pan)]

1. Caddy version (caddy version): latest

2. How I run Caddy:

docker rootless

a. System environment:

Ubuntu 20.04 LTS

b. Command:

docker-compose up

I know this error has been described on other posts related to not opened privileged ports.
However, I used this setting in docker-compose:

    cap_add:
      - NET_BIND_SERVICE 

Or typed this in terminal

sudo setcap cap_net_bind_service=ep $HOME/bin/rootlesskit

the domain DNS records A point to server

ERROR:

{"level":"error","ts":1614773127.8234901,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"example.com","challenge_type":"http-01","status_code":400,"problem_type":"urn:ietf:params:acme:error:connection","error":"Fetching http://example.com/.well-known/acme-challenge/TtHOyd8nydX-zD7B5pAKBXrLLMDz1J-8JlXTOBH7Vcc: Connection refused"}
{"level":"error","ts":1614773127.8235958,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"example.com","error":"authorization failed: HTTP 400 urn:ietf:params:acme:error:connection - Fetching http://example.com/.well-known/acme-challenge/TtHOyd8nydX-zD7B5pAKBXrLLMDz1J-8JlXTOBH7Vcc: Connection refused","order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/18372708/626124","attempt":1,"max_attempts":3}

Have no idea, where is the problem.

[Rob789 (Robbert)]

Hi,

Did you open port 443 in your firewall and configured the port forwarding to the correct host where Caddy runs?

[Kened33r (Pan)]

Iptables allows traffic 443 port in both directions.
Port forwarding is working by docker rules.

[Rob789 (Robbert)]

You also need to setup your router.

This Wiki has all the details to get this working:

Using Caddy as a reverse proxy in a home network

If you want to run a service inside a Local Area Network (LAN) such as your home or office – and especially if you want to be able to access it from outside that network – Caddy can help you accomplish this quite easily. This guide will show you how. It assumes you’ve never done this before, but that you have some technical proficiency and are somewhat knowledgable about your own network. Do this at your own risk. There are several ways to make private services accessible from other n…

[Kened33r (Pan)]

I don’t use it at home. It is VPS.

Caddy on rootless docker cannot open ports as I mentioned earlier, even after use of some commands.

[francislavoie]

What does your docker-compose.yml look like? Are you mapping ports 80 and 443 to your host for your caddy service?

[Kened33r (Pan)]

I was mapping ports and had the same problem. Now I use default privileged onces with

sudo setcap cap_net_bind_service=ep $HOME/bin/rootlesskit

Caddy should use and open 80 443 ports with this above, but doesn’t work. Caddy on docker-root works.

docker-compose caddy service

caddy:
    container_name: caddy
    image: caddy:latest
    restart: unless-stopped
    environment:
      - GHOST=example.com
      - ANALYTICS=example2.com
    volumes:
      - ./caddy/Caddyfile:/etc/caddy/Caddyfile
      - ./caddy/caddy_data:/data
      - ./caddy/caddy_config:/config

[Mohammed90 (Mohammed Al Sahaf)]

You docker-compose config isn’t exposing the ports 80 and 443. The ports should be exposed in the docker-compose.yml to be accessible from host. You can add this:

ports:
   - "80:80"
   - "443:443"

The syntax accepts more details, which you can find in the Docker Compose docs.

[Kened33r (Pan)]

I had to undo marking “the solution”.

I have no idea, but now your solution is not working and get the same 400 error. I did not changed docker settings, caddyfile etc. I am pissed off, why once it worked, now it doesn’t.

[Mohammed90 (Mohammed Al Sahaf)]

What do you see now? What’s in your logs? What changed in your system? What’s in your firewall configuration?

[Kened33r (Pan)]

Tricky part, I haven’t changed anything last time in iptables and caddy on root with the same iptables works fine.

Basically sudo setcap cap_net_bind_service=ep $HOME/bin/rootlesskit
should solve the problem, but it doesn’t.

I figured out, that hardening dockers e.g. making rootless docker breaks containers.

[Kened33r (Pan)]

here is the error

{"level":"error","ts":1615058999.0843666,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"www.example.com","challenge_type":"http-01","status_code":400,"problem_type":"urn:ietf:params:acme:error:connection","error":"Fetching http://www.example.com/.well-known/acme-challenge/zYMNLEIZnB0kn42w3pzR2f92a9j-qCiMuGVO_xJwD1Y: Connection refused"}
{"level":"error","ts":1615058999.0845687,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"www.example.com","error":"authorization failed: HTTP 400 urn:ietf:params:acme:error:connection - Fetching http://www.example.com/.well-known/acme-challenge/zYMNLEIZnB0kn42w3pzR2f92a9j-qCiMuGVO_xJwD1Y: Connection refused","order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/18430775/3351028","attempt":2,"max_attempts":3}
{"level":"error","ts":1615058999.4030497,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"example.com","challenge_type":"http-01","status_code":400,"problem_type":"urn:ietf:params:acme:error:connection","error":"Fetching http://example.com/.well-known/acme-challenge/Wa9xR4qPguJCw-8EF_dzdKx7j7dJlk1AHlISGpaw9W0: Connection refused"}
{"level":"error","ts":1615058999.4030766,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"example.com","error":"authorization failed: HTTP 400 urn:ietf:params:acme:error:connection - Fetching http://example.com/.well-known/acme-challenge/Wa9xR4qPguJCw-8EF_dzdKx7j7dJlk1AHlISGpaw9W0: Connection refused","order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/18430775/3351030","attempt":2,"max_attempts":3}
{"level":"error","ts":1615059000.5706189,"logger":"tls.obtain","msg":"will retry","error":"[www.example.com] Obtain: [www.example.com] solving challenges: www.example.com: no solvers available for remaining challenges (configured=[http-01 tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[dns-01]) (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/18430775/3351040) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)","attempt":6,"retrying_in":1200,"elapsed":1232.698359637,"max_duration":2592000}
{"level":"error","ts":1615059000.888961,"logger":"tls.obtain","msg":"will retry","error":"[example.com] Obtain: [example.com] solving challenges: example.com: no solvers available for remaining challenges (configured=[tls-alpn-01 http-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[dns-01]) (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/18430775/3351045) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)","attempt":6,"retrying_in":1200,"elapsed":1233.00461786,"max_duration":2592000}

[Kened33r (Pan)]

and now it worked again. Caddy got certs. Why? I have no idea, not changed anything or at least I am not aware of.

[Mohammed90 (Mohammed Al Sahaf)]

There’s no reason for Caddy to fail intermittently in this manner unless there are system level changes. Double check the system setup. Is DNS consistent? I’m genuinely thinking you have DNS issue. Look into that.