HTTP-01 solver attempting to listen on 5033

Help
1

Something strange happened with my caddy deployment. Caddy tried to renew certificates, and failed. According to the logs it appears that it was trying to bind to port 5033. It continued to fail and later the certificates expired.

Aug 06 04:23:27 web-n02 caddy[1097]: 2018/08/06 04:23:27 [INFO] Certificate for [texaslinuxfest.com] expires in 718h49m16.702478463s; attempting renewal
Aug 06 04:23:27 web-n02 caddy[1097]: 2018/08/06 04:23:27 [INFO][texaslinuxfest.com] acme: Trying renewal with 718 hours remaining
Aug 06 04:23:27 web-n02 caddy[1097]: 2018/08/06 04:23:27 [INFO][texaslinuxfest.com] acme: Obtaining bundled SAN certificate
Aug 06 04:23:27 web-n02 caddy[1097]: 2018/08/06 04:23:27 [INFO][texaslinuxfest.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz/7qOORROX_fwNT-hDG89LbQRnmItB2Fly0rsXU1haFFI
Aug 06 04:23:27 web-n02 caddy[1097]: 2018/08/06 04:23:27 [INFO][texaslinuxfest.com] acme: Could not find solver for: dns-01
Aug 06 04:23:27 web-n02 caddy[1097]: 2018/08/06 04:23:27 [INFO][texaslinuxfest.com] acme: Could not find solver for: tls-sni-01
Aug 06 04:23:27 web-n02 caddy[1097]: 2018/08/06 04:23:27 [INFO][texaslinuxfest.com] acme: Trying to solve HTTP-01
Aug 06 04:23:27 web-n02 caddy[1097]: 2018/08/06 04:23:27 [ERROR] Renewing [texaslinuxfest.com]: acme: Error -> One or more domains had a problem:
Aug 06 04:23:27 web-n02 caddy[1097]: [texaslinuxfest.com] [texaslinuxfest.com] error presenting token: presenting with standard HTTP provider server: Could not start HTTP server for challenge -> listen tcp :5033: bind: permission denied
Aug 06 04:23:27 web-n02 caddy[1097]: ; trying again in 10s

I restarted the service and it renewed the certificates without issue. Anyone else seen this behavior before or know of a fix?

/usr/lib/systemd/system/caddy.service:

[Unit]
Description=Caddy HTTP/2 web server
Documentation=https://caddyserver.com/docs
After=network.target

[Service]
User=caddy
Group=caddy
Environment=CADDYPATH=/var/lib/caddy
EnvironmentFile=-/etc/caddy/envfile
ExecStartPre=/usr/bin/caddy -conf /etc/caddy/caddy.conf -validate
ExecStart=/usr/bin/caddy -conf /etc/caddy/caddy.conf -log stdout -root /tmp -agree
ExecReload=/usr/bin/kill -USR1 $MAINPID
KillMode=mixed
KillSignal=SIGQUIT
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectHome=true
ProtectSystem=full
ReadWriteDirectories=/var/lib/caddy
AmbientCapabilities=CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

/etc/caddy/caddy.conf:

import conf.d/*.conf

/etc/caddy/conf.d/2018.texaslinuxfest.org.conf:

texaslinuxfest.org, www.texaslinuxfest.org, texaslinuxfest.com, www.texaslinuxfest.com, texaslinuxfest.net, www.texaslinuxfest.net, 2016.texaslinuxfest.org {
    redir https://2018.texaslinuxfest.org
}

2018.texaslinuxfest.org {
    root /srv/www/txlf2018
    proxy / http://127.0.0.1:8000 {
        transparent
        except /static
    }
    gzip
    errors stderr
    log stdout
}

# rpm -q caddy
caddy-0.11.0-1.el7.x86_64
# caddy --version
Caddy 0.11.0 (unofficial)
2

Caddy binds to port 5033 to solve the HTTP challenge because the solver implemented by lego, our ACME library, is told to listen on that port (Caddy then proxies HTTP challenge requests to that listener). I’m not sure what would cause the permissions error, though – seems like a system issue.

3

Thanks for the clarification. That appears to be the case, I found an selinux denial for port 5033.

type=AVC msg=audit(1536899161.184:2301604): avc:  denied  { name_bind } for  pid=1097 comm="caddy" src=5033 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket

Similar to 1608548 – caddy: selinux blocking QUIC, this is further confirmation that sharing the Apache HTTPD policy isn’t sufficient. I’ll have to create a policy to handle this.

I am confused about one thing. If it needs to bind to port 5033, and selinux was blocking that from happening, how did restarting the service correct it? Does Caddy always need to bind to 5033 when renewing certificates?

4

Nope. :slight_smile: Only when port 80 is already in use (by Caddy).

We could theoretically do it without binding another port at all, but this was easier and, at least at the time, it meant we didn’t have to copy a bunch of code; we could reuse it instead.

5

I think I’ve got a workable solution for this for the RPM packages. Are there any other ports I should whitelist also? Does TLS-SNI-01 also bind to 5033?

6

No need to worry about TLS-SNI-01 (or TLS-SNI-02) - they were found to be exploitable, so LetsEncrypt disabled them.

I think they were working on TLS-ALPN-02 as a replacement?

Anyway, TLS-SNI-01/02 aren’t coming back.

1 Like
7

Correct; and TLS-ALPN in Caddy (coming soon) does not require an extra port.

1 Like
8

In case anyone else runs across this problem, it should be fixed in caddy-0.11.0-3 (EL7, F27, F28, F29).