Using certbot behind caddy

Help
1

1. The problem I’m having:

I’m trying to configure an ssl certificate for a webpage running behind caddy with certbot. I have configured caddy to redirect to 443 on the webpage server however certbot and browsers fail to access the webpage. As the webpage does not use SSL yet I don’t think caddy can forward to it, so I get an HTTP 502 error when I go to the domain I have configured. If I can’t access the webpage without an SSL cert, and to get an SSL cert I need to have access to the webpage, how do I proceed? Can someone guide me to the answer to the chicken and the egg?

Network setup is 443 is the only port open on the network which redirects to caddy, and caddy distributes depending on the domain name used (I have another server behind caddy that works fine (subdomain1) so i think caddy is fine and it is just certbot and caddy that clash)

2. Error messages and/or full log output:

i kept getting error 502 if i pasted it directly in here so here is a link
https://www.mediafire.com/file/7wk00ibjdpvlfik/log-redacted.txt/file

3. Caddy version:

v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A=

4. How I installed and ran Caddy:

Installed using the Ubuntu packages

a. System environment:

Ubuntu 20.04.6 LTS, x86

b. Command:

caddy.service file

c. Service/unit/compose file:

# caddy.service
#
# For using Caddy with a config file.
#
# Make sure the ExecStart and ExecReload commands are correct
# for your installation.
#
# See https://caddyserver.com/docs/install for instructions.
#
# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.

[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target

[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

d. My complete Caddy config:

# The Caddyfile is an easy way to configure your Caddy web server.
#
# Unless the file starts with a global options block, the first
# uncommented line is always the address of your site.
#
# To use your own domain name (with automatic HTTPS), first make
# sure your domain's A/AAAA DNS records are properly pointed to
# this machine's public IP, then replace ":80" below with your
# domain name.
{
    debug # for the log file above only
}
subdomain1.domain.net {
        reverse_proxy 192.168.2.5:8096
}
subdomain2.domain.net {
        reverse_proxy 192.168.2.4:443
}.

Note that i have replaced the true domain names with subdomain 1 and 2, i have replaced them universally and i can not share the true domain names as their unknowingness is one line of security i have setup. subdomain1 works fine but subdomain2 is the one that doesnt work with certbot (im not saying subdomain1 works with certbot, im saying everything is fine with subdomain1)

2

Just proxy over HTTP instead, don’t proxy over HTTPS. There’s no benefit to proxy over HTTPS within a private network.

3

Is there a way to do so even though it is not recommended?

EDIT:
I feel like I got fixated on this solving the issue and completely neglected the main problem I’m having, sorry about that. What the full setup is, is a service that is running on a machine, and that machine is trying to use Certbot to obtain a certificate for the domain on that machine. This has nothing to do with https, but instead, the service it is running needs to verify the certificate, meaning I do need a certificate. I think Caddy is intercepting the challenge and not letting it pass on, is that the problem and can I get a certificate on the machine running behind caddy to use for the service.

In my research I came across this post which you answered in one reply saying caddy does not forward those challenges. Is there a way to enable it now?

4

You could probably use GitHub - mholt/caddy-l4: Layer 4 (TCP/UDP) app for Caddy

Then you could prevent the TLS Termination by caddy and send the TCP stream directly to the backend server.

Though if the backend service really needs a certificate for some reason, just use a self-signed one? Thats going to be way easier.

5

I don’t understand. What’s the purpose then?

I think you’d need to set up an HTTP site with a reverse_proxy using a /.well-known/acme-challenge/* matcher to only intercept those requests.

6

Sorry I don’t want to hijack this thread, but I’m really interested in this for a few reasons.
I always thought that Caddy would intercept all of these HTTP-01 ACME challenges and route them to itself.

Judging from your statement, something like this should be possible?

http://foo.example.com:80 {
        handle /.well-known/acme-challenge/* {
                reverse_proxy 192.168.1.1 {
                }
        }
}
foo.example.com:443 {
        reverse_proxy 192.168.1.1 {
        }
}

Expected behavior:

  • Caddy listens on Port 80 for foo.example.com, reverse proxying /.well-known/acme-challenge/* to 192.168.1.1:80
  • Caddy will get a certificate with TLS-ALPN-01 challenge (since HTTP-01 will fail) for foo.example.com:443 and reverse proxy all requests to 192.168.1.1:80.
  • Caddy listens on Port 80 for all other domains, hosting the /.well-known/acme-challenge/* for them.
7

@SpaceFlier here is the Electronic Frontier Foundation (EFF)
stance on Certbot’s future:

8

Yeah, this should work:

http://example.com {
	handle /.well-known/acme-challenge/* {
		reverse_proxy <your-upstream>
	}
	handle {
		redir https://{host}{uri} 308
	}
}

example.com {
	respond "https"
}

You need that 2nd handle to preserve the HTTP->HTTPS redirects

9

Nice thank you, that looks easy to template. It’s one of the more frequent questions I get. Although I don’t know why, I haven’t seen a single docker, container or other application yet that was hard coded for Let’s Encrypt in a way that it doesn’t allow for plain HTTP or self-signed certificates. So this one is a rather edge case.