Caddy and Certbot

Help
#1

1. Caddy version (caddy version):

v2.3.0 h1:fnrqJLa3G5vfxcxmOH/+kJOcunPLhSBnjgIvjXV/QTA=

2. How I run Caddy:

{
        experimental_http3
        admin off
}
(logging) {
        log {
                output file /var/log/caddy/caddy.log {
                        roll_size 1gb
                        roll_keep 5
                }
        }
}
(errors) {
        handle_errors {
                root * /var/www/internal/errors
                rewrite * /{http.error.status_code}.html
                file_server
        }
}
(php) {
        php_fastcgi unix//run/php/php8.0-fpm.sock
}
www.telesphoreo.me {
        import logging
        redir https://telesphoreo.me{uri}
}
telesphoreo.me {
        import logging
        import errors
        import php
        root * /var/www/telesphoreo.me
        file_server browse
        encode gzip zstd
        respond /assets/ 403
        respond /files/ 403
        respond /files/theme/ 403
        respond /files/theme/fonts/ 403
        respond /files/theme/images/ 403
        respond /uploads/2/6/3/9/26395298/ 403
        respond /uploads/2/6/3/9/26395298/background-images/ 403
        respond /nitrogen/ 403
        respond /wave/ 403
}
db.telesphoreo.me {
        import logging
        import php
        root * /usr/share/phpmyadmin
        file_server
}
i.telesphoreo.me {
        import logging
        import errors
        root * /var/www/i.telesphoreo.me
        file_server
}
forums.telesphoreo.me {
        import logging
        import php
        root * /var/www/forums.telesphoreo.me/public
        file_server
        header /assets {
                +Cache-Control "public, must-revalidate, proxy-revalidate"
                +Cache-Control "max-age=25000"
                Pragma "public"
        }
}
panel.telesphoreo.me {
        reverse_proxy https://panel.telesphoreo.me:4431
}
updater.telesphoreo.me {
        import logging
        import errors
        root * /var/www/updater.telesphoreo.me
        file_server browse
}
packs.smokes-crystal.rocks {
        import logging
        redir https://i.telesphoreo.me{uri}
}
www.smokes-crystal.rocks {
        import logging
        redir https://smokes-crystal.rocks{uri}
}
smokes-crystal.rocks {
        import logging
        import errors
        root * /var/www/smokes-crystal.rocks
        file_server
}

a. System environment:

Ubuntu 20.04, systemd

b. Command:

sudo service caddy start

c. Service/unit/compose file:

I don’t know where this file is

d. My complete Caddyfile or JSON config:

See above

3. The problem I’m having:

So basically I run Pterodactyl (https://pterodactyl.io) which is a game panel. The setup is that Pterodactyl is on Apache (it doesn’t like Caddy). I have Apache on port 4431 and Caddy reverse proxies that to port 443 since I don’t want users to have to enter a port name. Apache is setup with HTTPS, Caddy is just reverse proxying it. However, it is all setup with Certbot. I basically just run certbot certonly -d panel.telesphoreo.me which gives me a Certbot certificate for that domain only. The problem is that it is not able to run when Caddy is running. I guess both of them require port 80 for certificate renewals so they conflict. What happened in March was there was an outage because the certificate from Certbot expired and it couldn’t renew itself. I had to stop Caddy and renew it manually. I tried to have Apache use Caddy’s certificates but was unsuccessful. It’s also important to note that there is a daemon called Wings. This requires an SSL certificate as well and it just works with Certbot’s certificate. So I’m wondering if there is a better way than having to renew it manually everytime with Certbot

4. Error messages and/or full log output:

There are no errors really. This is the Certbot error


root@telesphoreo:/etc/apache2/sites-enabled# certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/panel.telesphoreo.me.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator standalone, Installer None
Account registered.
Simulating renewal of an existing certificate for panel.telesphoreo.me
Performing the following challenges:
http-01 challenge for panel.telesphoreo.me
Cleaning up challenges
Failed to renew certificate panel.telesphoreo.me with error: Problem binding to port 80: Could not bind to IPv4 or IPv6.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All simulated renewals failed. The following certificates could not be renewed:
  /etc/letsencrypt/live/panel.telesphoreo.me/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)

Of course, this is fixed by stopping Caddy and doing the renewal.

5. What I already tried:

Well, I mean that is a solution. I’m curious if there is a better way to do this instead of remembering in my head that I have to renew my certificates by a certain date. Supposedly I can use something by Cloudflare, but I simply could not get that working. I have also had really bad experiences with Cloudflare in the past as they have denied requests to my server on my behalf which broke a lot of things.

6. Links to relevant resources:

#2

If they’re running on the same machine, you don’t need certbot at all, and you don’t need to run Apache with HTTPS. It’s perfectly fine to proxy to Apache over HTTP.

Certbot won’t work because it requires access to ports 80 and 443 to solve the ACME challenges, but when running Caddy, Caddy does that itself.

There’s really never a situation where you should ever use both certbot and Caddy on the same machine. Caddy fully replaces the need for certbot.

1 Like
#3

They both are on the same machine. I have to reverse proxy HTTPS to HTTPS otherwise Pterodactyl thinks its HTTP which breaks some things. The only certificates I see from Caddy are panel.telesphoreo.me.crt, panel.telesphoreo.me.json, panel.telesphoreo.me.key
Apache requires this:

  SSLCertificateFile /etc/letsencrypt/live/panel.telesphoreo.me/fullchain.pem
  SSLCertificateKeyFile /etc/letsencrypt/live/panel.telesphoreo.me/privkey.pem

and so does Pterodactyl to work. I don’t see any pem files from Caddy so I’m not sure I can use Caddy certificates that way

#4

You should ask the Pterodactyl devs to handle the X-Forwarded-Proto header to determine that HTTPS is being used. Caddy sets that header automatically with reverse_proxy and apps can look at it to know whether it was loaded over HTTPS.

If you can find out what mechanism they actually use to determine whether it was HTTPS, then you can probably configure Apache to “fake” it.

.pem and .crt files are the same thing, just a different name. PEM is an encoding format, but Caddy uses .crt and .key to differentiate their purpose (.crt for certificates, .key for private keys).

It doesn’t make sense to configure Apache to use the certificates that Caddy manages. That won’t be reliable.

I’d like to know why “it doesn’t like Caddy” though, or what led you to believe that. You can probably configure Caddy such that it’ll work. But I can’t help without knowing what isn’t working.

1 Like
#5

Taking a look at the panel project, I find it hard to believe Caddy won’t just work out of the box with it using the php_fastcgi directive and php-fpm. It’s a standard Laravel app, and Caddy works great with Laravel apps with the default behaviour of the php_fastcgi directive.

1 Like
#6

Seeing that you already have php-fpm v8.0 installed for another site, it should work just fine for you if you’re using at least v1.3.0 of panel, which is the first version which supports 8.0 it seems, as per Releases · pterodactyl/panel · GitHub

1 Like
#7

I previously made a thread on why Pterodactyl doesn’t work with Caddy: Caddy with Pterodactyl

Interestingly, I just tried again with the exact config in that thread and I am not having any of the issues that I previously had.

However, the last problem is with WIngs (the daemon for the panel). That also uses the same certificate that Certbot made.
This is currently the configuration file

...
api:
  host: 0.0.0.0
  port: 8080
  ssl:
    enabled: true
    cert: /etc/letsencrypt/live/panel.telesphoreo.me/fullchain.pem
    key: /etc/letsencrypt/live/panel.telesphoreo.me/privkey.pem
  upload_limit: 100
...

I tried replacing it with the directory to Caddy’s certificates and now it gives this error
2021/05/09 14:23:25 http: TLS handshake error from <ip>:53012: remote error: tls: unknown certificate
Self explanatory, it doesn’t like that certificate.

    cert: /root/.local/share/caddy/certificates/acme-v02.api.letsencrypt.org-directory/panel.telesphoreo.me/panel.telesphoreo.me.crt
    key: /root/.local/share/caddy/certificates/acme-v02.api.letsencrypt.org-directory/panel.telesphoreo.me/panel.telesphoreo.me.key
#8

Just remove this stuff and proxy over HTTP, and you should be good to go.

#9

I cannot. If I set enabled to false it does not work. I get error 500’s everywhere. It acts as if it cannot establish a connection at all without SSL enabled. These are the options I have

I’m not sure what you mean exactly by proxy over HTTP.

#10

Pick Use HTTP Connection there and you probably want to change Fully Qualified Domain Name to just localhost or 127.0.0.1 probably. Probably also pick Behind Proxy (since the explanation for that says “select this to have the daemon skip looking for certificates on boot” which is the case here).

#11

What’s unclear to me is whether panel is the only thing talking to wings, or if you have clients connecting to wings directly from outside. If the latter, then add a site block like this to Caddy:

wings.telesphoreo.me {
	reverse_proxy localhost:8080
}

Then change the “Fully Qualified Domain Name” to wings.telesphoreo.me and pick Use SSL Connection (since Caddy will handle HTTPS) and pick Behind Proxy (since Caddy will proxy it).

And then make sure the clients use https://wings.telesphoreo.me (I assume it was using https://panel.telesphoreo.me:8080 before?)

#12

I believe you connect to Wings from the outside, yes. I did that but it looks like the DNS records haven’t propagated. I can go to https://wings.telesphoreo.me and ping it from my desktop. Running ping from the server itself says Name or service not found. I presume it’s just a DNS issue but it looks like in theory that will work. I’ll let you know the results when the server accepts it

#13

Unfortunately, this isn’t working. The panel returns a 500 error when trying to change the FQDN to wings.telesphoreo.me. Also, whenever I go to wings.telesphoreo.me it says HTTPS request sent over HTTP. If I go to wings.telesphoreo.me:8080, it does work but it gives an invalid certificate warning. I assume this is because it’s trying to connect to panel.telesphoreo.me. So I guess it isn’t possible to make Wings work like that. I’ll concede defeat, as what’s working now is better than what I had before.

#14

I’m sure there’s a way. I think the problem is what I wrote above:

I don’t know this app at all, but there must be a way to make it use port 443 for incoming requests, i.e. a different port publicly than what it’s using internally.

If there isn’t, then that’s an oversight on the end of their project because that’s not an atypical setup.

#15

Well the thing is that it’s still trying to use panel.telesphoreo.me:8080. The setting above for the FQDN is what I need to change. However, if I do it from the panel, it doesn’t let me do it. I get an error 500. Basically Wings is trying to communicate with the panel the same way the client does. The option to use SSL is for connecting externally from wings to the panel (I believe). The port (8080) can be changed.

If I change the FQDN in the config, it repeats this over and over again

DEBUG: [May  9 18:02:14.134] making request to external HTTP endpoint endpoint=https://wings.telesphoreo.me/api/remote/servers?page=0&per_page=50 headers=map[Accept:[application/vnd.pterodactyl.v1+json] Authorization:[(redacted)] Content-Type:[application/json] User-Agent:[Pterodactyl Wings/v1.4.2 (id:Ff8LtAelviPCoE4C)]] method=GET

If it would establish a connection, this line would appear only once but it repeats over and over, so it can’t. It tries to establish a connection like this when the service is starting, so it gets stuck here and never finishes starting. I think this is because it wants wings and panel on the same subdomain. I also think it could be because for wings.telesphoreo.me to establish a connection to the panel, it would have to be over HTTP. But Caddy is making wings.telesphoreo.me HTTPS but wings.telesphoreo.me:8080 ends up getting the certificate for panel.telesphoreo.me instead.
If I try to set the port to 443, Wings doesn’t start as the port is already in use.
Edit: so, it looks like wings.telesphoreo.me:8080 is actually using the Certbot certificate. wings.telesphoreo.me is using the Caddy certificate. I’m not exactly sure how to fix this issue

#16

So I changed the reverse proxy from localhost:8080 to panel.telesphoreo.me:8080. It let me do HTTPS requests from wings.telesphoreo.me.
However, I think I see the problem. Basically what’s happening is that we’re telling wings to connect to itself. What should be happening is that wings is connecting to the panel. It needs to do this over SSL. When the panel itself is using HTTPS, requests from wings to the panel also have to be HTTPS. I think the panel.telesphoreo.me/api is what needs proxying. I think the goal is to try to get wings to talk to the panel locally instead of the way it’s doing to get an SSL certificate

#17

So I’m not sure if this has anything to do with it, but if I try and connect to the panel via panel.telesphoreo.me:4431 (which is no longer being reverse proxied), it acts as if wings cannot even establish a connection. Whereas if I use panel.telesphoreo.me, wings “magically” appears like it’s back online (even though nothing was changed). So I think that it’s just a picky piece of software and it won’t work. I tried reverse proxying /api/* to https://panel.telesphoreo.me:8080 and localhost:8080 to see if that would work, but it didn’t

#18

This topic was automatically closed after 30 days. New replies are no longer allowed.