Nginx with Caddy

[ayoubamine]

1. Caddy version (caddy version):

v2.3.0 h1:fnrqJLa3G5vfxcxmOH/+kJOcunPLhSBnjgIvjXV/QTA=

2. How I run Caddy:

a. System environment:

• Ubuntu 20.04 LTS
• systemd

b. Command:

systemctl start caddy

c. Service/unit/compose file:

[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target

[Service]
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

d. My complete Caddyfile or JSON config:

{
  http_port 8880
  https_port 4443
}

http://ouipneu.net:8880 {
  redir https://ouipneu.net:4443{uri}
}

https://ouipneu.net:4443 {
  respond "Hello, privacy!"
}

3. The problem I’m having:

I installed Caddy on top Hestia to implement Custom Domain for my SaaS with On-Demand SSL generation.

4. Error messages and/or full log output:

With 4443 port all is fine https://ouipneu.net:4443, but without the 4443 port doesn’t work Error code: SSL_ERROR_RX_RECORD_TOO_LONG.

The domain name in this case is the customer domain, I will change the Caddyfile later to:

reverse_proxy * {
    to https://app.mysaas.com

    header_up Host {http.reverse_proxy.upstream.hostport}
    header_up X-Forwarded-For {http.request.remote}
    header_up X-Real-IP {http.reverse-proxy.upstream.address}
    header_up X-Forwarded-Proto {http.request.scheme}
    header_up X-Forwarded-Port {http.request.port}
    header_up X-Forwarded-Host {http.request.host}

    tls {
      on_demand
    }
}

5. What I already tried:

cat /etc/nginx/conf.d/172.31.51.120.conf

server {
    listen       172.31.51.120:80 default;
    server_name  _;

    location / {
        proxy_pass  http://localhost:8880;
   }
}

server {
    listen      172.31.51.120:443;
    server_name _;

    location / {
        proxy_pass  http://localhost:4443;
    }
}

I wonder if there are other solutions that you can recommend or maybe someone already worked with Caddy-On Demand SSL and can share their experience?

6. Links to relevant resources:

[francislavoie]

There’s not enough information here. What did you try, exactly? What did the request look like? Try with curl -v and show the output. What’s in your logs? You can run journalctl -u caddy --no-pager | less to see them, since you’re running as a service.

@ayoubamine

{
  http_port 8880
  https_port 4443
}

Be aware that ACME CAs require port 80 and 443 to be used for the HTTP and TLS-ALPN challenges, so this won’t work to complete certificate issuance.

@ayoubamine

    header_up X-Forwarded-For {http.request.remote}
    header_up X-Real-IP {http.reverse-proxy.upstream.address}
    header_up X-Forwarded-Proto {http.request.scheme}
    header_up X-Forwarded-Port {http.request.port}

You don’t need any of these lines. Caddy sets the appropriate proxy headers automatically:

reverse_proxy (Caddyfile directive) - Caddy Documentation

The Host and X-Forwarded-Host headers are fine to use though since you’ll be using HTTPS to proxy to your app.

@ayoubamine

    tls {
      on_demand
    }

This should be in the site block, not inside reverse_proxy. Mind where the braces are.

@ayoubamine

reverse_proxy * {
    to https://app.mysaas.com

You can shorten this to just reverse_proxy https://app.mysaas.com {

[ayoubamine]

Thank you for your quick reply, I’m working on a SaaS that allows customers to create online invoices and link to a custom domain.

After the customer verifying the domain ownership with TXT record and add CNAME record to custom.example.com, our server should open their invoices links with HTTPS enabled by default.

I’m using Hestia as a control panel for my server and I want to use Caddy to generate On-Demand SSL certificates for my customers.

The 80 and 443 ports are already opened but in use from Nginx, for that, I changed the default Caddy ports from 80 to 8880 and 443 to 4443:

{
  http_port 8880
  https_port 4443
}

and handle the redirections with Nginx:

ip-address:80 → localhost:8880
ip-address:443 → localhost:4443

server {
    listen       172.31.51.120:80 default;
    server_name  _;

    location / {
        proxy_pass  http://localhost:8880;
   }
}

server {
    listen      172.31.51.120:443;
    server_name _;

    location / {
        proxy_pass  http://localhost:4443;
    }
}

[francislavoie]

@ayoubamine

The 80 and 443 ports are already opened but in use from Nginx, for that, I changed the default Caddy ports from 80 to 8880 and 443 to 4443:

That won’t work. Turn off nginx and make Caddy use ports 80 and 443. Use Caddy to proxy to nginx on other ports if you must still use nginx for other services.

The reason is that Caddy must complete the TLS handshake with the client. If nginx is in the way, that can’t happen.

[ayoubamine]

Yes, I tried to turn off the Nginx and everything worked as expected, but I’m using now Hestia with Ngnix to manage a lot of Web Apps, APIs, and services…, to migrate from Nginx to Caddy will take a lot of time and effort, there is any workaround to solve this?

[francislavoie]

Yes, just make Caddy proxy to those services nginx is serving. It’s just a simple reverse_proxy localhost:8001 for one service, reverse_proxy localhost:8002 for the next, etc. Use whatever port numbers you want. But for Caddy to set up automatic HTTPS, it needs exclusive control of ports 80 and 443.

[ayoubamine]

Thank you, I will go forward with Caddy.