What is the correct way to configure Caddy behind a load balancer

1. Caddy version (caddy version):

v2.5.1

2. How I run Caddy:

On docker containers hosted on AWS ECS (Fargate)

a. System environment:

linux

d. My complete Caddyfile or JSON config:

{
    # Debug
    {$DEBUG}
    # HTTP/3 support
    servers {
        protocol {
            experimental_http3
        }
    }
}

{$SERVER_NAME}

log

route {
    root * /srv/app/public
    mercure {
        # Transport to use (default to Bolt)
        transport_url {$MERCURE_TRANSPORT_URL:bolt:///data/mercure.db}
        # Publisher JWT key
        publisher_jwt {env.MERCURE_PUBLISHER_JWT_KEY} {env.MERCURE_PUBLISHER_JWT_ALG}
        # Subscriber JWT key
        subscriber_jwt {env.MERCURE_SUBSCRIBER_JWT_KEY} {env.MERCURE_SUBSCRIBER_JWT_ALG}
        # Allow anonymous subscribers (double-check that it's what you want)
        anonymous
        # Enable the subscription API (double-check that it's what you want)
        subscriptions
        # Extra directives
        {$MERCURE_EXTRA_DIRECTIVES}
    }
    vulcain
    push
    php_fastcgi unix//var/run/php/php-fpm.sock
    encode zstd gzip
    file_server
}

3. The problem I’m having:

When I run one caddy server, and point the DNS to the ip address of the server, everything works and https is working properly. However, when I point the DNS to a load balancer, and put my server behind the load balancer, Caddy is no longer able to generate my certificate. Ideally, I would like to have a custom certificate tied to my load balancer, and disable Caddy from generating its own certificates, but to keep it accepting https requests.

5. What I already tried:

with my DNS pointing to a load balancer and my load balancer pointing to my server, I cannot access it with https enabled. I tried disabling https on Caddy, which made it so that I can access my site on http (not ideal, I want htttps).
I presume that If I try attaching a certificate to my load balancer, that the site won’t open on https, because I have Caddy only accepting http (since by enabling https, it tries to generate a tls and fails, and the site becomes unaccessible).

The load balancer needs to not terminate TLS, if you want Caddy to automate certificate issuance, etc.

This would mean running the load balancer in TCP mode, not in HTTP mode.

1 Like

I have since come across the auto_https which would work for me. Only problem is that in some documentation, it specifies an attribute “skip_certificates”: [""]
But in others it does not. Is this attribute still supported in the CADDYFILE? if so I can specify my domain name for it to not create a certificate for, while still keeping the local environment with automatic tls creation

To rephrase my last reply, is there any way to:
Disable Caddy creating its own certificate
Keep https on Caddy
Make Caddy accept connections via https with the certificate being on the load balancer?

Caddy needs a certificate for HTTPS. Which one would you have it use?

You could have Caddy generate its own with tls internal, but then there’s no trust between your load balancer and Caddy, unless you can configure the load balancer to trust Caddy’s root CA cert.

Trust is important for security, otherwise anyone/anything that can get itself in between your load balancer and Caddy can man-in-the-middle the connection, making HTTPS pointless.

To answer your question, I have a certificate created from AWS for my domain, and so I’d need a way to configure that certificate with Caddy, instead of it creating its own.

Then use the tls directive, which lets you specify the cert/key pair to use.

Thanks for your patience! If I configure this, do I still need to put the cert on the load balancer or is it “imported” into the Caddy server?

Well, if you have the cert/key that AWS gives you, you’d need to copy it to your server running Caddy, then configure Caddy to load that cert/key pair.

Great, thanks!
Is there anything I need to watch out for that’s not mentioned in the docs you linked?

Another approach:

Put your Fargate service in a private subnet in your VPC, and point your load balancer at that. Your LB will terminate SSL, and forward the request to Caddy in plaintext over the internal AWS network. Traffic internal to AWS is already encrypted at the network layer.

This way, your Fargate service itself won’t be exposed via public IP address, and you won’t need the complexity of adding a cert on Caddy (auto_https off). Incoming traffic will flow through the LB only.

This reddit thread offers some good food for thought.

1 Like

Hey!
I ended up solving it but did not have time to post about it. I essentially ended up doing what you said. For those coming to this thread with a similar issue, here is the breakdown:

  • Disable https on Caddy by specyfing the server name as http://servername in the caddyfile
  • point the DNS to the load balancer
  • configure the load balancer to listen on https. Make sure to create a certificate and put it on the load balancer
  • point the load balancer to a target group which communicates with the load balancer on http port 80.

This way, any users accessing your site will be using https (via the load balancer) and the load balancer will talk internally with Caddy on http (its secure since its internal to your network).

I’m going to mark your answer as correct since you took the time to type it out. Thanks for your efforts!

3 Likes

This topic was automatically closed after 30 days. New replies are no longer allowed.