On demand https with custom apex domain

Help
1

1. Caddy version (caddy version):

2.4.6

2. How I run Caddy:

docker-compose

a. System environment:

b. Command:

caddy run --config /dockerapp/caddy/Caddyfile

c. Service/unit/compose file:

d. My complete Caddyfile or JSON config:

{
  storage redis {
    host {$REDIS_HOST}
  }
  on_demand_tls {
    ask 0.0.0.0:3000/caddy-domain-check
    interval 2m
    burst 5
  }
  debug
}

(SecurityHeaders) {
  header_up X-Real-IP {remote_host}
  header_up X-Forwarded-Proto {scheme}
}


my-site.com, *.my-site.com {

    @notStatic {
        not file
    }
    reverse_proxy @notStatic web:3000
    request_body {
      max_size 100MB
    }
    log {
        output stdout
    }
    tls me@my-site.com {
        dns route53
    }
}


:443, :80 {
    @notStatic {
        not file
    }
    reverse_proxy @notStatic web:3000
    request_body {
      max_size 100MB
    }
    tls {
        on_demand
    }
}

3. The problem I’m having:

Thank you for this awesome software! I’m migrating from nginx specifically for Caddy’s ability to issue TLS certs on demand for custom domains and custom subdomains. It’s working perfectly for on demand custom subdomains (eg. testing.my-client.com or www.my-client.com), but I’m at a loss for how to configure Caddy + DNS settings so that Caddy can handle the apex domain (eg. my-client.com)

For the custom subdomain, I create a CNAME record going from test.my-client.com to client-subdomain.my-site.com. This is working great!

But when it comes to handling the apex domain, domain registrars generally don’t let you create CNAME records for the apex domain. I’ve seen other services handle this by telling their clients to create an A record for the apex domain pointing to an IP (presumably some sort of proxy).

But I don’t fully understand the inner-workings of this method - and I’m wondering if there’s a simpler way to do this with Caddy?

4. Error messages and/or full log output:

5. What I already tried:

I’ve tried finding workarounds for creating CNAMEs for apex domains but it seems like a dead end at this point.

6. Links to relevant resources:

I’ve read every related thread I could find on this forum, including the comprehensive thread at Serving tens of thousands of domains over HTTPS with Caddy , but I’ve been unable to find an answer.

1 Like
2

There isn’t really much you can do in Caddy, this is totally a DNS problem.

There’s a pretty good explanation as to why here:

1 Like
3

Thank you for the reply. I was worried that might be the case. I’ve spent a lot of time researching this and have seen multiple approaches to the problem (A record with proxy; Custom nameservers; www CNAME with a domain registrar configured redirect for apex domain), and am trying to settle on the most suitable.

I don’t like the “Create www CNAME and redirect your apex domain at registrar level” approach (used by Big Cartel) because having different instructions for every domain registrar complicates the solution for clients.

I’d prefer the simplicity of:

  1. create a www subdomain CNAME record pointing to client-subdomain.my-site.com
  2. create A record pointing to proxy IP managed by my-site.com

But I’m confused about what technically should be happening on the proxy server the A record points to. Is this proxy server running Caddy? What is it actually doing to route the A record to the content at client-subdomain.my-site.com?

I apologize for my ignorance on this matter - I’ve dedicated a lot of time to researching this, but I still have some big gaps in my understanding of the problem and how Caddy fits in.

4

It can be, I guess.

If it’s just a single app, then you can proxy to your app backend and making the routing decision based on the Host header, via a lookup table or whatever. Most backend routers should support that.

I’m not sure what else to say, I’m not sure what you’re stuck on :thinking:

1 Like
5

A DNS “A” record is meant to point at an IP. Obviously you don’t want to hand out a specific machine IP to your clients, because when that server fails (and it will, sooner or later), you don’t want to have to recontact all your clients to update their DNS (and everything is down in the meantime).

There are several ways to solve this problem:

  1. A “floating IP” (Digital Ocean terminology, some call it “static IP” or other term) — in case of server failure, the floating IP can be pointed at a different VM/server and there’s no DNS propagation involved.

This does not solve the problem of a data center going offline, so you can ask your clients to create two “A” records pointing to floating IPs in two different data centers. Though the dual “A” record strategy is controversial, modern browsers seem to do a great job of caching both and switching quickly if one returns a bad status or nothing. (This doesn’t apply to non-browser use cases like APIs, but we’re discussing custom domains here, so it doesn’t matter.)

  1. An “Anycast IP”
    I’ve only found this available to mere mortals here: https://support.stackpath.com/hc/en-us/articles/360022801751-Learn-About-Global-Anycast-IP-Addresses which looks great, a static IP that can route to multiple hosts in different regions, following the path of least latency. Unfortunately, in testing this service, I found some dashboard bugs they are still working on, if they get everything fixed and we can successfully test that we can revise our workload at any time and not lose the static IP, and that requests will not be routed to an unhealthy host, we don’t mind paying almost double the VM cost (compared to Digital Ocean/Vultr/Linode/UpCloud etc.), the Anycast IP feature is worth it!

  2. Cloudflare DNS load balancing for high availability
    Your clients can use Cloudflare DNS load balancing for apex domains, with healthchecks so that requests are only routed to healthy hosts. Obviously asking clients to move their DNS hosting is probably not feasible with most SaaS companies, however, CF DNS LBs do, after an awkward (but short, typically 30 - 60 seconds) downtime period waiting for a health check, automatically updates DNS, and wow, their propagation time is pretty amazing with their 250 (or whatever) PoP DNS network (typically within a minute or two globally for 95% of clients in our tests).

Your idea of using the www subdomain will only work if all end users type the “www” into their browser (most won’t). So the only way to handle APEX domains is at the DNS level, you have to tell your clients to set up one or two “A” records pointing to specific IPs.

And while I agree that providing instructions to clients for a dozen popular DNS hosting options, from GoDaddy to Cloudflare, is terrible, there isn’t really a better way that I know of. And when you research companies that allow you to bring your own apex domain, they provide these instructions, it seems like if someone had a better method, you or I would have found it already :sunglasses:.

You mention a proxy server, you could use Caddy on a load balancing server to route requests to multiple upstream hosts, this is a great HA strategy, but you need to provide redundancy (at least two LB servers, I would recommend putting them in two different data centers (regions)).

Apologies for not actually answering your questions directly, this post is mainly about HA, not a solution for providing DNS instructions, but hopefully some of this info will be helpful to someone!

3 Likes
6

By the way, here is the Caddy config we use for custom apex domains, it redirects requests to the www subdomain. (As you can see, to distinguish different tenants, we use a query param ?e=).

(Also, this is a JS function that plugs in some variables and generates the route, I thought it would be more clear this way?)

Maybe the geniuses here can advise on any improvements!

exports.caddyRouteTemplate = ({
  vars: {
    ENVIRONMENT_ID,
    PORT,
    HOST_DOMAIN,
  },
}) => {
  return {
    '@id': ENVIRONMENT_ID,
    handle: [
      {
        handler: 'subroute',
        routes: [
          {
            handle: [
              {
                handler: 'rewrite',
                uri: `?{http.request.uri.query}&e=${ENVIRONMENT_ID}`,
              },
              {
                handler: 'reverse_proxy',
                upstreams: [{ dial: `localhost:${PORT}` }],
              },
            ],
            match: [
              {
                host: [
                  `www.${HOST_DOMAIN}`,
                ],
              },
            ],
          },
          {
            handle: [
              {
                handler: 'static_response',
                headers: {
                  Location: [
                    'https://www.{http.request.host}{http.request.uri}',
                  ],
                },
                status_code: 302,
              },
            ],
            match: [{ host: [HOST_DOMAIN] }],
          },
        ],
      },
    ],
    match: [
      {
        host: [
          HOST_DOMAIN,
          `www.${HOST_DOMAIN}`,
        ],
      },
    ],
    terminal: true,
  }
}
2 Likes
7

Thanks for the replies!

What I was failing to understand last week is that the A record IP address could be a static IP for my app’s load balancer.

My app is a Rails monolith app, and I was under the impression that the server on the IP used for the A record would have to be a seperate microservice. Everything is working perfectly now (Thanks Caddy team!) - although I still have some doubts about whether my architecture design is ideal.

For folks stuck on AWS, AWS Global Accelerator gives you an “Anycast IP” that @Josh_Mellicker refers to in his post.

2 Likes
8

This topic was automatically closed after 30 days. New replies are no longer allowed.