Caddy adress regex match

1. Caddy version (caddy version):

2.4.3-alpine

2. How I run Caddy:

Our customers can CNAME their own domain names to our Caddy webserver. We enabled on-demand TLS with Redis as storage backend. Our infrastructure runs on AWS EC2 with Docker.

a. System environment:

Docker 20.10.7 on AWS EC2 (Ubuntu 20.04) with a Network Load Balancer

b. Command:

X

c. Dockerfile:

###################
#  CADDY BUILDER  #
###################
FROM caddy:builder-alpine AS caddy-builder

RUN xcaddy build \
    --with github.com/gamalan/caddy-tlsredis \
    --with github.com/caddy-dns/cloudflare


###########
#  CADDY  #
###########
FROM caddy:2-alpine AS caddy
LABEL maintainer="Sherin Bloemendaal <sherin@maglr.com>"

COPY caddy/Caddyfile /etc/caddy/Caddyfile
COPY --from=caddy-builder /usr/bin/caddy /usr/bin/caddy

WORKDIR /app

VOLUME /app
EXPOSE 80
EXPOSE 443

d. My complete Caddyfile or JSON config:

{
        debug
        email sherin@maglr.com
        auto_https disable_redirects
        on_demand_tls {
                # Testing
                ask https://httpstat.us/200
        }
        admin caddy:8080 {
                origins 172.31.0.0/16:8080
        }
        storage redis {
                host {$CADDY_CLUSTERING_REDIS_HOST}
                port {$CADDY_CLUSTERING_REDIS_PORT}
                db {$CADDY_CLUSTERING_REDIS_DB}
                tls_enabled {$CADDY_CLUSTERING_REDIS_TLS}
        }
}

http:// {
        @health_path path /health
        @others not path /health
        respond @health_path "OK" 200
        redir @others https://{host}{uri}
}

https:// {
        root * /app/public
        php_fastcgi php:9000
        encode zstd gzip
        file_server
        tls sherin@maglr.com {
                on_demand
        }
}

3. The problem I’m having:

When i visit a ip adress on https, it does not use the internal tls anymore but instead it requests a letsencrypt certificate (wich fails) and it keeps retrying. Is there anyway to exclude regex matching hosts or something like that or maybe i can let them fallback to the internal certificate if on-demand fails? Or maybe its possible to use a matcher inside the address part?

Why we use Caddy: We currently have arround 2000 domains that we host, we looked into Cloudflare for SaaS but their Enterprise licence starts at $3000 per month wich is really expensive (higher than our total hosting costs at AWS). We also looked into Fastly, they offered $20 dollar per domain per month, wich is also extremely expensive when having 2000 domains. So we decided to host our own service and then Caddy said hello. We love the idea and its a really nice concept and specially the on-demand TLS feature is very great.

Also i think i misunderstood the term “Automatic HTTPS” and “On-demand TLS”, they’re not the same or am i wrong? Since “Automatic HTTPS” is enabled by default but “On-demand TLS” isn’t.

Anyways, help would be really appreciated! I am looking for some regex address matching.
For example: directly visiting our ip AWS EC2 ip adress should not request letsencrypt or zerossl but return tls internal instead.

4. Error messages and/or full log output:

X

5. What I already tried:

Request matcher:

  • header_regexp ip_regex Host ^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$. The regex should match ipv4 adresses. But this method doesnt work, i’ve read something that the Host header is exceptional when using the header/header_regexp matcher.
  • Tried expression but it seems it does not support regex like checking.
  • Create another siteblock for 127.0.0.1 as test:
{
        ... same as above
}

https://127.0.0.1 {
    tls internal
    respond "OK" 200
    handle {
        abort
    }
}

http:// {
        @health_path path /health
        @others not path /health
        respond @health_path "OK" 200 {
            close
        }
        redir @others https://{host}{uri}
}

https:// {
        root * /app/docs
        php_fastcgi php:9000
        encode zstd gzip
        file_server
        tls sherin@maglr.com {
                on_demand
        }
}

6. Links to relevant resources:

X

1 Like

Hmm, good question. I need to spend more time to investigate but I’m going to assume it’s because there’s a catch-all TLS automation policy, which of course includes IP addresses. (caddy adapt should reveal that.)

it does not use the internal tls anymore but instead it requests a letsencrypt certificate (wich fails) and it keeps retrying.

The proper way to do this is to use your ask endpoint, rather than abuse it. :wink: (with https://httpstat.us/200). Your ask endpoint should return a non-200 status for any subject names you don’t want to allow to get a certificate.

Also i think i misunderstood the term “Automatic HTTPS” and “On-demand TLS”, they’re not the same or am i wrong? Since “Automatic HTTPS” is enabled by default but “On-demand TLS” isn’t.

Automatic HTTPS is using HTTPS by default. On-demand TLS defers certificate management to handshake-time, and sounds like the right feature for your use case.

Cloudflare will charge thousands per month, but Caddy is free to use. I strongly recommend that companies at least sponsor the project so I can keep working on it full time, but also as a good look for your company and your customers. Most companies your size sponsor at the $499 or $999 / mo tier to support their critical infrastructure. I can also offer some help in private (email or whatever) as well to sponsors. Please consider it! :slight_smile:

1 Like

This is fine for testing, but make sure to actually use an ask endpoint backed by a database of trusted domains when you start using this for real. This is an avenue for abuse, because a bad actor could point a wildcard domain to your server then make requests with different subdomains of that domain until infinity, making your server continually issue certificates.

You can simplify this a bit:

http:// {
	handle /health {
		respond "OK" 200
	}
	handle {
		redir https://{host}{uri}
	}
}

It’s redundant to specify the email here, because you already have it in global options. You can omit it here.

This is the job of the ask endpoint. Only respond with a 200 response for domains you actually support. Respond with a 400 response (or any other status) to reject it.

Falling back to the internal CA is not a good idea; this would add a lot of complexity. That’s not something Caddy supports (via the Caddyfile anyways).

You can make a site block that uses tls internal though, for the specific domains you want to use the internal CA.

Yes, they are different. On-Demand TLS is a subset of the Automatic HTTPS features Caddy provides.

Automatic HTTPS is essentially the broad term that covers all the automation Caddy provides for issuing certificates, enabling automatic HTTP->HTTPS redirects, etc. Typically, this is via managing certificates for domains Caddy knows about via the configuration (i.e. specifying the domain name as the site address, instead of using https:// as the site address).

On-Demand TLS is specifically issuing certificates that Caddy is not managing via configuration, on the fly, when a request comes in with that domain in SNI during the TLS handshake. The TLS handshake involves multiple roundtrips between the client and server to establish a connection. Caddy takes the opportunity here to try to quickly get a certificate if the ask option or the rate limits allow for it, before actually responding to the incoming request if the issuance + handshake complete. From then on, a certificate for that domain is will be managed (automatically renewed when necessary, etc).

2 Likes

Thank you for the suggestions! All clear.

The ask endpoint was just for testing, but i will create an actual endpoint in our api so we have a better view of how it would work on our production environment. Thanks!

Thanks!

Currently i am looking into localhost https when using Docker, can’t find a lot about this in the docs (caddy trust / caddy untrust) but those commands are inside the container and not the host system. I don’t know if its possible but i will lets you guys know.

Thanks so far anyways!

1 Like

Hi Matt,

Thanks everything is clear!

We love your product so far. We are currently still testing out Caddy if it fits all of our needs. We think it can be for our production environment, but we will actually test some customers this week. We are very excited!

But ofcourse, if we decide to deploy Caddy to our production environment, we would love to sponsor your project!

Kind Regards,
Sherin

2 Likes

The caddy trust command attempts to install the root CA cert into the system’s trust store. When running in Docker, it’s not possible to run directly, because of Docker’s isolation.

You should have set up a volume to persist /data from the Caddy container. In there, you can find the root CA cert, I think in /data/pki/authorities/local/root.crt (possibly with a caddy dir near the start – I need to double check that). The root certificate has a long lifetime, so you can install it in the trust stores of whatever systems/browsers that should trust your server. Google is your friend here.

You may also download a build of Caddy to run on the host machine, and run the caddy trust command like this:

sudo HOME=/path/to/data/volume caddy trust

Overriding the HOME directory is important to correctly point Caddy to the right storage location, and sudo is necessary to gain write permissions to add the root certificate where necessary.

1 Like

Thanks! I will look into this!

We’ve speaked internally about sponsoring. How can we contact you about this?

1 Like

Either via:

Or via:

You can message me here if you have questions! But yeah the first link Francis posted is recommended. (GitHub)

Ok thanks!

I have an another issue, when i visit caddy with a http:// url, it returns: Client sent an HTTP request to an HTTPS server. but i have a handle { redir... } for http:// requests like you can see in my config. When i visit a domain pointed to caddy (http://proxy-ssl.maglr.com) and i go to the /health path it says the same.
Here is my current config, am i missing something?

        email sherin@maglr.com
        auto_https disable_redirects
        on_demand_tls {
                ask {$CADDY_ON_DEMAND_ASK_URL}
        }
        admin caddy:8080 {
                origins {$CADDY_ADMIN_ALLOWED_ORIGINS}
        }
        storage redis {
                host {$CADDY_CLUSTERING_REDIS_HOST}
                port {$CADDY_CLUSTERING_REDIS_PORT}
                db {$CADDY_CLUSTERING_REDIS_DB}
                tls_enabled {$CADDY_CLUSTERING_REDIS_TLS}
        }
}

http:// {
        handle /health {
            respond "OK" 200
        }
        handle {
            redir https://{host}{uri}
        }
}

*.maglr.localhost, *.localhost, https://127.0.0.1 {
    root * /app/docs
    php_fastcgi php:9000
    encode zstd gzip
    file_server
    tls internal
}

https:// {
        root * /app/docs
        php_fastcgi php:9000
        encode zstd gzip
        file_server
        tls {
            on_demand
        }
}

What does it look like when you make the request with curl -v?

Sorry for my late reaction, we’ve been busy with security preparation/strengthening.

sherin@sherin:~/projects/maglr/frontend$ curl -v http://proxy-ssl.maglr.com/
*   Trying xxx.xxx.xxx.x:80...
* TCP_NODELAY set
* Connected to proxy-ssl.maglr.com (xxx.xxx.xxx.64) port 80 (#0)
> GET / HTTP/1.1
> Host: proxy-ssl.maglr.com
> User-Agent: curl/7.68.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
* HTTP 1.0, assume close after body
< HTTP/1.0 400 Bad Request
< 
Client sent an HTTP request to an HTTPS server.
* Closing connection 0

I don’t understand how that could happen.

What’s your full Caddyfile at this point?

Are you accidentally mapping port 80 to 443 in some networking config somewhere?

1 Like

My bad, when you said mapping ports i instantly knew it was the AWS target group :slight_smile:

Today we’re continuing testing so i will let you know how it went!

Thanks so far!

1 Like

Everything works perfectly! So awesome! We are really happy.
One question about storage, we use redis as storage backend but how do i add a custom certificate to it?
I cant find anything in the docs about it. Or should i create a separate thread for this?

This is only possible via JSON using the load_storage module:

Otherwise, load it from file using the tls directive:

This topic was automatically closed after 30 days. New replies are no longer allowed.