[Solved] Caddy not proxying requests with tls on_demand

1. Caddy version (caddy version):

v.2.21

2. How I run Caddy:

I have caddy running on a DigitalOcean droplet with this IP address: 159.223.204.176

I have the Caddyfile set up to proxy requests upstream to a blogging platform (to handle custom domains).

a. System environment:

Digital ocean droplet running Ubuntu 18.04 using systemctl

b. Command:

systemctl reload caddy

d. My complete Caddyfile or JSON config:

{
    on_demand_tls {
        ask     https://bear-blog.herokuapp.com/ping
        interval 2m
        burst    5
    }
}

:80 {
    reverse_proxy https://bear-blog.herokuapp.com {
        header_up Host {upstream_hostport}
        header_up X-Forwarded-Host {host}
    }
}

https:// {
    tls {
        on_demand
    }

    reverse_proxy https://bear-blog.herokuapp.com {
        header_up Host {upstream_hostport}
        header_up X-Forwarded-Host {host}
    }

    log {
        output file /var/log/caddy/access.log
    }
}

3. The problem I’m having:

The root :80 is working fine and you can see it running at 159.223.204.176
The issue I’m having is that when I point an A or AAAA record to the droplet it just doesn’t find the server. The ping for the on_demand_tls is also not firing.

The tests are: https://test.justcors.com and https://test2.justcors.com

You can see the records are set correctly here:
dig test.justcors.com AAAA
dig test2.justcors.com A

4. Error messages and/or full log output:

curl: (6) Could not resolve host: test.justcors.com

My log file specified in the Caddyfile is empty/doesn’t exist. There are no noticeable errors.

5. What I already tried:

I have tried quite a few variations on of the Caddyfile the only thing that seems to work is this set up the root :80 to make sure that it’s actually being used (which it seems it is).

6. Links to relevant resources:

I was using this as my guide: Serving tens of thousands of domains over HTTPS with Caddy

I appreciate the help :slight_smile:

That means your DNS records aren’t set; or at least that your DNS resolver isn’t able to find the domain’s A/AAAA records. Please be sure your DNS is set up properly.

That’s not a valid version. Did you mistype this?

Latest is v2.5.1. Make sure you’re on that version.

As Matt said, that’s not a Caddy problem. It’s a DNS problem.

FWIW, this is what I get:

$ curl -v https://test.justcors.com
*   Trying 2604:a880:4:1d0::30e:7000:443...
* Immediate connect fail for 2604:a880:4:1d0::30e:7000: Network is unreachable
* Closing connection 0
curl: (7) Couldn't connect to server


$ curl -v https://test2.justcors.com
*   Trying 159.223.204.176:443...
* Connected to test2.justcors.com (159.223.204.176) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS alert, internal error (592):
* error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
* Closing connection 0
curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error

The first is because I don’t have IPv6 support in my network. Make sure that domain also has an A record as fallback for IPv4.

The second seems to reach your server, but the TLS handshake fails. Are you sure your ask endpoint is properly set up to allow that domain?

1 Like

I appreciate the help. The DNS records are set up correctly I’ve added IPv4 for http://test.justcors.com and changed the Caddy file to not do the tls handshake and it works fine (but is insecure, I’m working on it). Now I’ve readied the tls ask for https://andathird.justcors.com and it’s failing.

{
    on_demand_tls {
        ask https://bear-blog.herokuapp.com/ping
        interval 1m
        burst 5
    }
}

http:// {
    tls {
        on_demand
    }

    reverse_proxy https://bear-blog.herokuapp.com {
        header_up Host {upstream_hostport}
        header_up X-Forwarded-Host {host}
    }

    log {
        output file /var/log/caddy/access.log
    }
}

The ping at https://bear-blog.herokuapp.com/ping should work for the handshake. Am I missing anything here?

Also, I mistyped the version number. It’s v2.2.1

That’s way too old. Upgrade now.

Ah, okay. It’s the one that installs by default on the DigitalOcean Caddy droplet on the DigitalOcean marketplace. I’ve updated the version to v2.5.1.

The issue does seem to be the

on_demand_tls {
        ask https://bear-blog.herokuapp.com/ping
        ...

cause when I remove it it works well. Are there any specific permissions I need to set to allow caddy to check https://bear-blog.herokuapp.com/ping ?

Hmmm, I see that Heroku is returning 301s instead of 200s for some obscure reason. I think if I can figure that out I’ll be gravy. Thanks for the help!

1 Like

Okay, I’ve found the solution finally!

Posting it here just in case anyone has the same issue as me.

In the on_demand_tls ask there should be a forward slash after the url. This seems to be a quirk of Heroku.

Note that curl -I https://bear-blog.herokuapp.com/ping returns 301, where curl -I https://bear-blog.herokuapp.com/ping/ returns 200.

1 Like

That’s kind of my fault, I didn’t know anyone really used that image – or even can find it. The DO vendor portal makes updating images painfully manual and tedious, even though @Mohammed90 has done an excellent job of automating it, the submission process is still manual and requires a review, and it’s really flaky.

So I might just remove it from the DO marketplace.

I don’t think that would be a heroku quirk, that sounds like your app is triggering the redirect. It would depend on the server/framework you’re running on there.

1 Like

This topic was automatically closed after 30 days. New replies are no longer allowed.