Error when using caddy as reverse_proxy when handling https connection

1. Caddy version (caddy version):

➜ caddy version
v2.4.6 h1:HGkGICFGvyrodcqOOclHKfvJC0qTU7vny/7FhYp9hNw=

2. How I run Caddy:

# Using the command belows

➜  caddy run

a. System environment:

➜ cat /etc/*release                                
CentOS Linux release 7.9.2009 (Core)
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

CentOS Linux release 7.9.2009 (Core)
CentOS Linux release 7.9.2009 (Core)

b. Command:

# I think that's all I typed:
➜  caddy run

c. Service/unit/compose file:

# Nope

d. My complete Caddyfile or JSON config:

cdn.myjquery.club {
        #tls /root/tool/CS/https/server.pem /root/tool/CS/https/server.key
        tls /root/tool/CS/https/letsencrypt/cdn.myjquery.club.crt /root/tool/CS/https/letsencrypt/cdn.myjquery.club.key
        reverse_proxy  https://127.0.0.1:8443 {

        }

}

3. The problem I’m having:

I’m trying to use caddy’s reverse_proxy for a https site, the certificate is from letsencrypt by default. But something was wrong and it seems that caddy couldn’t establish ssl connection with the real backend. And sadly backend is enforcing ssl connection so I can’t replace https with http.

4. Error messages and/or full log output:

➜ caddy run  
2021/11/16 12:08:57.995 INFO    using adjacent Caddyfile
2021/11/16 12:08:57.998 WARN    input is not formatted with 'caddy fmt' {"adapter": "caddyfile", "file": "Caddyfile", "line": 2}
2021/11/16 12:08:58.002 INFO    admin   admin endpoint started  {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127.0.0.1:2019"]}
2021/11/16 12:08:58.006 INFO    tls.cache.maintenance   started background certificate maintenance      {"cache": "0xc0000d16c0"}
2021/11/16 12:08:58.020 INFO    http    skipping automatic certificate management because one or more matching certificates are already loaded  {"domain": "cdn.myjquery.club", "server_name": "srv0"}
2021/11/16 12:08:58.021 INFO    http    enabling automatic HTTP->HTTPS redirects        {"server_name": "srv0"}
2021/11/16 12:08:58.021 INFO    http    server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server {"server_name": "srv1", "http_port": 80}
2021/11/16 12:08:58.022 INFO    tls     cleaning storage unit   {"description": "FileStorage:/root/.local/share/caddy"}
2021/11/16 12:08:58.022 INFO    http    enabling automatic TLS certificate management   {"domains": ["chizier.myababa.xyz", "myababa.xyz"]}
2021/11/16 12:08:58.024 INFO    tls     finished cleaning storage units
2021/11/16 12:08:58.025 INFO    autosaved config (load with --resume flag)      {"file": "/root/.config/caddy/autosave.json"}
2021/11/16 12:08:58.025 INFO    serving initial configuration
2021/11/16 12:09:01.975 ERROR   http.log.error  x509: cannot validate certificate for 127.0.0.1 because it doesn't contain any IP SANs  {"request": {"remote_addr": "202.117.43.80:3131", "proto": "HTTP/1.1", "method": "GET", "host": "cdn.myjquery.club", "uri": "/jquery-3.3.2.slim.min.js", "headers": {"Connection": ["Keep-Alive"], "Cache-Control": ["no-cache"], "Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"], "Accept-Language": ["en-US,en;q=0.5"], "Referer": ["http://cdn.myjquery.club/"], "Accept-Encoding": ["gzip, deflate"], "User-Agent": ["Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko"]}, "tls": {"resumed": false, "version": 771, "cipher_suite": 49196, "proto": "", "proto_mutual": true, "server_name": "cdn.myjquery.club"}}, "duration": 0.103971633, "status": 502, "err_id": "4sitxy1hs", "err_trace": "reverseproxy.statusError (reverseproxy.go:886)"}

5. What I already tried:

  1. Change certificate (tried certs from cloudflare) , not working.
  2. Bind 127.0.0.1 with cdn.myjquery.club, and change reverse_proxy directive to https://cdn.myjquery.club:8443 , not working. And the error goes like this:
2021/11/16 11:35:28.304 ERROR   http.log.error  x509: certificate signed by unknown authority   {"request": {"remote_addr": "202.117.43.80:3119", "proto": "HTTP/1.1", "method": "GET", "host": "cdn.myjquery.club", "uri": "/jquery-3.3.2.slim.min.js", "headers": {"Referer": ["http://cdn.myjquery.club/"], "Accept-Encoding": ["gzip, deflate"], "User-Agent": ["Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko"], "Connection": ["Keep-Alive"], "Cache-Control": ["no-cache"], "Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"], "Accept-Language": ["en-US,en;q=0.5"]}, "tls": {"resumed": false, "version": 771, "cipher_suite": 49200, "proto": "", "proto_mutual": true, "server_name": "cdn.myjquery.club"}}, "duration": 0.146226355, "status": 502, "err_id": "2htt0buif", "err_trace": "reverseproxy.statusError (reverseproxy.go:886)"}

I’ve tried import the cert into system’s root certs, not working.

6. Links to relevant resources:

I’ve googled for solutions, but most of them are either docker issues or that their certs are self-signed. Any hint would be kindly appreciated.

See the section in the docs that speaks about proxy headers:

If you’re proxying to an HTTPS backend, you need to make sure the Host header is set to the right thing such that the TLS handshake can succeed.

If the upstream’s certificate is indeed signed by an unknown CA (e.g. self-signed) then you’ll need to either establish trust by telling Caddy how to trust it with the HTTP transport’s tls_trusted_ca_certs option (recommended), or skip cert verification altogether with tls_insecure_skip_verify (very much not recommended, last resort).

2 Likes

Thanks! The problem is solved. Looks like I made a mistake, sorry for my poor eye. I search thoroughly for the documentation but I was focusing on the “tls” directive and I found nothing I want. And now I realize the “tls” directive is something about the downstream connection rather than upstream ones. Thanks for the guide again!

3 Likes

This topic was automatically closed after 30 days. New replies are no longer allowed.