Help to make a reverse proxy without "tls_insecure_skip_verify"

1. The problem I’m having:

Hello everyone

This is my first post, so I hope I fill out the template correctly.

I have several services configured at home on a Raspi 4 through a Cloudflare tunnel that are working correctly. I’ve decided to set up a bastioned Raspi 3 where Caddy receives tunnel requests and acts as a reverse proxy for Raspi 4.

This is how the flow would work:

I use the Cloudflare tunnel URL → Caddy as a reverse proxy (Raspi 3) → services (Raspi 4).

On Raspi 4, I’ve generated some self-signed certificates that I use for HTTPS services. The CN is 192.168.8.20.

I can only get it to work with the “tls_insecure_skip_verify” directive.

I’ve read the Caddy documentation and think I should establish the trust relationship by copying the public key, but it’s not clear to me who should trust whom or where to store the key.

The ultimate goal is caddy + coraza + crowdsec, which is why the Dockerfile includes these modules. However, I’m testing slowly, which is why docker-compose only includes the caddy part.

Thank you very much in advance, and I hope I’ve explained myself well.

2. Error messages and/or full log output:

caddy  | {"level":"debug","ts":1754002882.6457455,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"192.168.8.20:8443","duration":0.018820018,"request":{"remote_ip":"172.19.0.1","remote_port":"36880","client_ip":"172.19.0.1","proto":"HTTP/1.1","method":"GET","host":"bvw.enunlugarignotoeinefable.com","uri":"/favicon.ico","headers":{"Cf-Warp-Tag-Id":["b10ccddf-d637-4d45-98ad-4e6efcb3ff28"],"Cf-Connecting-Ip":["79.116.55.196"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Sec-Ch-Ua-Platform":["\"Android\""],"Accept-Language":["es-ES,es;q=0.9"],"Sec-Ch-Ua-Mobile":["?1"],"Via":["1.1 Caddy"],"Sec-Ch-Ua":["\"Not)A;Brand\";v=\"8\", \"Chromium\";v=\"138\", \"Google Chrome\";v=\"138\""],"Cf-Ray":["9680b7a04bc09931-MAD"],"X-Forwarded-For":["172.19.0.1"],"Accept":["image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8"],"Cdn-Loop":["cloudflare; loops=1"],"Referer":["https://bvw.enunlugarignotoeinefable.com/"],"X-Forwarded-Host":["bvw.enunlugarignotoeinefable.com"],"Sec-Fetch-Site":["same-origin"],"Cf-Ipcountry":["ES"],"Sec-Fetch-Mode":["no-cors"],"X-Forwarded-Proto":["https"],"Save-Data":["on"],"User-Agent":["Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Mobile Safari/537.36"],"Priority":["u=1, i"],"Sec-Fetch-Dest":["image"],"Accept-Encoding":["gzip, br"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"","server_name":"bvw.enunlugarignotoeinefable.com"}},"error":"tls: failed to verify certificate: x509: certificate relies on legacy Common Name field, use SANs instead"}
caddy  | {"level":"error","ts":1754002882.6465626,"logger":"http.log.error.log1","msg":"tls: failed to verify certificate: x509: certificate relies on legacy Common Name field, use SANs instead","request":{"remote_ip":"172.19.0.1","remote_port":"36880","client_ip":"172.19.0.1","proto":"HTTP/1.1","method":"GET","host":"bvw.enunlugarignotoeinefable.com","uri":"/favicon.ico","headers":{"User-Agent":["Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Mobile Safari/537.36"],"Cf-Warp-Tag-Id":["b10ccddf-d637-4d45-98ad-4e6efcb3ff28"],"Cf-Ipcountry":["ES"],"X-Forwarded-Proto":["https"],"Sec-Ch-Ua-Platform":["\"Android\""],"Priority":["u=1, i"],"Sec-Ch-Ua-Mobile":["?1"],"Sec-Fetch-Site":["same-origin"],"Cdn-Loop":["cloudflare; loops=1"],"Sec-Ch-Ua":["\"Not)A;Brand\";v=\"8\", \"Chromium\";v=\"138\", \"Google Chrome\";v=\"138\""],"Sec-Fetch-Mode":["no-cors"],"X-Forwarded-For":["79.116.55.196"],"Sec-Fetch-Dest":["image"],"Cf-Connecting-Ip":["79.116.55.196"],"Save-Data":["on"],"Accept":["image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8"],"Accept-Language":["es-ES,es;q=0.9"],"Accept-Encoding":["gzip, br"],"Connection":["keep-alive"],"Referer":["https://bvw.enunlugarignotoeinefable.com/"],"X-Forwarded-Host":["bvw.enunlugarignotoeinefable.com"],"Cf-Ray":["9680b7a04bc09931-MAD"],"Cf-Visitor":["{\"scheme\":\"https\"}"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"","server_name":"bvw.enunlugarignotoeinefable.com"}},"duration":0.019425487,"status":502,"err_id":"fvu8nfqgk","err_trace":"reverseproxy.statusError (reverseproxy.go:1390)"}



3. Caddy version:

2.10.0

4. How I installed and ran Caddy:

a. System environment:

Raspberry pi 3B
Raspbian x64 Debian version, 12 (bookworm)

b. Command:

sudo docker compose up --build to see the output 
sudo docker compose up -d when everyting works

c. Service/unit/compose file:

Dockerfile

FROM caddy:2.10.0-builder AS builder

RUN xcaddy build \
    --with github.com/corazawaf/coraza-caddy/v2@latest \
    --with github.com/hslatman/caddy-crowdsec-bouncer/http@main \
    --with github.com/hslatman/caddy-crowdsec-bouncer/appsec@main \
    --with github.com/hslatman/caddy-crowdsec-bouncer/layer4@main

FROM caddy:2.10.0

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

docker-compose.yml

services:
  caddy:
    build:
       context: .
       dockerfile: Dockerfile
    container_name: caddy
    restart: unless-stopped
    cap_add:
      - NET_ADMIN

    ports:
      - 80:80
      - 443:443
#      - 443:443/udp
      - 8123:8123
      - 8443:8443
      - 4080:4080
    volumes:
      - $DOCKERDIR/caddy/log:/var/log
      - $DOCKERDIR/caddy/ruleset:/ruleset
      - $DOCKERDIR/caddy/Caddyfile:/etc/caddy/Caddyfile
      - $DOCKERDIR/caddy/data:/data
      - $DOCKERDIR/caddy/config:/config

d. My complete Caddy config:

{
#auto_https off

# Ponemos el log de debug (crowsec usa los log para detectar anomalías)
  log {
   level DEBUG
  }

#Habilitamos coraza
    order coraza_waf first

#Habilitamos y configuramos crowdsec
  crowdsec {
    api_url http://localhost:8080
    api_key <api_key>
    ticker_interval 15s
    appsec_url http://localhost:7422
    disable_streaming
    #enable_hard_fails
  }
}

#Configuración de reserve proxy (esto incluye la configuración para la inspección de coraza)

#VaultWarden
https://bvw.enunlugarignotoeinefable.com {
        tls internal
#        coraza_waf {
#                load_owasp_crs
#                directives `
#                   Include /ruleset/coraza.conf
#                   Include @owasp_crs/*.conf
#                   SecRuleEngine On
#                `
#          }
          reverse_proxy https://192.168.8.20:8443 {
#                transport http {
#                       tls_insecure_skip_verify
#                }
          }
                log {
                output file /var/log/caddy/vaultwarden.log
                }
}

5. Links to relevant resources:

In this case, Caddy is a client to your Raspi3, so you need to make Caddy trust Raspi3. IN other words, Caddy needs to be able to verify that the certificate installed on Raspi3 is trusted.

Take a look at tls_trust_pool

In your case, you may need something like this:

# ...
          reverse_proxy https://192.168.8.20:8443 {
                transport http {
                       tls_trust_pool file /path/to/cert.pem
                }
          }
# ...

where /path/to/cert.pem is your Raspi3’s self-signed certificate.

2 Likes

Thanks!!

I think I understand :slight_smile:

Just to clarify.

caddy is in the raspi3 and 192.168.8.20 is the raspi 4.
When you say raspi3 is raspi4, don’t you?

On the other hand.
My raspi4 self signed certificates are .crt and .key so I try with
ls_trust_pool file /path/to/cert.crt.

Thanks again. I’ll post the results ASAP :slight_smile:

Hi again

I’ve been trying with no success, the same error persists.

  1. I’ve created a folder where I put the raspi4 certificated and caddy can access it. (/data/caddy/certificates/raspi4/)

  2. I’ve converted my .crt file to .pem file with

openssl x509 -in raspi4-apache-selfsigned.crt -outform PEM -out raspi4-apache-selfsigned.pem
  1. I’ve put on it the raspi4 public key (raspi4-apache-selfsigned.pem) because when I created the certificates I get a .crt and a .key. I don’t have de CA file :frowning:

  2. I’ve tried with tls_trust_pool file path/certificate.pem => same error

  3. I’ve added tls_server _name raspi4IP => same error

  4. I’ve created in the same directory as the .crt file a raspi4-apache-selfsigned.json with a SAN name (like caddy does) => same error

raspi4-apache-selfsigned.json

{
        "sans": [
                "192.168.8.20"
        ],
        "issuer_data": null
}

This is the Caddyfile

#VaultWarden
https://bvw.enunlugarignotoeinefable.com {
        tls internal
#        coraza_waf {
#                load_owasp_crs
#                directives `
#                   Include /ruleset/coraza.conf
#                   Include @owasp_crs/*.conf
#                   SecRuleEngine On
#                `
#          }
          reverse_proxy https://192.168.8.20:8443 {
                transport http {
                        tls_trust_pool file /data/caddy/certificates/raspi4/raspi4-apache-selfsigned.pem
                        tls_server_name 192.168.8.20
#                       tls_insecure_skip_verify
                }
          }
                log {
                output file /var/log/caddy/vaultwarden.log
                }

So, I don’t know how to do it with the files I have, a .crt and a .key.

Thanks in advance.

Yeah, Caddy is the client Caddy talking to Raspi4

If it’s a plain text file containing a base64 encoded string, no need to convert - it’s already PEM.

No key, just the certificate.

No need in your case here.

2 Likes

Finally It’s working, THANKS!!!

There would be a problem with the certificate because I’ve created another one and now works :slight_smile:

Now I have to face the problem that ACME can’t create the certificate, this is the reason I use tls internal.

Final Caddyfile

#VaultWarden
https://bvw.enunlugarignotoeinefable.com {
        tls internal
#        coraza_waf {
#                load_owasp_crs
#                directives `
#                   Include /ruleset/coraza.conf
#                   Include @owasp_crs/*.conf
#                   SecRuleEngine On
#                `
#          }
          reverse_proxy https://192.168.8.20:8443 {
                transport http {
                        tls_trust_pool file /data/caddy/certificates/raspi4/ca.pem
#                       tls_insecure_skip_verify
                }
          }
                log {
                output file /var/log/caddy/vaultwarden.log
                }
}

I thought that may be was related to the raspi4 certiticate, because ACME has crerated another certificates suscesfully in the same machine, but no.

Do I have to ask in another post os can I use this one?

I post the logs error in case we can use this post.

caddy  | {"level":"error","ts":1754048813.6199138,"msg":"challenge failed","identifier":"bvw.enunlugarignotoeinefable.com","challenge_type":"tls-alpn-01","problem":{"type":"urn:ietf:params:acme:error:unauthorized","title":"","detail":"Cannot negotiate ALPN protocol \"acme-tls/1\" for tls-alpn-01 challenge","instance":"","subproblems":null},"stacktrace":"github.com/mholt/acmez/v3.(*Client).pollAuthorization\n\tgithub.com/mholt/acmez/v3@v3.1.2/client.go:557\ngithub.com/mholt/acmez/v3.(*Client).solveChallenges\n\tgithub.com/mholt/acmez/v3@v3.1.2/client.go:378\ngithub.com/mholt/acmez/v3.(*Client).ObtainCertificate\n\tgithub.com/mholt/acmez/v3@v3.1.2/client.go:136\ngithub.com/caddyserver/certmagic.(*ACMEIssuer).doIssue\n\tgithub.com/caddyserver/certmagic@v0.23.0/acmeissuer.go:489\ngithub.com/caddyserver/certmagic.(*ACMEIssuer).Issue\n\tgithub.com/caddyserver/certmagic@v0.23.0/acmeissuer.go:382\ngithub.com/caddyserver/caddy/v2/modules/caddytls.(*ACMEIssuer).Issue\n\tgithub.com/caddyserver/caddy/v2@v2.10.0/modules/caddytls/acmeissuer.go:288\ngithub.com/caddyserver/certmagic.(*Config).obtainCert.func2\n\tgithub.com/caddyserver/certmagic@v0.23.0/config.go:626\ngithub.com/caddyserver/certmagic.doWithRetry\n\tgithub.com/caddyserver/certmagic@v0.23.0/async.go:104\ngithub.com/caddyserver/certmagic.(*Config).obtainCert\n\tgithub.com/caddyserver/certmagic@v0.23.0/config.go:700\ngithub.com/caddyserver/certmagic.(*Config).ObtainCertAsync\n\tgithub.com/caddyserver/certmagic@v0.23.0/config.go:505\ngithub.com/caddyserver/certmagic.(*Config).manageOne.func1\n\tgithub.com/caddyserver/certmagic@v0.23.0/config.go:415\ngithub.com/caddyserver/certmagic.(*jobManager).worker\n\tgithub.com/caddyserver/certmagic@v0.23.0/async.go:73"}
caddy  | {"level":"error","ts":1754048813.6207075,"msg":"validating authorization","identifier":"bvw.enunlugarignotoeinefable.com","problem":{"type":"urn:ietf:params:acme:error:unauthorized","title":"","detail":"Cannot negotiate ALPN protocol \"acme-tls/1\" for tls-alpn-01 challenge","instance":"","subproblems":null},"order":"https://acme-v02.api.letsencrypt.org/acme/order/2566972211/413244008721","attempt":2,"max_attempts":3,"stacktrace":"github.com/mholt/acmez/v3.(*Client).ObtainCertificate\n\tgithub.com/mholt/acmez/v3@v3.1.2/client.go:152\ngithub.com/caddyserver/certmagic.(*ACMEIssuer).doIssue\n\tgithub.com/caddyserver/certmagic@v0.23.0/acmeissuer.go:489\ngithub.com/caddyserver/certmagic.(*ACMEIssuer).Issue\n\tgithub.com/caddyserver/certmagic@v0.23.0/acmeissuer.go:382\ngithub.com/caddyserver/caddy/v2/modules/caddytls.(*ACMEIssuer).Issue\n\tgithub.com/caddyserver/caddy/v2@v2.10.0/modules/caddytls/acmeissuer.go:288\ngithub.com/caddyserver/certmagic.(*Config).obtainCert.func2\n\tgithub.com/caddyserver/certmagic@v0.23.0/config.go:626\ngithub.com/caddyserver/certmagic.doWithRetry\n\tgithub.com/caddyserver/certmagic@v0.23.0/async.go:104\ngithub.com/caddyserver/certmagic.(*Config).obtainCert\n\tgithub.com/caddyserver/certmagic@v0.23.0/config.go:700\ngithub.com/caddyserver/certmagic.(*Config).ObtainCertAsync\n\tgithub.com/caddyserver/certmagic@v0.23.0/config.go:505\ngithub.com/caddyserver/certmagic.(*Config).manageOne.func1\n\tgithub.com/caddyserver/certmagic@v0.23.0/config.go:415\ngithub.com/caddyserver/certmagic.(*jobManager).worker\n\tgithub.com/caddyserver/certmagic@v0.23.0/async.go:73"}
caddy  | {"level":"error","ts":1754048813.621369,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"bvw.enunlugarignotoeinefable.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 403 urn:ietf:params:acme:error:unauthorized - Cannot negotiate ALPN protocol \"acme-tls/1\" for tls-alpn-01 challenge"}

Thanks!!

1 Like

It’s better to separate issues to their own posts.

Let’s Encrypt can’t issue a certificate for your server because it can’t get to it.

For tls-alpn-01 to work, Let’s Encrypt needs to be able to establish connection to your Caddy to verify the challenge.

In your case, though, https://bvw.enunlugarignotoeinefable.com is not your Caddy but Cloudflare.

The challenge is done one the TLS layer, but here it’s Cloudflare, so Caddy eventually errors out with that error message.

If you don’t want to use tls internal or a custom certificate/key for Caddy, you’ll have to switch to DNS-01 challenge instead. Look at the Caddy’s Cloudflare module

Sorry about typos - typing on my phone while walking.

2 Likes

You don’t have to apologize at all, really.

I really appreciate it when someone takes their time to help others, so I’m the one who has to thank you again, a thousand times over :slight_smile:

I’ll read the information carefully and post my progress here :slight_smile:

By the way, for a private service like mine, is a certificate from a recognized CA (Let’s Encrypt) better than a self-signed one (Caddy)?

With a self-signed certificate, you have to tell Cloudflare to skip the TLS verification, and I don’t understand the importance of that.

Best regards and thanks again.

If you’re always going to talk to your Caddy via Cloudflare and never directly then it doesn’t really matter. You can always take that self-signed certificate or Caddy’s internal CA certificate and tell Cloudflare to trust it.

you don’t want to do that, otherwise you’re exposing yourself to a man-in-the-middle attack.

1 Like

Mmmm very interesting. At this time I’m using “not TLS validation” with cloudflare, I will investigate how say Cloudflare to trust in a self-signed certificate.

Update: In a quick search, seem that upload a custom CA certificate it’s not a free plan feature.

I just wanted to comment that I have already gotten the ACME client to work with Cloudflare using the module you told me :smiley:

thank you so much for everything again

1 Like