Caddy with Nginx

1. The problem I’m having:

I have setup 2 container groups with Caddy as a side car container and they’re both running, in the frontend. The problem I have is that my requests to my backend are being sent through IP address and not using FQDN even after specifying in the proxy_pass.
I have tried curling the backend and realized the Fqdn works but using the IP doesn’t really work, it doesnt resolve.

2. Error messages and/or full log output:

2024/03/22 12:41:00 [error] 184#184: *1 SSL_do_handshake() failed (SSL: error:0A000438:SSL routines::tlsv1 alert internal error:SSL alert number 80) while SSL handshaking to upstream, client: 127.0.0.1, server: madiba-app.germanywestcentral.azurecontainer.io, request: "POST /auth/register HTTP/1.1", upstream: "https://4.182.8.54:443/auth/register", host: "madiba-app.germanywestcentral.azurecontainer.io", referrer: "https://madiba-app.germanywestcentral.azurecontainer.io/register"
127.0.0.1 - - [22/Mar/2024:12:41:00 +0000] "POST /auth/register HTTP/1.1" 502 497 "https://madiba-app.germanywestcentral.azurecontainer.io/register" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" "10.92.0.11"

And a curl to the backend looks like this

mtsumi@DESKTOP-RCE5MCO:~$ curl -vk https://4.182.8.54:443/auth/register
*   Trying 4.182.8.54:443...
* Connected to 4.182.8.54 (4.182.8.54) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Unknown (21):
* TLSv1.3 (IN), TLS alert, internal error (592):
* error:0A000438:SSL routines::tlsv1 alert internal error
* Closing connection 0
curl: (35) error:0A000438:SSL routines::tlsv1 alert internal error
mtsumi@DESKTOP-5MCO:~$ curl -vk https://madibapp.germanywestcentral.azurecontainer.io/auth/register
*   Trying 4.182.8.54:443...
* Connected to madibapp.germanywestcentral.azurecontainer.io (4.182.8.54) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=madibapp.germanywestcentral.azurecontainer.io
*  start date: Mar 18 21:29:36 2024 GMT
*  expire date: Jun 16 21:29:35 2024 GMT
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* Using Stream ID: 1 (easy handle 0x56113ca72eb0)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /auth/register HTTP/2
> Host: madibapp.germanywestcentral.azurecontainer.io
> user-agent: curl/7.81.0
> accept: */*
>
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
< HTTP/2 405
< allow: POST
< alt-svc: h3=":443"; ma=2592000
< content-type: application/json
< date: Fri, 22 Mar 2024 13:19:50 GMT
< server: Caddy
< server: uvicorn
< content-length: 31
<
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection #0 to host madibapp.germanywestcentral.azurecontainer.io left intact
{"detail":"Method Not Allowed"}

3. Caddy version:

I am using Caddy from docker Caddy2.6

4. How I installed and ran Caddy:

I am using Caddy image from docker, Caddy 2.6

a. System environment:

Docker

b. Command:

N/A

d. My complete Caddy config:

Frontend

madiba-app.germanywestcentral.azurecontainer.io {
    reverse_proxy http://localhost:3000
}

Backend

madibapp.germanywestcentral.azurecontainer.io {
    reverse_proxy http://localhost:8000
}

Below is my nginx config

user  nginx;
worker_processes  1;
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
  worker_connections  1024;
}

http {
  include       /etc/nginx/mime.types;
  default_type  application/octet-stream;
  client_max_body_size 100M;
  log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';
  access_log  /var/log/nginx/access.log  main;
  sendfile        on;
  keepalive_timeout  65;

  # SSL Configuration
  server {
    listen 3000;
    server_name madiba-app.germanywestcentral.azurecontainer.io;

    add_header Access-Control-Allow-Origin *;

    location / {
        root   /app;
        index  index.html;
        try_files $uri $uri/ /index.html;
    }

    resolver 8.8.8.8;  # Use Google DNS resolver
    set $api_backend https://madibapp.germanywestcentral.azurecontainer.io;
    set $auth_backend https://madibapp.germanywestcentral.azurecontainer.io;

    location /api/ {
        proxy_pass $api_backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 800s;
        proxy_read_timeout 800s;
    }

    location /auth/ {
        proxy_pass $auth_backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 300s;
        proxy_read_timeout 300s;
    }

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
  }
}

I don’t really understand the setup here. Why are you using Nginx? Caddy can probably replace it.

If Caddy is serving TLS, then your need to connect to Caddy with TLS-SNI set, which means you need to use the domain when connecting. Caddy needs to know which domain to use during certificate selection. If there’s a mismatch with the domain and the certificate chosen, then the connection can’t be trusted, so it’s necessary.

You could configure Caddy to issue IP certs using its internal CA, but that also means you’d need to set up trust by copying Caddy’s root CA cert to any connecting client. Probably not what you want.

Hi Francislavoie,

I managed to solve it by using Caddy only.
I realized nginx brought some complexities, but serving on local host and using only Caddy to reverse proxy solved the issue I had.

1 Like