Docker Caddy with CloudFlare Full mode returns Invalid SSL certificate Error code 526

randikaisuru · July 13, 2023, 4:12am

I have developed two e-commerce website using MERN stack.

one is already deployed using docker and it’s running on Digitalocean without any issue.

What I am trying to do is deploy another e-commerce website using docker ( different instance ) on same Digitalocean droplet (same IP address)

I am using

Cloudflare - For DNS configuration DigitalOcean droplet - to deploy my docker instances (frontend and backend) Caddy - For get automatic HTTPS

This is how my docker, dockcer compose and caddy file looks like

caddy file

www.treblemart.com:443 {
    tls <my gmail address>
    root * /srv
    route {
        reverse_proxy /api* products-backend:5100
        try_files {path} {path}/ /index.html
        file_server
    }
}

docker file

first Stage
FROM node:16 AS builder
WORKDIR /usr/src/app
COPY ./package.json ./
COPY ./yarn.lock ./
RUN npm install
COPY . .
ARG BASE_URL
ENV REACT_APP_BASE_URL=${BASE_URL} NODE_ENV=${NODE_ENV}
RUN yarn build
second stage
FROM caddy:2.4.6
ARG CADDYFILE
COPY ${CADDYFILE} /etc/caddy/Caddyfile
COPY --from=builder /usr/src/app/build/ /srv
EXPOSE 80
EXPOSE 443

dockcer compose

version: "3"
   services:
      products-frontend:
         image: products-frontend-prod:${ENV}
         restart: unless-stopped
         ports: "81:80"
               "8001:443"
   networks:
      treble-ecommerce-variant-products-app
   volumes:
      caddy-data:/data
      caddy-config:/config
      /root/treble/uploads/trebleecommercevariantproductsuploads/:/srv/trebleecommercevariantproductsuploads
   products-backend:
      image: products-backend
      restart: unless-stopped
      env_file: ./products-backend/config/${ENV}.env
       ports: "5100:5000"
    networks:
        treble-ecommerce-variant-products-app
           volumes:
                /var/log/services/products-backend/:/var/log/services/products-backend/
               /root/treble/uploads/productsuploads/:/usr/src/app/productsuploads
   networks:
           treble-ecommerce-variant-products-app:
                driver: bridge
   volumes:
       mongo-data:
       driver: local
  caddy-data:
      driver: local
  caddy-config:
      driver: local

Now I am getting Issue Invalid SSL certificate Error code 526. But I can’t figure out why.

my previous website is running on

ports: "80"
       "443".

So I did port binding for my new website as mentioned on docker compose file.

I am trying to deploy my new website on same droplet ( same IP address )

I have configured Cloudflare DNS to my new domain ( same droplet in digitalocean )

Anyone can help me to resolve this Invalid SSL certificate Error code 526 issue?

francislavoie · July 13, 2023, 4:15am

When you use full mode, Cloudflare redirects HTTP requests to HTTPS, which means that ACME issuers can’t reach Caddy because they get served a redirect for HTTP requests instead of reaching Caddy to solve the ACME HTTP challenge.

Instead, use whatever mode allows HTTP traffic as well, or just use the grey-cloud mode (i.e. no proxy, only DNS).

randikaisuru · July 13, 2023, 4:21am

Hi @francislavoie Thanks for quick response.
but when I am changed it to Flexible mode I am getting ERR_TOO_MANY_REDIRECTS error.

And on the frontend logs I can see this

{"level":"error","ts":1689222332.993821,"logger":"tls.obtain","msg":"will retry","error":"[www.treblemart.com] Obtain: [www.treblemart.com] solving challenge: www.treblemart.com: [www.treblemart.com] authorization failed: HTTP 0  -  (ca=https://acme.zerossl.com/v2/DV90)","attempt":1,"retrying_in":60,"elapsed":15.900283337,"max_duration":2592000}

francislavoie · July 13, 2023, 4:40am

I don’t use Cloudflare myself. But I think you need to use Full, but turn off “Always Use HTTPS”.

randikaisuru · July 13, 2023, 6:37am

Hi @francislavoie
After changing to caddy file like this

www.treblemart.com:443 {
    tls internal
    root * /srv
    route {
        reverse_proxy /api* products-backend:5100
        try_files {path} {path}/ /index.html
        file_server
    }
}

tls → internal

I saw in the backend there is no error now. but Invalid SSL certificate Error code 526 issue is still there.

can you please explain after configure tls internal do I need to provide CA from server itself?

Whitestrake · July 13, 2023, 7:05am

Remove tls internal, it will prevent Caddy from ever presenting a publicly trusted HTTPS certificate.

Leave the encryption mode as Full (Strict).

Then, you have two options:

Turn off HTTP->S redirects globally for the Cloudflare site. This option isn’t in the most intuitive place - currently you’ll need to go to SSL/TLS → Edge Certificates.

OR

a. Go to Rules → Configuration Rules and make a new rule. Call it “Disable HTTP->S for well-known URIs” or something similar.
b. Select “Custom filter expression”. Set Field = URI Path, Operator = Starts With, Value = /.well-known/acme-challenge
c. Add the “Automatic HTTPS Rewrites (optional)” rule and DISABLE it.
d. Save and deploy the configuration rule. Now Cloudflare will not redirect HTTP-01 challenges and Caddy can complete them.

randikaisuru · July 19, 2023, 2:33am

Hi @Whitestrake
I am still getting the same error after trying both of your suggestions. Here are logs

Here is the my settings

Did I do something wrorng?

Whitestrake · July 19, 2023, 2:47am

Could you please post those logs as text?

randikaisuru · July 19, 2023, 3:04am

Hi @Whitestrake

treble-ecommerce-variant-products-frontend-admin_1  | {"level":"info","ts":1689733685.3340282,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"admin.treblemart.com","challenge_type":"tls-alpn-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
treble-ecommerce-variant-products-frontend_1        | {"level":"info","ts":1689733685.3860726,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"www.treblemart.com","challenge_type":"http-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
treble-ecommerce-variant-products-frontend-admin_1  | {"level":"error","ts":1689733686.1390095,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"admin.treblemart.com","challenge_type":"tls-alpn-01","problem":{"type":"urn:ietf:params:acme:error:unauthorized","title":"","detail":"Cannot negotiate ALPN protocol \"acme-tls/1\" for tls-alpn-01 challenge","instance":"","subproblems":[]}}
treble-ecommerce-variant-products-frontend-admin_1  | {"level":"error","ts":1689733686.1391003,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"admin.treblemart.com","problem":{"type":"urn:ietf:params:acme:error:unauthorized","title":"","detail":"Cannot negotiate ALPN protocol \"acme-tls/1\" for tls-alpn-01 challenge","instance":"","subproblems":[]},"order":"https://acme-v02.api.letsencrypt.org/acme/order/1213544587/195841339277","attempt":1,"max_attempts":3}
treble-ecommerce-variant-products-backend_1         | 2023-07-19T02:28:06.559Z info : Connected to mongodb...
treble-ecommerce-variant-products-frontend-admin_1  | {"level":"info","ts":1689733687.725417,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"admin.treblemart.com","challenge_type":"http-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
treble-ecommerce-variant-products-frontend_1        | {"level":"error","ts":1689733716.7266731,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"www.treblemart.com","challenge_type":"http-01","problem":{"type":"urn:ietf:params:acme:error:unauthorized","title":"","detail":"2606:4700:3031::ac43:87b7: Invalid response from http://www.treblemart.com/.well-known/acme-challenge/IWEN3u0gcTbgk05DviqcjxtfZo2q_3solYb3pHnusbE: 522","instance":"","subproblems":[]}}
treble-ecommerce-variant-products-frontend_1        | {"level":"error","ts":1689733716.7274108,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"www.treblemart.com","problem":{"type":"urn:ietf:params:acme:error:unauthorized","title":"","detail":"2606:4700:3031::ac43:87b7: Invalid response from http://www.treblemart.com/.well-known/acme-challenge/IWEN3u0gcTbgk05DviqcjxtfZo2q_3solYb3pHnusbE: 522","instance":"","subproblems":[]},"order":"https://acme-v02.api.letsencrypt.org/acme/order/1213544587/195841339387","attempt":1,"max_attempts":3}
treble-ecommerce-variant-products-frontend_1        | {"level":"error","ts":1689733718.0216587,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"www.treblemart.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 429 urn:ietf:params:acme:error:rateLimited - Error creating new order :: too many failed authorizations recently: see https://letsencrypt.org/docs/failed-validation-limit/"}
treble-ecommerce-variant-products-frontend_1        | {"level":"info","ts":1689733718.023132,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["www.treblemart.com"],"ca":"https://acme.zerossl.com/v2/DV90","account":"treblesolution@gmail.com"}
treble-ecommerce-variant-products-frontend_1        | {"level":"info","ts":1689733718.023534,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["www.treblemart.com"],"ca":"https://acme.zerossl.com/v2/DV90","account":"treblesolution@gmail.com"}
treble-ecommerce-variant-products-frontend-admin_1  | {"level":"error","ts":1689733719.1811903,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"admin.treblemart.com","challenge_type":"http-01","problem":{"type":"urn:ietf:params:acme:error:unauthorized","title":"","detail":"2606:4700:3035::6815:1a53: Invalid response from http://admin.treblemart.com/.well-known/acme-challenge/ofU0OaBF7iBwZbIPcIbVCtQZUAdPXLMFknfvjpGVLp0: 522","instance":"","subproblems":[]}}
treble-ecommerce-variant-products-frontend-admin_1  | {"level":"error","ts":1689733719.1819825,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"admin.treblemart.com","problem":{"type":"urn:ietf:params:acme:error:unauthorized","title":"","detail":"2606:4700:3035::6815:1a53: Invalid response from http://admin.treblemart.com/.well-known/acme-challenge/ofU0OaBF7iBwZbIPcIbVCtQZUAdPXLMFknfvjpGVLp0: 522","instance":"","subproblems":[]},"order":"https://acme-v02.api.letsencrypt.org/acme/order/1213544587/195841346117","attempt":2,"max_attempts":3}
treble-ecommerce-variant-products-frontend_1        | {"level":"info","ts":1689733720.621203,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"www.treblemart.com","challenge_type":"http-01","ca":"https://acme.zerossl.com/v2/DV90"}
treble-ecommerce-variant-products-frontend-admin_1  | {"level":"error","ts":1689733721.048058,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"admin.treblemart.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[admin.treblemart.com] solving challenges: admin.treblemart.com: no solvers available for remaining challenges (configured=[http-01 tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/1213544587/195841422947) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
treble-ecommerce-variant-products-frontend-admin_1  | {"level":"info","ts":1689733721.0491822,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["admin.treblemart.com"],"ca":"https://acme.zerossl.com/v2/DV90","account":"treblesolution@gmail.com"}

randikaisuru · July 19, 2023, 3:11am

Hi @Whitestrake
And my folder structure is like this

Main project :
I
I ____> Frontend admin :
I | ____> Caddyfile
I I _____> Dockerfile
I
I ____> Frontend :
I | ____> Caddyfile
I I _____> Dockerfile
I ____> backend :
I I _____> Dockerfile

Whitestrake · July 19, 2023, 3:28am

Your settings in Cloudflare look correct.

In your logs you get repeated 522 status errors:

Status 522 from Cloudflare indicates that their proxy timed out when attempting to connect to your server, so the next thing for you to troubleshoot is port 80 access to Caddy.

randikaisuru · July 19, 2023, 4:00am

Hi @Whitestrake
Just to double check with you is my Caddyfiles correct?

Admin Caddyfile

admin.treblemart.com:443 {
    tls ***@gmail.com
    root * /srv
    route {
        reverse_proxy treble-ecommerce-variant-products-frontend-admin:8080
        reverse_proxy /api* treble-ecommerce-variant-products-backend:5000
        try_files {path} {path}/ /index.html
        file_server
    }
}

Frontend Caddyfile

www.treblemart.com:443 {
    tls ***@gmail.com
    root * /srv
    route {
        reverse_proxy treble-ecommerce-variant-products-frontend:9090
        reverse_proxy /api* treble-ecommerce-variant-products-backend:5000
        try_files {path} {path}/ /index.html
        file_server
    }
}

Whitestrake · July 19, 2023, 4:13am

Correct is a strong assurance for me to be able to give for your use case.

I can tell you that they look like valid Caddyfiles, and that I would expect Caddy to be listening on port 80 and 443 in that configuration.

I might personally make a few minor changes, but only in presentational details, by personal preference.

system · August 18, 2023, 4:13am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.