Reverse Proxy Error Connection refused

1. The problem I’m having:

I use Caddy 2.7 as a reverse proxy in a project with Api Platform on the backend and Nuxt on the frontend. It works fine in dev but in prod I get the following error:

2. Error messages and/or full log output:

Here is my logs:

{
    "container_name": "app-php-1",
    "image": "app-php",
    "label": {
        "com.docker.compose.oneoff": "False",
        "com.docker.compose.project": "app",
        "com.docker.compose.project.config_files": "/app/compose.yaml,/app/compose.prod.yaml",
        "com.docker.compose.project.working_dir": "/app",
        "com.docker.compose.service": "php",
        "com.docker.compose.version": "2.24.5",
        "org.opencontainers.image.source": "https://github.com/dunglas/frankenphp",
        "org.opencontainers.image.title": "FrankenPHP",
        "org.opencontainers.image.url": "https://frankenphp.dev",
        "org.opencontainers.image.vendor": "Kévin Dunglas",
        "org.opencontainers.image.version": "v1.1.0"
    },
    "message": {
        "duration": 0.000750297,
        "err_trace": "reverseproxy.statusError (reverseproxy.go:1267)",
        "level": "error",
        "logger": "http.log.error.log0",
        "msg": "dial tcp 172.29.0.4:3000: connect: connection refused",
        "request": {
            "client_ip": "123.456.789.135",
            "headers": {
                "Accept": [
                    "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"
                ],
                "Accept-Encoding": [
                    "gzip, deflate, br"
                ],
                "Accept-Language": [
                    "en-US,en;q=0.9"
                ],
                "Connection": [
                    "keep-alive"
                ],
                "Sec-Ch-Ua": [
                    "Google Chrome\";v=\"111\", \"Not(A:Brand\";v=\"8\", \"Chromium\";v=\"111"
                ],
                "Sec-Fetch-Mode": [
                    "navigate"
                ],
                "User-Agent": [
                    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36"
                ]
            },
            "host": "preprod.mywebsite.fr",
            "method": "GET",
            "proto": "HTTP/1.1",
            "remote_ip": "123.456.789.135",
            "remote_port": "11177",
            "tls": {
                "cipher_suite": 4865,
                "proto": "http/1.1",
                "resumed": false,
                "server_name": "preprod.mywebsite.fr",
                "version": 772
            },
            "uri": "/"
        },
        "status": 502,
        "ts": 1708192143.2819047
    },
    "source_type": "docker_logs",
    "stream": "stderr",
    "timestamp": "2024-02-17T17:49:03.281992398Z"
}

3. Caddy version: 2.7

4. How I installed and ran Caddy:

My project is installed here: /app/ with my Caddyfile in /app/api/frackenphp/.

So the backend is in /app/api/
And the frontend in /app/pwa/.

a. System environment:

I use Docker and Docker Compose 2 on Ubuntu 22.04

b. Command:

I launch the application with the following command:

SERVER_NAME=preprod.mywebsite.fr \
APP_SECRET=12345 \
MARIADB_ROOT_PASSWORD=12345 \
CADDY_MERCURE_JWT_SECRET=ChangeThisMercureHubJWTSecretKey \
docker compose -f compose.yaml -f compose.prod.yaml up --wait

c. Service/unit/compose file:

My compose.yaml file :

services:
  php:
    image: ${IMAGES_PREFIX:-}app-php
    depends_on:
      - database
    restart: unless-stopped
    environment:
      PWA_UPSTREAM: pwa:3000
      SERVER_NAME: ${SERVER_NAME:-localhost}, php:80
      MERCURE_PUBLISHER_JWT_KEY: ${CADDY_MERCURE_JWT_SECRET:-!ChangeThisMercureHubJWTSecretKey!}
      MERCURE_SUBSCRIBER_JWT_KEY: ${CADDY_MERCURE_JWT_SECRET:-!ChangeThisMercureHubJWTSecretKey!}
      TRUSTED_PROXIES: ${TRUSTED_PROXIES:-127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16}
      TRUSTED_HOSTS: ^${SERVER_NAME:-example\.com|localhost}|php$$
      DATABASE_URL: mysql://${MARIADB_USER:-mywebsite}:${MARIADB_PASSWORD:-12345}@database:3306/${MARIADB_DATABASE:-mywebsite}?serverVersion=mariadb-${MARIADB_VERSION:-10.11.5}&charset=utf8mb4
      MERCURE_URL: ${CADDY_MERCURE_URL:-http://php/.well-known/mercure}
      MERCURE_PUBLIC_URL: https://${SERVER_NAME:-localhost}/.well-known/mercure
      MERCURE_JWT_SECRET: ${CADDY_MERCURE_JWT_SECRET:-!ChangeThisMercureHubJWTSecretKey!}
    volumes:
      - caddy_data:/data
      - caddy_config:/config
    ports:
      # HTTP
      - target: 80
        published: ${HTTP_PORT:-80}
        protocol: tcp
      # HTTPS
      - target: 443
        published: ${HTTPS_PORT:-443}
        protocol: tcp
      # HTTP/3
      - target: 443
        published: ${HTTP3_PORT:-443}
        protocol: udp

  pwa:
    image: ${IMAGES_PREFIX:-}app-pwa

My compose.prod.yaml file:

# Production environment override
services:
  php:
    build:
      context: ./api
      target: frankenphp_prod
    environment:
      APP_SECRET: ${APP_SECRET}
      MERCURE_PUBLISHER_JWT_KEY: ${CADDY_MERCURE_JWT_SECRET}
      MERCURE_SUBSCRIBER_JWT_KEY: ${CADDY_MERCURE_JWT_SECRET}

  pwa:
    build:
      context: ./pwa
      target: prod

d. My complete Caddy config:

{
	{$CADDY_GLOBAL_OPTIONS}

	frankenphp {
		{$FRANKENPHP_CONFIG}
	}

	# https://caddyserver.com/docs/caddyfile/directives#sorting-algorithm
	order mercure after encode
	order vulcain after reverse_proxy
	order php_server before file_server
}

{$CADDY_EXTRA_CONFIG}

{$SERVER_NAME:localhost} {
	log {
		# Redact the authorization query parameter that can be set by Mercure
		format filter {
			wrap console
			fields {
				uri query {
					replace authorization REDACTED
				}
			}
		}
	}

	root * /app/public
	encode zstd gzip

	mercure {
		# Transport to use (default to Bolt)
		transport_url {$MERCURE_TRANSPORT_URL:bolt:///data/mercure.db}
		# Publisher JWT key
		publisher_jwt {env.MERCURE_PUBLISHER_JWT_KEY} {env.MERCURE_PUBLISHER_JWT_ALG}
		# Subscriber JWT key
		subscriber_jwt {env.MERCURE_SUBSCRIBER_JWT_KEY} {env.MERCURE_SUBSCRIBER_JWT_ALG}
		# Allow anonymous subscribers (double-check that it's what you want)
		anonymous
		# Enable the subscription API (double-check that it's what you want)
		subscriptions
		# Extra directives
		{$MERCURE_EXTRA_DIRECTIVES}
	}

	vulcain

	# Add links to the API docs and to the Mercure Hub if not set explicitly (e.g. the PWA)
	header ?Link `</docs.jsonld>; rel="http://www.w3.org/ns/hydra/core#apiDocumentation", </.well-known/mercure>; rel="mercure"`
	# Disable Topics tracking if not enabled explicitly: https://github.com/jkarlin/topics
	header ?Permissions-Policy "browsing-topics=()"

	# Matches requests for HTML documents, for static files and for Nuxt files,
	# except for known API paths and paths with extensions handled by API Platform
	@pwa expression `(
			header({'Accept': '*text/html*'})
			&& !path(
				'/media*', '/docs*', '/graphql*', '/bundles*', '/media*', '/contexts*', '/_profiler*', '/_wdt*',
				'*.json*', '*.html', '*.csv', '*.yml', '*.yaml', '*.xml'
			)
		)
		|| path('/favicon.ico', '/manifest.json', '/robots.txt', '/_nuxt*', '/sitemap*')`

	# Comment the following line if you don't want Nuxt to catch requests for HTML documents.
	# In this case, they will be handled by the PHP app.
	reverse_proxy @pwa http://{$PWA_UPSTREAM}

	php_server
}

Thank you for your help.

Are you sure your pwa container is running properly? Is it actually listening on port 3000? Check your pwa container’s logs.

When I run docker container ps:

CONTAINER ID   IMAGE                           COMMAND                  CREATED              STATUS                        PORTS                                                                                                                       NAMES
1170910d957c   app-php                         "docker-entrypoint f…"   About a minute ago   Up About a minute (healthy)   0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp, 0.0.0.0:443->443/udp, :::443->443/udp, 2019/tcp   app-php-1
594545fbdb1b   mariadb:10.11.5                 "docker-entrypoint.s…"   About a minute ago   Up About a minute             3306/tcp                                                                                                                    app-database-1
f214e04e8e43   app-pwa                         "docker-entrypoint.s…"   About a minute ago   Up About a minute             3000/tcp                                                                                                                    app-pwa-1
6a17efc01f05   timberio/vector:0.34.0-alpine   "/usr/local/bin/vect…"   4 hours ago          Up 4 hours

And when I run docker logs f214e04e8e43:

Listening on http://127.0.0.1:3000

It should listen to https://preprod.mywebsite.fr, shouldn’t it?

If it’s listening on specifically 127.0.0.1 then it might be rejecting connections from other containers. Make sure it listens for 0.0.0.0 i.e. all IPs.

No, it’s Caddy’s job to handle HTTPS + domains. Your PWA app just needs to be accessible to Caddy over HTTP.

I changed the HOST in my dockerfile from localhost to 0.0.0.0 :

#syntax=docker/dockerfile:1.4

# Versions
FROM node:lts AS node_upstream

# Base stage for dev and build
FROM node_upstream AS base

WORKDIR /srv/app

RUN corepack enable && \
	corepack prepare --activate pnpm@latest && \
	pnpm config -g set store-dir /.pnpm-store

# Development image
FROM base as dev

EXPOSE 3000
ENV PORT 3000
ENV HOST 0.0.0.0

CMD ["sh", "-c", "pnpm install; pnpm dev"]

FROM base AS builder

COPY --link pnpm-lock.yaml ./
RUN pnpm fetch --prod

COPY --link package*.json ./

RUN	pnpm install --frozen-lockfile --offline --prod

COPY --link . .
RUN pnpm run build
RUN pnpm prune

# Production image, copy all the files and run nuxt
FROM node_upstream AS prod

WORKDIR /srv/app

ENV NODE_ENV production

RUN addgroup --system --gid 1001 nodejs; \
	adduser --system --uid 1001 nuxt

COPY --from=builder /srv/app/.output ./.output

USER nuxt

EXPOSE 3000

ENV PORT 3000
ENV HOST 0.0.0.0

CMD [ "node", ".output/server/index.mjs" ]

I have no more errors in my logs, but it still doesn’t work, my browser says: SSL_ERROR_INTERNAL_ERROR_ALERT.

Here are the latest Caddy logs:

2024-02-18 01:17:36.999 [caddy] {"level":"info","logger":"http.acme_client","msg":"validations succeeded; finalizing order","order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/136847483/14613610883","ts":1708215453.5320551}
      
2024-02-18 01:17:40.026 [caddy] {"count":2,"first_url":"https://acme-staging-v02.api.letsencrypt.org/acme/cert/2b99e810a26f21ddff7aca471736ac57e0c8","level":"info","logger":"http.acme_client","msg":"successfully downloaded available certificate chains","ts":1708215457.3290174}
  
2024-02-18 01:17:40.026 [caddy] {"account":"","ca":"https://acme-v02.api.letsencrypt.org/directory","identifiers":["preprod.mywebsite.fr"],"level":"info","logger":"http","msg":"waiting on internal rate limiter","ts":1708215457.3302665}
        
 2024-02-18 01:17:40.026 [caddy] {"account":"","ca":"https://acme-v02.api.letsencrypt.org/directory","identifiers":["preprod.mywebsite.fr"],"level":"info","logger":"http","msg":"done waiting on internal rate limiter","ts":1708215457.3302991}
      
2024-02-18 01:17:40.026 [caddy] {"error":"HTTP 429 urn:ietf:params:acme:error:rateLimited - Error creating new order :: too many certificates (5) already issued for this exact set of domains in the last 168 hours: preprod.mywebsite.fr, retry after 2024-02-19T04:37:24Z: see https://letsencrypt.org/docs/duplicate-certificate-limit/","identifier":"preprod.mywebsite.fr","issuer":"acme-v02.api.letsencrypt.org-directory","level":"error","logger":"tls.obtain","msg":"could not get certificate from issuer","ts":1708215458.0976484}
           
2024-02-18 01:17:40.026 [caddy] {"level":"warn","logger":"http","msg":"missing email address for ZeroSSL; it is strongly recommended to set one for next time","ts":1708215458.097868}
        
2024-02-18 01:17:42.811 [caddy] {"error":"account pre-registration callback: failed getting EAB credentials: HTTP 200: failed_creating_eab_account (code 2902)","identifier":"preprod.mywebsite.fr","issuer":"acme.zerossl.com-v2-DV90","level":"error","logger":"tls.obtain","msg":"could not get certificate from issuer","ts":1708215458.9978538}

2024-02-18 01:17:42.811 [caddy] {"attempt":5,"elapsed":627.978305607,"error":"[preprod.mywebsite.fr] Obtain: account pre-registration callback: failed getting EAB credentials: HTTP 200: failed_creating_eab_account (code 2902)","level":"error","logger":"tls.obtain","max_duration":2592000,"msg":"will retry","retrying_in":600,"ts":1708215458.9979155}

Do you know how I can reuse my certificate?

I’ve got it! While we wait for the week to pass, I’ve added this line

acme_ca https://acme-staging-v02.api.letsencrypt.org/directory

at the top of my Caddyfile, in the first block.

Thanks a lot!

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.