Lookup server misbehaving

1. The problem I’m having:

I am trying to use caddy as a reverse proxy to have https connection with 2 docker container hosting web services. However it seems that I misconfigured something because I can only get empty answer from the server.
And in the caddy logs it seems that caddy is unable to lookup the container.
I already look at a similar post on the forum that didn’t help me.
you can do a curl -vL https://security4media.ebu.ch/admin

2. Error messages and/or full log output:

This is the error I get from caddy

2025/04/25 11:50:04.871	ERROR	http.log.error.log0	dial tcp: lookup headplane on 127.0.0.53:53: server misbehaving	{"request": {"remote_ip": "128.179.129.117", "remote_port": "60404", "proto": "HTTP/2.0", "method": "GET", "host": "security4media.ebu.io", "uri": "/admin", "headers": {"User-Agent": ["curl/8.7.1"], "Accept": ["*/*"]}, "tls": {"resumed": false, "version": 772, "cipher_suite": 4867, "proto": "h2", "server_name": "security4media.ebu.io"}}, "duration": 0.002272897, "status": 502, "err_id": "4spyccdkv", "err_trace": "reverseproxy.statusError (reverseproxy.go:1272)"}

this is the answer to the curl command:

* Host security4media.ebu.io:443 was resolved.
* IPv6: (none)
* IPv4: 13.51.161.23
*   Trying 13.51.161.23:443...
* Connected to security4media.ebu.io (13.51.161.23) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=security4media.ebu.io
*  start date: Apr 25 07:38:20 2025 GMT
*  expire date: Jul 24 07:38:19 2025 GMT
*  subjectAltName: host "security4media.ebu.io" matched cert's "security4media.ebu.io"
*  issuer: C=US; O=Let's Encrypt; CN=E6
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://security4media.ebu.io/admin
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: security4media.ebu.io]
* [HTTP/2] [1] [:path: /admin]
* [HTTP/2] [1] [user-agent: curl/8.7.1]
* [HTTP/2] [1] [accept: */*]
> GET /admin HTTP/2
> Host: security4media.ebu.io
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/2 502 
< alt-svc: h3=":443"; ma=2592000
< server: Caddy
< content-length: 0
< date: Fri, 25 Apr 2025 11:50:04 GMT
< 
* Connection #0 to host security4media.ebu.io left intact

3. Caddy version:

2.6.2

4. How I installed and ran Caddy:

a. System environment:

linux package and systems, aws EC2 instance running Ubuntu 24.04.1 LTS x86_64

b. Command:

caddy start

c. Service/unit/compose file:

The docker compose file for the 2 container

services:
  headplane:
    # I recommend you pin the version to a specific release
    image: ghcr.io/tale/headplane:0.5.10
    container_name: headplane
    restart: unless-stopped
    ports:
      - '3000:3000'
    volumes:
      - './config.yaml:/etc/headplane/config.yaml'
      # This should match headscale.config_path in your config.yaml
      - './headscale-config/config.yaml:/etc/headscale/config.yaml'

      # Headplane stores its data in this directory
      - './headplane-data:/var/lib/headplane'

      # If you are using the Docker integration, mount the Docker socket
      - '/var/run/docker.sock:/var/run/docker.sock:ro'
  headscale:
    image: headscale/headscale:latest
    container_name: headscale
    restart: unless-stopped
    command: serve
    ports:
      - '8080:8080'
      - '127.0.0.1:9090:9090'
    volumes:
      - './headscale-data:/var/lib/headscale'
      - './headscale-config:/etc/headscale'

d. My complete Caddy config:

security4media.ebu.io {
	reverse_proxy * headscale:8080
	reverse_proxy /admin* headplane:3000
	log {
		output file ~/headscale/headscale-data/headscale.log {
			roll_size 100MiB
			roll_keep 5
			roll_keep_for 100d
		}
		format json
		level INFO
	}
}

5. Links to relevant resources:

Okay I played a bit with the configuration and figured out what was off. It solved now.

Please share it for future wanderers

1 Like

If you can share some details about your solution, it might help someone else facing a similar problem.

1 Like