Reverse proxy dns resolver issues

edit: i’ve since hard coded in the resolver which seems to have fixed that error; however now i get the below error; the ip appears to be load balancers sitting in front of caddy; Any thoughts?

{"level":"info","ts":1670259690.3408582,"logger":"tls.obtain","msg":"releasing lock","identifier":"10.64.7.25"}

{"level":"info","ts":1670259860.3330655,"logger":"tls.on_demand","msg":"obtaining new certificate","remote_ip":"10.142.0.197","remote_port":"45951","server_name":"10.64.7.25"}

{"level":"info","ts":1670259860.3358788,"logger":"tls.obtain","msg":"acquiring lock","identifier":"10.64.7.25"}

{"level":"info","ts":1670259860.3361945,"logger":"tls.obtain","msg":"lock acquired","identifier":"10.64.7.25"}

{"level":"info","ts":1670259860.3368263,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"10.64.7.25"}

{"level":"error","ts":1670259860.3379474,"logger":"tls.obtain","msg":"will retry","error":"[10.64.7.25] Obtain: subject does not qualify for a public certificate: 10.64.7.25","attempt":1,"retrying_in":60,"elapsed":0.001737673,"max_duration":2592000}

1. Output of caddy version:

2.6.2

2. How I run Caddy:

via docker using caddy build

a. System environment:

Docker

b. Command:

caddy run

c. Service/unit/compose file:

ROM caddy:2.6.2-builder AS builder

RUN xcaddy build \
    --with github.com/gamalan/caddy-tlsredis 
    
FROM caddy:2.6.2

COPY --from=builder /usr/bin/caddy /usr/bin/caddy


COPY Caddyfile /etc/caddy/


RUN caddy start

d. My complete Caddy config:

{
	on_demand_tls {
		interval 1m
		burst 5
	}

	storage redis {
		host "{$REDIS_HOST}"
		port {$REDIS_PORT}
		address "" // no default, but is build from host+":"+port, if set, then host and port is ignored
		username ""
		password "{$REDIS_PASSWORD}"
		db {$REDIS_DB}
		key_prefix "caddytls"
		value_prefix "caddy-storage-redis"
		timeout 5
		tls_enabled "false"
		tls_insecure "true"
	}
}
http:// {
	respond /health 200
}
https:// {
	reverse_proxy {$PROXY_UPSTREAM} {
	transport http {
	resolvers {$PROXY_RESOLVER}
	}
}
}

3. The problem I’m having:

I’m getting the below error only when having my reverse proxy upstream somewhere the system can’t resolve; and using the transport http resolvers options. If the reverse proxy upstream is set somewhere else and I remove that resolvers line it works. Unfortunately in the production enviro the resolver is required. The resolver is listening on standard port 53 and i’m having trouble understanding from the log error whether it’s having problems with resolving; or connecting after it’s resolved. I’m assuming it’s resolving since it works fine without the resolver step but :shrug: So i’m assuming it’s an error in my configuration. Any insight would be greatly appreciated.

I’ve tried changing resolvers {$PROXY_RESOLVER} to resolvers {$PROXY_RESOLVER}:53 as well with no change.

$PROXY_RESOLVER is an address in the form of something.something.local and the upstream is an http address

4. Error messages and/or full log output:

{"level":"error","ts":1670209405.8280604,"logger":"http.log.error","msg":"dial tcp: lookup php-.1130453904.svc.cluster.local on 10.62.240.10:53: dial tcp 10.62.240.10:0: connect: connection refused","request":{"remote_ip":"10.142.0.111","remote_port":"36603","proto":"HTTP/2.0","method":"POST","host":"php1130453904.staging231121.com","uri":"/query","headers":{"User-Agent":["curl/7.58.0"],"Accept":["*/*"],"Content-Type":["application/json"],"Content-Length":["178"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"php.1130453904.staging231121.com"}},"duration":0.040958068,"status":502,"err_id":"89zp9xfg1","err_trace":"reverseproxy.statusError (reverseproxy.go:1272)"}


6. Links to relevant resources:

Be careful with this; it’s dangerous to use on_demand_tls without an ask endpoint. You’re vulnerable to denial of service attacks (forcing your server to issue certs infinitely, for random garbage domains). See the docs, which explain:

If something tried to make a request without a domain, then Caddy won’t be able to issue a TLS cert for it. No public ACME issuers support certificates for IP addresses, they require domains.

1 Like

This topic was automatically closed after 30 days. New replies are no longer allowed.