Can't get pihole reverse_proxy working

Help
1

1. Caddy version (caddy version):

Latest docker

2. How I run Caddy:

Reverse proxy for all internal services on home network

a. System environment:

Ubuntu 20 LTS all Dockerized

b. Command:

sudo docker run -d \
    --name pihole \
    --network=mynetworkname \
    -p 0.0.0.0:53:53/tcp -p 0.0.0.0:53:53/udp \
    -p 6968:80 \
    -p 6967:443 \
    -e TZ="Europe/London" \
    -v "$(pwd)/etc-pihole/:/etc/pihole/" \
    -v "$(pwd)/etc-dnsmasq.d/:/etc/dnsmasq.d/" \
    --dns=127.0.0.1 --dns=1.1.1.1 \
    --restart=unless-stopped \
    pihole/pihole:latest

sudo docker run -d -p 80:80 -p 443:443 \
    --restart always \
    --name caddy \
    --network mynetworkname \
    -v /home/mydir/Caddy/usr/share/caddy:/usr/share/caddy \
    -v /home/mydir/Caddy/data:/data \
    -v /home/mydir/Caddy/config:/config \
    -v /home/mydir/Caddy/caddy:/caddy \
    -v /home/mydir/Caddy/www:/var/www \
    -v /home/mydir/Caddy/logs:/logs \
    -v /home/mydir/Caddy/etc/caddy/Caddyfile:/etc/caddy/Caddyfile \
    caddy/caddy:2.0.0-rc.3

c. Service/unit/compose file:

paste full file contents here

d. My complete Caddyfile or JSON config:

(theheaders) {
    header_up X-Forwarded-Ssl on
    header_up Host {host}
    header_up X-Real-IP {remote}
    header_up X-Forwarded-For {remote}
    header_up X-Forwarded-Port {server_port}
    header_up X-Forwarded-Proto {scheme}
    header_up X-Url-Scheme {scheme}
    header_up X-Forwarded-Host {host}
}
(MANY OTHER SERVICES GO HERE WHICH WORK FINE)

pihole.mydomain.com {
    reverse_proxy pihole:6968 {
      import theheaders //have tried with and without this 
    }
    @notLocal {
      not remote_ip 192.168.1.1/24
    }
    basicauth @notLocal {
      me
xxx    }
    log {
        output file         /logs/access.log
        format single_field common_log
    }
}

3. The problem I’m having:

502 error for just pihole- all other services run fine.
Pihole fully accessible by IP at 192.168.1.2:6968/admin

4. Error messages and/or full log output:

{“level”:“error”,“ts”:1590430795.3831286,“logger”:“http.log.access.log9”,“msg”:“handled request”,“request”:{“method”:“GET”,“uri”:"/admin",“proto”:“HTTP/2.0”,“remote_addr”:“192.168.1.1:64558”,“host”:“pihole.mydomain.com”,“headers”:{“Sec-Fetch-Dest”:[“document”],“Accept-Language”:[“en-US,en;q=0.9”],“Dnt”:[“1”],“Accept”:[“text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.9”],“Sec-Fetch-Site”:[“none”],“Sec-Fetch-User”:["?1"],“Upgrade-Insecure-Requests”:[“1”],“User-Agent”:[“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36”],“Sec-Fetch-Mode”:[“navigate”],“Accept-Encoding”:[“gzip, deflate, br”]},“tls”:{“resumed”:false,“version”:772,“ciphersuite”:4865,“proto”:“h2”,“proto_mutual”:true,“server_name”:“pihole.mydomain.com”}},“common_log”:“192.168.1.1 - - [25/May/2020:18:19:55 +0000] “GET /admin HTTP/2.0” 502 0”,“latency”:0.001302416,“size”:0,“status”:502,“resp_headers”:{“Server”:[“Caddy”]}}

192.168.1.1 - - [25/May/2020:18:16:11 +0000] “GET /api/v3/command HTTP/2.0” 200 926
192.168.1.1 - - [25/May/2020:18:16:11 +0000] “GET /api/v3/queue/status HTTP/2.0” 200 99
192.168.1.1 - - [25/May/2020:18:16:11 +0000] “GET /api/v3/health HTTP/2.0” 200 22
192.168.1.1 - - [25/May/2020:18:16:12 +0000] “GET /api/v3/series HTTP/2.0” 200 216850
192.168.1.1 - - [25/May/2020:18:16:17 +0000] “GET /admin HTTP/2.0” 502 0
192.168.1.1 - - [25/May/2020:18:16:17 +0000] “GET /signalr/reconnect?transport=serverSentEvents&messageId=d-ABE24591-B%2
C7%7CC%2C2&clientProtocol=2.1&apiKey=1dfec57d66db4f0db90a08dd363caa34&connectionToken=LtlK2gieYwU2RHHdFzEo0sA0kRxE1jqg7k
AHbzkxx0ZAza0386t7jwWmSwclBeTSYFjJPvpcWdLHaA%2BNUwZM7lMSxGb3sncNvjXndkcZetY19KQl02Y%2FlPvQnmJ1B4N3OGwbxrNHcRLFINf11UgmTT
%2B55pfFEOCNVR2hwV4s2CRbD%2Bqkzsg8IWRk0iYgpm5%2FmbqNtBB11Zb%2FI2ZKJvEFiMYUza%2FGvyosSYdEC2ZDhN6pPzdQtGQ0M259lZqjqnjksaWw
I4VMtRO8tmT20UyJ0ggYUYxUc2ooKomxQxmegCvcNTQuUQBcVrsbuwNMBtQI&tid=3 HTTP/2.0” 200 109
192.168.1.1 - - [25/May/2020:18:16:17 +0000] “POST /signalr/abort?transport=serverSentEvents&clientProtocol=2.1&apiKey=1
dfec57d66db4f0db90a08dd363caa34&connectionToken=LtlK2gieYwU2RHHdFzEo0sA0kRxE1jqg7kAHbzkxx0ZAza0386t7jwWmSwclBeTSYFjJPvpc
WdLHaA%2BNUwZM7lMSxGb3sncNvjXndkcZetY19KQl02Y%2FlPvQnmJ1B4N3OGwbxrNHcRLFINf11UgmTT%2B55pfFEOCNVR2hwV4s2CRbD%2Bqkzsg8IWRk
0iYgpm5%2FmbqNtBB11Zb%2FI2ZKJvEFiMYUza%2FGvyosSYdEC2ZDhN6pPzdQtGQ0M259lZqjqnjksaWwI4VMtRO8tmT20UyJ0ggYUYxUc2ooKomxQxmegC
vcNTQuUQBcVrsbuwNMBtQI HTTP/2.0” 0 0
192.168.1.1 - - [25/May/2020:18:16:20 +0000] “GET /admin HTTP/2.0” 502 0

5. What I already tried:

Tried lots of different header configs but nothing seems to work properly

6. Links to relevant resources:

2

You’re using a slightly old version of Caddy. Please upgrade to caddy:2.0.0 instead (we now have an official docker image!)

If you get a shell into the Caddy container, can you access the pihole container?

$ docker exec -it caddy /bin/sh
$ ping pihole

This is just to check whether it can see the other container (whether the docker networking is messed up).

3

Yep same network so caddy can reach it just fine. Updating the image had no effect either.

4

Ah, I think I see it. Instead, try pihole:80. Port 6968 is bound on the host, not inside the Docker network. That’s just a port mapping for the host only.

1 Like
5

Thanks for the response! That’s not the case though as the port 80 is remapped in docker, so I can access it on the local network by IP
sudo docker run -d
–name pihole
–network=mynetworkname
-p 0.0.0.0:53:53/tcp -p 0.0.0.0:53:53/udp
-p 6968:80
-p 6967:443
-e TZ=“Europe/London”
-v “$(pwd)/etc-pihole/:/etc/pihole/”
-v “$(pwd)/etc-dnsmasq.d/:/etc/dnsmasq.d/”
–dns=127.0.0.1 --dns=1.1.1.1
–restart=unless-stopped
pihole/pihole:latest

6

Did you try reverse_proxy pihole:80 in your Caddyfile? What happened? What do the logs say?

We need more info to continue debugging the issue.

7

Yes I tried that but I can’t start pihole as other ports use 80 on Docker (i.e., Caddy). So I have to remap the port as they’re on the same network

Running it again now I’m getting this error from Caddy

{"level":"error","ts":1590512424.1101646,"logger":"http.log.error.log9","msg":"dial tcp 172.19.0.10:6968: connect: connection refused","request":{"method":"GET","uri":"/","proto":"HTTP/2.0","remote_addr":"192.168.1.1:49223","host":"pihole.mydomain.com","headers":{"Upgrade-Insecure-Requests":["1"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Sec-Fetch-Site":["none"],"Sec-Fetch-User":["?1"],"Accept-Encoding":["gzip, deflate, br"],"Accept-Language":["en-US,en;q=0.9"],"Dnt":["1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-Dest":["document"]},"tls":{"resumed":false,"version":772,"ciphersuite":4865,"proto":"h2","proto_mutual":true,"server_name":"pihole.mydomain.com"}},"status":502,"err_id":"d3z6nvy5f","err_trace":"reverseproxy.(*Handler).ServeHTTP (reverseproxy.go:362)"}
{"level":"error","ts":1590512424.1102757,"logger":"http.log.access.log9","msg":"handled request","request":{"method":"GET","uri":"/","proto":"HTTP/2.0","remote_addr":"192.168.1.1:49223","host":"pihole.mydomain.com","headers":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Sec-Fetch-Site":["none"],"Sec-Fetch-User":["?1"],"Accept-Encoding":["gzip, deflate, br"],"Accept-Language":["en-US,en;q=0.9"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-Dest":["document"],"Dnt":["1"]},"tls":{"resumed":false,"version":772,"ciphersuite":4865,"proto":"h2","proto_mutual":true,"server_name":"pihole.mydomain.com"}},"common_log":"192.168.1.1 - - [26/May/2020:17:00:24 +0000] \"GET / HTTP/2.0\" 502 0","latency":0.001539901,"size":0,"status":502,"resp_headers":{"Server":["Caddy"]}}
{"level":"error","ts":1590512427.4823966,"logger":"http.log.error.log9","msg":"dial tcp 172.19.0.10:6968: connect: connection refused","request":{"method":"GET","uri":"/admin","proto":"HTTP/2.0","remote_addr":"192.168.1.1:49223","host":"pihole.mydomain.com","headers":{"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-User":["?1"],"Sec-Fetch-Dest":["document"],"Accept-Encoding":["gzip, deflate, br"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Sec-Fetch-Site":["none"],"Dnt":["1"],"Accept-Language":["en-US,en;q=0.9"]},"tls":{"resumed":false,"version":772,"ciphersuite":4865,"proto":"h2","proto_mutual":true,"server_name":"pihole.mydomain.com"}},"status":502,"err_id":"1e0xh2an4","err_trace":"reverseproxy.(*Handler).ServeHTTP (reverseproxy.go:362)"}
{"level":"error","ts":1590512427.4824908,"logger":"http.log.access.log9","msg":"handled request","request":{"method":"GET","uri":"/admin","proto":"HTTP/2.0","remote_addr":"192.168.1.1:49223","host":"pihole.mydomain.com","headers":{"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Sec-Fetch-Site":["none"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-User":["?1"],"Sec-Fetch-Dest":["document"],"Accept-Encoding":["gzip, deflate, br"],"Upgrade-Insecure-Requests":["1"],"Accept-Language":["en-US,en;q=0.9"],"Dnt":["1"]},"tls":{"resumed":false,"version":772,"ciphersuite":4865,"proto":"h2","proto_mutual":true,"server_name":"pihole.mydomain.com"}},"common_log":"192.168.1.1 - - [26/May/2020:17:00:27 +0000] \"GET /admin HTTP/2.0\" 502 0","latency":0.00139448,"size":0,"status":502,"resp_headers":{"Server":["Caddy"]}}
2020/05/26 17:04:56 http: TLS handshake error from 185.234.217.182:42552: no certificate available for '172.19.0.2'

I’m guessing this has something to do with how pihole is using DNS as 192.168.1.1 is just the router and the 172.x.x.x IP is the internal docker IP.

8

You misunderstood me - don’t change your Docker commands, just change your Caddyfile. You can keep the -p 6968:80 port mapping, but in your Caddyfile you need to use pihole:80.

9

HERO!!! That solved it!! Thank you @francislavoie!!

1 Like
10

This topic was automatically closed after 30 days. New replies are no longer allowed.