1. Output of caddy version:
2.6.0
2. How I run Caddy:
a. System environment:
Raspberry Pi 4 4gb, RaspberryPi OS lite
docker
b. Command:
sudo docker-compose up -d
c. Service/unit/compose file:
services:
caddy:
image: caddy:latest
container_name: caddy
restart: always
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- ./caddy-config:/config
- ./caddy-data:/data
environment:
- DOMAIN=myexampledomain.de
- EMAIL=mail@myexampledomain.de
- LOG_FILE=/data/access.log
network_mode: host
d. My complete Caddy config:
portainer.{$DOMAIN}:443 {
tls {$EMAIL}
reverse_proxy 192.168.178.3:9233
}
portainer.{$DOMAIN}:8000 {
tls {$EMAIL}
reverse_proxy 192.168.178.3:8111
}
3. The problem I’m having:
Since a while I try now to connect my portainer instance at home to the home server at my mothers house. For that I red, the safest way to use is edge agent.
At home I run portainer behind a caddy reverse proxy on a raspberry pi 4.
Caddy is configured to route portainer.myexampledomain.de:443 to 192.168.178.3:9233 and portainer.myexampledomain.de:8000 to port 192.168.178.3:8111. In Docker I configured, 8111:8000 and 9233:9000 in the portainer compose file on my server.
I opened up port 8000 and 443 in my router (tcp and udp). Also in ufw I allowed port 443 and 8000.
Portainer and caddy for other services is working well so far.
On my mothers server I opened up port 9001 in the router. Ufw is also configured so far.
Now I tried to set up edge agent. I used:
sudo docker run -d \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /var/lib/docker/volumes:/var/lib/docker/volumes \
-v /:/host \
-v portainer_agent_data:/data \
--restart always \
-e EDGE=1 \
-e EDGE_ID=----------------------------------- \
-e EDGE_KEY=-------------------------------------------------------------- \
-e EDGE_INSECURE_POLL=1 \
--name portainer_edge_agent \
portainer/agent:2.15.0
4. Error messages and/or full log output:
I now can see a heartbeat under Environments but if I try to connect it says Failed loading environment Environment is unreachable.
The portainer logs say:
time="2022-09-15T22:18:25+02:00" level=info msg="2022/09/15 22:18:25 [DEBUG] [chisel,monitoring] [endpoint_id: 8] [status: ACTIVE] [status_time_seconds: 7.157875] [message: environment tunnel monitoring]"
time="2022-09-15T22:18:28+02:00" level=info msg="2022/09/15 22:18:28 http: proxy error: dial tcp 127.0.0.1:51018: connect: connection refused"
time="2022-09-15T22:23:30+02:00" level=info msg="2022/09/15 22:23:30 http error: Unable to find the container (err=Error: No such container: 3bfdd889277c8539ed7f13f4df61339c6821c53ad3a5a404730793545eab88c6) (code=404)"
time="2022-09-15T22:23:30+02:00" level=info msg="2022/09/15 22:23:30 http error: Unable to find the container (err=Error: No such container: dae984b1b0af5e2ab7d8a7d4a8f4d04f8d278091412641c87250d3700a5d10dd) (code=404)"
time="2022-09-15T22:34:45+02:00" level=info msg="2022/09/15 22:34:45 [DEBUG] [chisel,monitoring] [endpoint_id: 8] [status: ACTIVE] [status_time_seconds: 3.817940] [message: environment tunnel monitoring]"
time="2022-09-15T22:34:50+02:00" level=info msg="2022/09/15 22:34:50 http: proxy error: dial tcp 127.0.0.1:64692: connect: connection refused"
time="2022-09-15T22:37:04+02:00" level=info msg="2022/09/15 22:37:04 [DEBUG] [chisel,monitoring] [endpoint_id: 8] [status: ACTIVE] [status_time_seconds: 8.822090] [message: environment tunnel monitoring]"
time="2022-09-15T22:37:05+02:00" level=info msg="2022/09/15 22:37:05 http: proxy error: dial tcp 127.0.0.1:55147: connect: connection refused"
time="2022-09-15T22:41:24+02:00" level=info msg="2022/09/15 22:41:24 [DEBUG] [chisel,monitoring] [endpoint_id: 8] [status: REQUIRED] [status_time_seconds: 0.182232] [message: environment tunnel monitoring]"
time="2022-09-15T22:41:34+02:00" level=info msg="2022/09/15 22:41:34 http: proxy error: dial tcp 127.0.0.1:65013: connect: connection refused"
time="2022-09-15T23:12:44+02:00" level=info msg="2022/09/15 23:12:44 [DEBUG] [chisel,monitoring] [endpoint_id: 8] [status: REQUIRED] [status_time_seconds: 1.361693] [message: environment tunnel monitoring]"
time="2022-09-15T23:12:53+02:00" level=info msg="2022/09/15 23:12:53 http: proxy error: dial tcp 127.0.0.1:60140: connect: connection refused"
time="2022-09-15T23:13:34+02:00" level=info msg="2022/09/15 23:13:34 [DEBUG] [chisel,monitoring] [endpoint_id: 8] [status: ACTIVE] [status_time_seconds: 2.269864] [message: environment tunnel monitoring]"
time="2022-09-15T23:13:41+02:00" level=info msg="2022/09/15 23:13:41 http: proxy error: dial tcp 127.0.0.1:61949: connect: connection refused"
time="2022-09-15T23:14:34+02:00" level=info msg="2022/09/15 23:14:34 [DEBUG] [chisel,monitoring] [endpoint_id: 8] [status: ACTIVE] [status_time_seconds: 2.271884] [message: environment tunnel monitoring]"
time="2022-09-15T23:14:41+02:00" level=info msg="2022/09/15 23:14:41 http: proxy error: dial tcp 127.0.0.1:60159: connect: connection refused"
The Agent logs:
2022/09/16 08:59:53 [INFO] [main] [message: Agent running on Docker platform]
2022/09/16 08:59:53 [INFO] [edge] [message: Edge key loaded from options]
2022/09/16 08:59:53 [INFO] [edge,registry] [message: Starting registry credential server]
2022/09/16 08:59:53 [INFO] [http] [server_addr: 172.01.02.03] [server_port: 9001] [use_tls: false] [api_version: 2.15.0] [message: Starting Agent API server]
2022/09/16 09:00:38 client: Connecting to ws://portainer.myexampledomain.de:8000
2022/09/16 09:00:38 client: Connection error: websocket: bad handshake
2022/09/16 09:00:38 client: Give up
2022/09/16 09:01:38 client: Connecting to ws://portainer.myexampledomain.de:8000
2022/09/16 09:01:38 client: Connection error: websocket: bad handshake
2022/09/16 09:01:38 client: Give up
2022/09/16 09:02:38 client: Connecting to ws://portainer.myexampledomain.de:8000
2022/09/16 09:02:38 client: Connection error: websocket: bad handshake
2022/09/16 09:02:38 client: Give up
2022/09/16 09:03:38 client: Connecting to ws://portainer.myexampledomain.de:8000
2022/09/16 09:03:38 client: Connection error: websocket: bad handshake
And for the sake of completeness here also my portainer docker-compose.yml:
version: '3'
networks:
caddy:
external: true
services:
portainer:
image: portainer/portainer-ce:latest
command: -H unix:///var/run/docker.sock
container_name: portainer
restart: unless-stopped
security_opt:
- no-new-privileges:true
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./portainer-data:/data
ports:
- 9233:9000
- 8111:8000
networks:
caddy:
ipv4_address: 192.168.112.8
ipv6_address: 2001:ab12::8
5. What I already tried:
I tried a few different caddy configurations, like:
tcp://portainer.{$DOMAIN}:8000 {
tls {$EMAIL}
reverse_proxy 192.168.178.3:8111
}
or
ws://portainer.{$DOMAIN}:8000 {
tls {$EMAIL}
reverse_proxy ws://192.168.178.3:8111
}
I oppened up ports, disabled ufw, cleared up iptables and a lot of little stuff more…
Im not sure if it is a problem with the agent, portainer or caddy but I hope some of you has an idea!