1. Caddy version (caddy version):
v2.5.1 h1:bAWwslD1jNeCzDa+jDCNwb8M3UJ2tPa8UZFFzPVmGKs=
2. How I run Caddy:
I have an ubuntu-server installation (ip addr: 192.168.1.30) running docker compose which is running Caddy with version v2.5.1 h1:bAWwslD1jNeCzDa+jDCNwb8M3UJ2tPa8UZFFzPVmGKs=.
I will refer to the above Caddy as LocalCaddy.
I also run another Caddy instance on a VPS running a near identical docker-compose. This Caddy I will call CloudCaddy and runs version: v2.5.1 h1:bAWwslD1jNeCzDa+jDCNwb8M3UJ2tPa8UZFFzPVmGKs=.
There is a Wireguard VPN connection between CloudCaddy machine (wg server: 10.0.0.1) and LocalCaddy machine (wg client: 10.0.0.2).
My OPNsense router on the network is set to route all DNS requests to veritablevalor.com
a. System environment:
LocalCaddy
OS (cat /etc/os-release)
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
CloudCaddy
OS (cat /etc/os-release)
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
b. Command:
sudo docker compose caddy up -d
c. Service/unit/compose file:
LocalCaddy docker-compose.yml
version: '3.7'
services:
caddy:
container_name: caddy
image: caddy
restart: unless-stopped
ports:
- 80:80
- 443:443
volumes:
- ./caddyfile/Caddyfile:/etc/caddy/Caddyfile
- caddy_data:/data
- caddy_config:/config
volumes:
caddy_data:
driver_opts:
type: "nfs"
o: "addr=192.168.1.10,nolock,soft,rw"
device: "<Path on NAS>"
caddy_config:
driver_opts:
type: "nfs"
o: "addr=192.168.1.10,nolock,soft,rw"
device: "<Path on NAS>"
CloudCaddy docker-compose.yml
version: '3.7'
services:
caddy:
container_name: caddy
image: caddy
restart: unless-stopped
ports:
- 80:80
- 443:443
volumes:
- ./html:/var/www/html
- ./caddy:/etc/caddy
d. My complete Caddyfile or JSON config:
LocalCaddy Caddyfile
veritablevalor.com {
reverse_proxy 192.168.1.30:80
}
gitea.veritablevalor.com {
reverse_proxy 192.168.1.30:3003
}
firefly.veritablevalor.com {
reverse_proxy 192.168.1.30:8080
}
planka.veritablevalor.com {
reverse_proxy 192.168.1.30:3005
}
nextcloud.veritablevalor.com {
reverse_proxy 192.168.1.30:444
}
jellyfin.veritablevalor.com {
reverse_proxy 192.168.1.30:8096
}
CloudCaddy Caddyfile
veritablevalor.com {
reverse_proxy 10.0.0.2:80
}
gitea.veritablevalor.com {
reverse_proxy 10.0.0.2:3003
}
3. The problem I’m having:
A little bit more context to my situation
My homelab (from which I am hosting all of the endpoint services) is behind my OPNsense AP which is behind a CGNAT. As such, to get external access to my services without using a VPN I have set up CloudCaddy (with a VPN connection to LocalCaddy) to redirect all relevant requests to LocalCaddy which then handles them appropriately. My A and AAAA records point to CloudCaddy. I have confirmed that this basic setup works by navigating over to https://gitea.veritablevalor.com.
Broader Problem I am trying to solve
There are 2 problems I want to fix with the setup as described above.
When navigating to https://gitea.veritablevalor.com from the local network, my connection will bounce onto the internet and back down to the same network. I would rather redirect local traffic to that domain directly to the relevant server (192.168.1.30 in my case).
I want this because… Many services require a “base url” variable to be set in their configuration (gitea and jellyfin are just two examples) which aid in navigation of the app in the browser. Now in the event where I don’t have internet access, these services will break if I can’t navigate to them using the domain base url specified, which I wouldn’t be able to, because I wouldn’t have internet access. But if I leave the base url set to the IP address of the service, external access through my domain would have limited/broken functionality using my service.
What I have attempted
Luckily I am using OPNsense, so there is some strong control over network traffic available to me. I added a rule in DNS overrides that redirect all outbound LAN traffic that is headed towards veritablevalor.com to my local server at 192.168.1.30 (the eventual endpoint of the domain anyways). Now on the 192.168.1.30 server I override the DNS server to NOT be the gateway (instead something like 1.1.1.1 or 8.8.8.8). This way it can reach the TRUE veritablevalor.com domain if need be.
Current Problem
Navigating to a service gives me a Error code: SSL_ERROR_INTERNAL_ERROR_ALERT.
4. Error messages and/or full log output:
CloudCaddy error logs
{
"data":
[
{"level":"info","ts":1657513555.0060794,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"},
{"level":"info","ts":1657513555.0145984,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]},
{"level":"info","ts":1657513555.0148351,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443},
{"level":"info","ts":1657513555.0148873,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"},
{"level":"info","ts":1657513555.017637,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"},
{"level":"info","ts":1657513555.0182605,"logger":"tls","msg":"finished cleaning storage units"},
{"level":"info","ts":1657513555.0187798,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0001b08c0"},
{"level":"info","ts":1657513555.0191932,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["gitea.veritablevalor.com","veritablevalor.com"]},
{"level":"info","ts":1657513555.0203779,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"},
{"level":"info","ts":1657513555.0206492,"msg":"serving initial configuration"},
{"level":"info","ts":1657513555.0236657,"logger":"tls.obtain","msg":"acquiring lock","identifier":"gitea.veritablevalor.com"},
{"level":"info","ts":1657513555.0281208,"logger":"tls.obtain","msg":"lock acquired","identifier":"gitea.veritablevalor.com"},
{"level":"info","ts":1657513555.0329814,"logger":"tls.obtain","msg":"acquiring lock","identifier":"veritablevalor.com"},
{"level":"info","ts":1657513555.0350692,"logger":"tls.obtain","msg":"lock acquired","identifier":"veritablevalor.com"},
{"level":"info","ts":1657513555.4861112,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["gitea.veritablevalor.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":""},
{"level":"info","ts":1657513555.4868543,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["gitea.veritablevalor.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":""},
{"level":"info","ts":1657513555.4968984,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["veritablevalor.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":""},
{"level":"info","ts":1657513555.4972441,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["veritablevalor.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":""},
{"level":"info","ts":1657513555.903911,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"veritablevalor.com","challenge_type":"http-01","ca":"https://acme-v02.api.letsencrypt.org/directory"},
{"level":"info","ts":1657513555.9956768,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"gitea.veritablevalor.com","challenge_type":"http-01","ca":"https://acme-v02.api.letsencrypt.org/directory"},
{"level":"info","ts":1657513556.0571225,"logger":"tls.issuance.acme","msg":"served key authentication","identifier":"veritablevalor.com","challenge":"http-01","remote":"3.21.169.57:15264","distributed":false},
{"level":"info","ts":1657513556.2818708,"logger":"tls.issuance.acme","msg":"served key authentication","identifier":"veritablevalor.com","challenge":"http-01","remote":"64.78.149.164:46690","distributed":false},
{"level":"info","ts":1657513556.309469,"logger":"tls.issuance.acme","msg":"served key authentication","identifier":"veritablevalor.com","challenge":"http-01","remote":"52.26.120.116:35134","distributed":false},
{"level":"info","ts":1657513556.4052205,"logger":"tls.issuance.acme","msg":"served key authentication","identifier":"veritablevalor.com","challenge":"http-01","remote":"18.193.108.87:26248","distributed":false},
{"level":"info","ts":1657513556.4095523,"logger":"tls.issuance.acme","msg":"served key authentication","identifier":"gitea.veritablevalor.com","challenge":"http-01","remote":"3.21.169.57:15268","distributed":false},
{"level":"error","ts":1657513556.5286098,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"firefly.veritablevalor.com","error":"no information found to solve challenge for identifier: firefly.veritablevalor.com"},
{"level":"error","ts":1657513556.529114,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"firefly.veritablevalor.com","error":"no information found to solve challenge for identifier: firefly.veritablevalor.com"},
{"level":"error","ts":1657513556.5456789,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"firefly.veritablevalor.com","error":"no information found to solve challenge for identifier: firefly.veritablevalor.com"},
{"level":"error","ts":1657513556.546162,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"firefly.veritablevalor.com","error":"no information found to solve challenge for identifier: firefly.veritablevalor.com"},
{"level":"error","ts":1657513556.6097732,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"firefly.veritablevalor.com","error":"no information found to solve challenge for identifier: firefly.veritablevalor.com"},
{"level":"error","ts":1657513556.6105597,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"firefly.veritablevalor.com","error":"no information found to solve challenge for identifier: firefly.veritablevalor.com"},
{"level":"error","ts":1657513556.67968,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"firefly.veritablevalor.com","error":"no information found to solve challenge for identifier: firefly.veritablevalor.com"},
{"level":"error","ts":1657513556.6807985,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"firefly.veritablevalor.com","error":"no information found to solve challenge for identifier: firefly.veritablevalor.com"},
{"level":"info","ts":1657513556.6837645,"logger":"tls.issuance.acme.acme_client","msg":"validations succeeded; finalizing order","order":"https://acme-v02.api.letsencrypt.org/acme/order/626596506/105698922416"},
{"level":"info","ts":1657513557.4753182,"logger":"tls.issuance.acme","msg":"served key authentication","identifier":"gitea.veritablevalor.com","challenge":"http-01","remote":"18.193.108.87:26254","distributed":false},
{"level":"info","ts":1657513557.75396,"logger":"tls.issuance.acme","msg":"served key authentication","identifier":"gitea.veritablevalor.com","challenge":"http-01","remote":"64.78.149.164:46826","distributed":false},
{"level":"info","ts":1657513557.797906,"logger":"tls.issuance.acme","msg":"served key authentication","identifier":"gitea.veritablevalor.com","challenge":"http-01","remote":"52.26.120.116:35136","distributed":false},
{"level":"info","ts":1657513558.0872753,"logger":"tls.issuance.acme.acme_client","msg":"validations succeeded; finalizing order","order":"https://acme-v02.api.letsencrypt.org/acme/order/626596496/105698922546"},
{"level":"info","ts":1657513558.15614,"logger":"tls.issuance.acme.acme_client","msg":"successfully downloaded available certificate chains","count":2,"first_url":"https://acme-v02.api.letsencrypt.org/acme/cert/04e8640488c8502e1506420ff3256d565863"},
{"level":"info","ts":1657513558.1576016,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"veritablevalor.com"},
{"level":"info","ts":1657513558.1577506,"logger":"tls.obtain","msg":"releasing lock","identifier":"veritablevalor.com"},
{"level":"info","ts":1657513559.5788848,"logger":"tls.issuance.acme.acme_client","msg":"successfully downloaded available certificate chains","count":2,"first_url":"https://acme-v02.api.letsencrypt.org/acme/cert/04a3be9a2676ba5541a548bfceb0bf53bca3"},
{"level":"info","ts":1657513559.5795212,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"gitea.veritablevalor.com"},
{"level":"info","ts":1657513559.5795314,"logger":"tls.obtain","msg":"releasing lock","identifier":"gitea.veritablevalor.com"},
{"level":"error","ts":1657513853.479024,"logger":"tls","msg":"tls-alpn challenge","server_name":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513854.4513657,"logger":"tls","msg":"tls-alpn challenge","server_name":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513854.8445227,"logger":"tls","msg":"tls-alpn challenge","server_name":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513856.5327828,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513856.532892,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513856.5658796,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513856.5659857,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513856.6083639,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513856.6085377,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513856.719472,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513856.7199929,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513856.7280574,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513856.7284656,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513857.0832646,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513857.0847573,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513857.9762537,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513857.976913,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513858.0724702,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
{"level":"error","ts":1657513858.0729134,"logger":"tls.issuance.acme","msg":"looking up info for HTTP challenge","host":"gitea.veritablevalor.com","error":"no information found to solve challenge for identifier: gitea.veritablevalor.com"},
]
}
5. What I already tried:
End of section 4.
6. Links to relevant resources:
https://gitea.veritablevalor.com?
Please let me know if there is something obvious I am missing, or if what I am trying to do is just a really bad idea. I am all ears at this point. Thanks!