1. The problem I’m having:
I have a self-hosted Gitlab instance, which listens over a Unix socket. I’m also connected to a Tailnet, and would like to be able to access it via its bare hostname http://gitlab inside the tailnet, as well as using the full ts.net domain name with proper TLS.
The second one is pretty simple, for the bare hostname, but I cannot figure out how to configure a reverse proxy to a Unix socket that doesn’t attempt to retrieve a certificate for [gitlab], without disabling TLS for the entire server. Every guide and question I can find seems to indicate that the canonical way to disable TLS only for a specific reverse proxy is to explicitly specify http:// for the upstream, but for a Unix socket upstream this isn’t possible to do, AFAICT.
There’s unix+h2c/ described in the docs which seems related to what I want, but using that form for the upstream doesn’t seem to change anything (I still see errors trying to retrieve impossible certs).
2. Error messages and/or full log output:
Jan 05 16:19:08 gitlab systemd[1]: Started Caddy.
Jan 05 16:19:09 gitlab caddy[485]: {"level":"error","ts":1704489549.3198774,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"gitlab","issuer":"acme-v02.api.letsencrypt.org-directory","error":"registering account [] with server: provisioning client: performing request: Get \"https://acme-v02.api.letsencrypt.org/directory\": dial tcp: lookup acme-v02.api.letsencrypt.org: no such host"}
...{repeats, no other log messages}...
3. Caddy version:
2.7.6
4. How I installed and ran Caddy:
On NixOS. I just set services.caddy.enable = true in my config. It runs on systemd, no docker image.
a. System environment:
❯ nixos-version
24.05.20231224.5f64a12 (Uakari)
❯ uname -mro
6.1.69 x86_64 GNU/Linux
❯ systemctl --version
systemd 254 (254.6)
+PAM +AUDIT -SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT -QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -XKBCOMMON +UTMP -SYSVINIT default-hierarchy=unified
b. Command:
(from another node in the tailnet)
❯ curl -I https://gitlab.tail19498.ts.net
HTTP/2 302 ...
❯ curl -I http://gitlab
^C⏎ # timeout
c. Service/unit/compose file:
[root@gitlab:~]# systemctl show caddy
Type=notify
Type=notify
ExitType=main
Restart=on-failure
RestartMode=normal
NotifyAccess=main
RestartUSec=5s
RestartSteps=0
Type=notify
ExitType=main
Restart=on-failure
RestartMode=normal
NotifyAccess=main
RestartUSec=5s
RestartSteps=0
RestartMaxDelayUSec=infinity
RestartUSecNext=5s
TimeoutStartUSec=1min 30s
TimeoutStopUSec=5s
TimeoutAbortUSec=5s
TimeoutStartFailureMode=terminate
TimeoutStopFailureMode=terminate
RuntimeMaxUSec=infinity
RuntimeRandomizedExtraUSec=0
WatchdogUSec=0
WatchdogTimestampMonotonic=0
RootDirectoryStartOnly=no
RemainAfterExit=no
GuessMainPID=yes
RestartPreventExitStatus=1
MainPID=485
...skipping...
DefaultDependencies=yes
OnSuccessJobMode=fail
OnFailureJobMode=replace
IgnoreOnIsolate=no
NeedDaemonReload=no
JobTimeoutUSec=infinity
JobRunningTimeoutUSec=infinity
JobTimeoutAction=none
ConditionResult=yes
AssertResult=yes
ConditionTimestamp=Fri 2024-01-05 16:19:08 EST
ConditionTimestampMonotonic=6100237966
AssertTimestamp=Fri 2024-01-05 16:19:08 EST
AssertTimestampMonotonic=6100237967
Transient=no
Perpetual=no
StartLimitIntervalUSec=4h
StartLimitBurst=10
StartLimitAction=none
FailureAction=none
SuccessAction=none
InvocationID=dfc9c88d065d43f792e77b1e9c954ca2
CollectMode=inactive
d. My complete Caddy config:
[root@gitlab:~]# cat /etc/caddy/caddy_config
{
log {
level ERROR
}
}
gitlab {
log {
output file /var/log/caddy/access-gitlab.log
}
reverse_proxy unix+h2c//run/gitlab/gitlab-workhorse.socket {
transport http {
versions h2c
}
}
}
gitlab.tail19498.ts.net {
log {
output file /var/log/caddy/access-gitlab.tail19498.ts.net.log
}
reverse_proxy unix//run/gitlab/gitlab-workhorse.socket
}