Caddyfile failure partly switching to wildcard cert

[adlib (Ted Jangius)]

1. Caddy version (caddy version):

v2.4.3 h1:Y1FaV2N4WO3rBqxSYA8UZsZTQdN+PwcoOcAiZTM8C0I=

2. How I run Caddy:

a. System environment:

FreeBSD 12.2-RELEASE-p2 jail inside TrueNAS-12.0-U5.1

b. Command:

service caddy start

c. Service/unit/compose file:

d. My complete Caddyfile or JSON config:

{
	order jwt first
	dynamic_dns {
		provider cloudflare {env.CLOUDFLARE_DYNDNS_API_TOKEN}
		check_interval 1s
		domains {
			my.site home
		}
		ip_source upnp
		ip_source simple_http https://api64.ipify.org
		ip_source simple_http https://api.ipify.org
		ip_source simple_http https://icanhazip.com
	}
}

(dns) {
	tls {
		dns cloudflare {env.CLOUDFLARE_DYNDNS_API_TOKEN}
	}
}

(encoding) {
	encode gzip zstd
}

(header-sec) {
	header {
		# Enable HTTP Strict Transport Security (HSTS) to force clients to always
		# connect via HTTPS (do not use if only testing)
		Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
		# Enable cross-site filter (XSS) and tell browser to block detected attacks
		X-XSS-Protection "1; mode=block"
		# Prevent some browsers from MIME-sniffing a response away from the declared Content-Type
		X-Content-Type-Options nosniff
		# Disallow the site to be rendered within a frame (clickjacking protection)
		X-Frame-Options SAMEORIGIN
		# Remove Server field
		-Server
		# Upgrade insecure requests
		Content-Security-Policy "upgrade-insecure-requests"
		# Don't include referrer info in requests
		# Referrer-Policy "no-referrer"
	}
}

(logging) {
	log {
		# output stdout
		output file /config/logs/access.{args.0}.log
		# level DEBUG
	}
}

(primary-auth) {
	@external not remote_ip 192.168.0.0/16
	route @external {
		jwt {
			primary yes
			set auth url https://auth.my.site/auth
			crypto key {env.CADDY_KEY_ID} verify {env.CADDY_SHARED_KEY}
			allow roles allowed
		}
	}
}

(auth) {
	@external not remote_ip 192.168.0.0/16
	route @external {
		jwt {
			set auth url https://auth.my.site/auth
			allow roles allowed
		}
	}
}

*.my.site {
	import auth
	import dns
	import encoding
	import logging subdomain

	map {host} {upstream} {
		dns https://dns.my.site:443
		duplicati http://duplicati:8200
		homebridge http://homebridge.lan:8080
		ipmi http://ipmi.lan:80
		# nextcloud       nextcloud.my.site.:443
		nas https://nas.my.site:443
		unifi https://unifi.my.site:8443
		default https://proxy.my.site:443
	}

	reverse_proxy {upstream}
}

proxy.my.site {
	import primary-auth
	import dns
	import encoding
	import logging proxy
	templates
	file_server
	root * /config/www/home
}

nextcloud.my.site, cloud.my.site {
	import dns
	import header-sec
	import encoding
	import logging nextcloud
	redir /.well-known/caldav /remote.php/dav 301
	redir /.well-known/carddav /remote.php/dav 301
	redir /.well-known/nodeinfo /index.php/nodeinfo 301
	redir /.well-known/webfinger /index.php/webfinger 301
	reverse_proxy https://nextcloud.my.site.:443
}

auth.my.site {
	import encoding
	import logging auth
	route /auth* {
		authp {
			# path /auth
			cookie domain my.site
			cookie lifetime 3600
			backend google {env.GOOGLE_CLIENT_ID} {env.GOOGLE_CLIENT_SECRET}
			crypto key {env.CADDY_KEY_ID} sign-verify {env.CADDY_SHARED_KEY}
			transform user {
				suffix match email @my.site
				add role allowed
			}
		}
	}
	route /sso/oauth2/google* {
		jwt {
			set auth url /auth/oauth2/google
		}
		respond * "google oauth2 sso" 200
	}
	route /* {
		redir https://{hostport}/auth 302
	}
}

sale.my.site {
	import dns
	import encoding
	import logging sale
	file_server
	root * /config/www/sale
}

# previous solution
# import www/sites/*.caddysite

3. The problem I’m having:

I’ve long run caddy with several subdomains that I’ve defined in *.caddysite files that has worked well, but I’ve come to realise that the subdomain list is now growing too large to maintain individually, why I figured that switching to a wildcard certificate for them all would work well.
In trying to achieve this, I searched a bit and found this which pointed me toward the map directive.
The problem now though, is that using the caddyfile above doesn’t work for the wildcard domains, surely due to configuration errors on my part.

Switching from the previous method (in part found commented at the end of the caddyfile above) also seems to have screwed with the oAuth stuff. I now receive an Error 401: invalid_client from google upon authentication attempts, and I’m not really sure why.

4. Error messages and/or full log output:

just holler if there’s any logs that would help

5. What I already tried:

I’ve tried reading the different resources and skimming the logs, but I’m not really getting any wiser.
I’m sure I’ve done something really stupid, please help me sort it out!

Preferrably, I want all of my sites managed under a wildcard cert, but I assume this is hard if at all possible to achieve, but hopefully the amount of separate sites can be minimized.

Also, I’ve tried searching for solutions like this or this for caddy to ”dynamically” log to files named after the current site, but I’m not getting that to work either, and from the reading I’ve done, this can’t be done with placeholders, am I right?

Thanks in advance!

6. Links to relevant resources:

map (Caddyfile directive) - Caddy Documentation

Serving tens of thousands of domains over HTTPS with Caddy

This guide is a free sample of what is available exclusively for sponsors in my Expert Caddy series, where I help you master the ways of the Caddy web server. (This guide is still WIP.) Many online businesses have invested thousands, even millions, of dollars for custom software and proprietary solutions to serve their customers’ websites over HTTPS. These traditional methods then cost thousands of dollars per year to maintain it in infrastructure, engineering, and payroll. I’ll show you how …

[francislavoie]

Unfortunately, when using a placeholder for reverse_proxy upstreams, you must not include the scheme (https:// or http://). It must be only the host and port.

If you need to enable TLS, then you’ll need to have a separate reverse_proxy that explicitly enables tls.

	map {host}     {upstream}          {use-tls} {
		dns        dns.my.site:443     yes
		duplicati  duplicati:8200      no
		homebridge homebridge.lan:8080 no
		ipmi       ipmi.lan:80         no
		nas        nas.my.site:443     yes
		unifi      unifi.my.site:8443  yes
		default    proxy.my.site:443   yes
	}

	@proxy-https expression `{use-tls} == "yes"`
	reverse_proxy @proxy-https {upstream} {
		transport http {
			tls
		}
	}

	reverse_proxy {upstream}

The reason is that the reverse_proxy handler needs to provision a TLS config at startup, and all requests using that handler will use TLS, so to do both HTTP and HTTPS, you need to use two separate handlers.

Basically https:// is a Caddyfile-specific shortcut for enabling TLS (and setting the port to 443 if you didn’t otherwise specify a port) but placeholders are evaluated at runtime, and is separate from the Caddyfile adapter logic.

[basil (Basil Hendroff)]

@adlib The map directive is one of my favourite Caddy commands when dealing with a number of subdomains (or sub-paths). I love the ‘at a glance’ overview it provides me across all subdomains. However, it’s worth noting that coupling map and the upstream services will close off other options that may be of interest at some point e.g. determining the status of upstream servers API — Caddy Documentation. You just need to be aware of this. If those options aren’t going to be of interest, what @francislavoie has suggested works perfectly. For a broader discussion of the issue, and discussion of a hybrid map approach that uses snippets to decouple `map and the upstream servers, refer to Load balancing queries