Installing LE cert on FreeBSD jail running Caddy that is behind a Proxy

1. The problem I’m having:

I have multiple FreeBSD jails running apps using Caddy, that are behind a Reverse Proxy server that faces the internet. I would like to get LE certs installed on the FreeBSD jails, so that communication between them and the Proxy is secure, and so that I can access the Jails securely without needing to go through the Proxy server.

2. Error messages and/or full log output:

No errors to paste / share

3. Caddy version:

v2.7.4

4. How I installed and ran Caddy:

Caddy is installed / created as part of a script that creates a FreeBSD jail, installs Caddy, and downloads various apps (Nextcloud, etc.) and loads them to be served by Caddy.

a. System environment:

TrueNAS 13.0, running Jails with FreeBSD 13.2-Release. Reverse Proxy is a Rocky Linux Virtual Machine running Virtualmin / Apache.

d. My complete Caddy config:

zenphoto.aiskon.net:80, 10.0.0.248:80 {
	root * /usr/local/www/zenphoto
	file_server
	php_fastcgi 127.0.0.1:9000   
	log {
		output file /var/log/zenphoto.aiskon.net.log
	}
	@forbidden {
		path /.htaccess
		path *.cfg
		path *.log
		path *.zdb
		path *.(redacted for filter - is 3 x's)
	}
	respond @forbidden 404
}

5. Links to relevant resources:

The above applies to Apache, but hopefully can help with Caddy calling for the LE cert.

I’m not following. Your Caddy config is serving from port 80, so Automatic HTTPS will be disabled.

OK, that may be the issue, then. The Caddy config is automatically generated by the script that sets up the jail, installs Caddy, etc. - I may need to go back to the authors and see where it’s specifying that during the setup.

Let me play around a bit more with trying to re-create the jails, too… I wanted to be sure I could narrow down what point in the process is causing the issue, and this helps. Thank you!

I realized I re-created the jail without SSL at one point, so that Caddyfile is invalid. Here’s the correct one after re-creating the jail and attempting to retrieve a LE cert using HTTP (as opposed to DNS).

{
	acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
	email gjschaller@gmail.com
}

zenphoto.aiskon.net {

	root * /usr/local/www/zenphoto
	file_server

	php_fastcgi 127.0.0.1:9000    
  
	log {
		output file /var/log/zenphoto.aiskon.net.log
	}

	@forbidden {
		path /.htaccess
		path *.cfg
		path *.log
		path *.zdb
		path *.(three x's)
	}
	respond @forbidden 404
}

And is there still a problem? I’m not really following.

Yes. When the Jail is created and Caddy installed, it is not pulling the SSL cert correctly from LE. When I browse locally to the server from within my network (not using the Proxy), I get an SSL Protocol Error.

You’re getting a cert from LE’s staging endpoint. This isn’t publicly trusted. Remove that to get a publicly trusted cert.

Thank you. Removed that line, restarted the Caddy service, still getting an SSL Protocol Error in Chrome, and similar in a “fresh” instance of Firefox.

Checked caddy.log, this is the last few lines:

{"level":"info","ts":1694733596.8507757,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"zenphoto.aiskon.net","challenge_type":"tls-alpn-01","ca":"https://acme-staging-v02.api.letsencrypt.org/directory"} {"level":"error","ts":1694733597.309533,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"zenphoto.aiskon.net","challenge_type":"tls-alpn-01","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"76.117.62.249: Error getting validation data","instance":"","subproblems":[]}} {"level":"error","ts":1694733597.309642,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"zenphoto.aiskon.net","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"76.117.62.249: Error getting validation data","instance":"","subproblems":[]},"order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/118474324/10931810344","attempt":2,"max_attempts":3} {"level":"error","ts":1694733597.3096979,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"zenphoto.aiskon.net","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:connection - 76.117.62.249: Error getting validation data"} {"level":"info","ts":1694733599.147808,"logger":"tls.issuance.zerossl.acme_client","msg":"trying to solve challenge","identifier":"zenphoto.aiskon.net","challenge_type":"http-01","ca":"https://acme.zerossl.com/v2/DV90"} {"level":"error","ts":1694733610.6846194,"logger":"tls.issuance.zerossl.acme_client","msg":"challenge failed","identifier":"zenphoto.aiskon.net","challenge_type":"http-01","problem":{"type":"","title":"","detail":"","instance":"","subproblems":[]}} {"level":"error","ts":1694733610.6846917,"logger":"tls.issuance.zerossl.acme_client","msg":"validating authorization","identifier":"zenphoto.aiskon.net","problem":{"type":"","title":"","detail":"","instance":"","subproblems":[]},"order":"https://acme.zerossl.com/v2/DV90/order/BaYXO2-ltMfe0m8l3oD-nQ","attempt":1,"max_attempts":3} {"level":"error","ts":1694733610.6848001,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"zenphoto.aiskon.net","issuer":"acme.zerossl.com-v2-DV90","error":"HTTP 0 - "} {"level":"error","ts":1694733610.684861,"logger":"tls.obtain","msg":"will retry","error":"[zenphoto.aiskon.net] Obtain: [zenphoto.aiskon.net] solving challenge: zenphoto.aiskon.net: [zenphoto.aiskon.net] authorization failed: HTTP 0 - (ca=https://acme.zerossl.com/v2/DV90)","attempt":2,"retrying_in":120,"elapsed":133.2845792,"max_duration":2592000}

… and, it’s working! I think I had to fix the Proxy on the Apache server, but I do have it working now.

Thank you VERY much for your help. There were a lot of moving parts, so I had to look at each of them and make sure they were all playing well with each other.

Thank you again!