How to use an existing Let's Encrypt ACME Account ID?

Help
1

1. The problem I’m having:

I have successfully migrated from nginx + certbot to caddy.

However, I had registered my Account ID used by and with certbot here: ECDSA availability in production environment - API Announcements - Let's Encrypt Community Support

This allowed me to get certificates with full ECDSA chain (ISRG Root X2 / E1 as shown here Chain of Trust - Let's Encrypt).

In order to get this same ECDSA chain from caddy, I must reuse the same Account ID as in certbot.

I did find this ID in certbot data files and tried to put it in caddy_data/caddy/acme/acme-v02.api.letsencrypt.org-directory/users/my@email.com/letsencrypt.json

I also copied the pkey to caddy_data/caddy/acme/acme-v02.api.letsencrypt.org-directory/users/my@email.com/letsencrypt.key
(I had to convert it to PEM because certbot stores it in JWK)

I then delete the certificate files and restart caddy.

But it seems that caddy doesn’t pickup these values and registers a new account each time.
In the logs hereunder, 1608793527 is a new Account ID generated on the fly, not the one I’m trying to use.

I also set preferred_chains in Caddyfile.

What am I missing?

2. Error messages and/or full log output:

caddy  | 2024-03-08T16:29:54.437117775Z {"level":"info","ts":1709915394.4368443,"logger":"http","msg":"waiting on internal rate limiter","identifiers":["abc.example.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"my@email.com"}
caddy  | 2024-03-08T16:29:54.437216126Z {"level":"info","ts":1709915394.4370458,"logger":"http","msg":"done waiting on internal rate limiter","identifiers":["abc.example.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"my@email.com"}
caddy  | 2024-03-08T16:29:54.817002937Z {"level":"info","ts":1709915394.816721,"logger":"http.acme_client","msg":"trying to solve challenge","identifier":"abc.example.com","challenge_type":"dns-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
caddy  | 2024-03-08T16:29:59.596502148Z {"level":"info","ts":1709915399.5958266,"logger":"http.acme_client","msg":"authorization finalized","identifier":"abc.example.com","authz_status":"valid"}
caddy  | 2024-03-08T16:29:59.596688553Z {"level":"info","ts":1709915399.5960453,"logger":"http.acme_client","msg":"validations succeeded; finalizing order","order":"https://acme-v02.api.letsencrypt.org/acme/order/1608793527/250653778277"}
caddy  | 2024-03-08T16:30:00.532433354Z {"level":"info","ts":1709915400.5321703,"logger":"http.acme_client","msg":"successfully downloaded available certificate chains","count":2,"first_url":"https://acme-v02.api.letsencrypt.org/acme/cert/0383e04defea93bbb346fbbe28b5cdf69c50"}
caddy  | 2024-03-08T16:30:00.533385715Z {"level":"warn","ts":1709915400.533001,"logger":"http","msg":"did not find chain matching preferences; using first"}
caddy  | 2024-03-08T16:30:00.534770312Z {"level":"info","ts":1709915400.5345626,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"abc.example.com"}
caddy  | 2024-03-08T16:30:00.535024495Z {"level":"info","ts":1709915400.5349102,"logger":"tls.obtain","msg":"releasing lock","identifier":"abc.example.com"}

3. Caddy version:

v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A=

4. How I installed and ran Caddy:

a. System environment:

Docker version 25.0.3, build 4debf41

b. Command:

c. Service/unit/compose file:

d. My complete Caddy config:

{
	# Global options block.
	# Customizes the admin API endpoint. Accepts placeholders. Takes network addresses.
	# Default: localhost:2019, unless the CADDY_ADMIN environment variable is set.
	# If set to off, then the admin endpoint will be disabled. When disabled, config changes will be impossible without stopping and starting the server, since the caddy reload command uses the admin API to push the new config to the running server.
	admin off

	log {
		output stderr
		format json
	}

	# Disable HTTP-to-HTTPS redirects.
	# https://caddyserver.com/docs/caddyfile/options#auto-https
	auto_https disable_redirects

	# TLS Options
	email my@email.com
	# key_type  ed25519|p256|p384|rsa2048|rsa4096
	key_type p384
	cert_issuer acme {
		email my@email.com
		preferred_chains {
			root_common_name "ISRG Root X2"
		}
	}
	acme_dns ovh {
		# https://github.com/caddy-dns/ovh
		# https://github.com/ovh/go-ovh
		endpoint ovh-eu
		application_key redacted
		application_secret redacted
		consumer_key redacted
	}

	# GeoIP2
	# https://github.com/zhangjiayin/caddy-geoip2
	order geoip2_vars first
	# Only configure databaseDirectory and editionID when autoupdate is not desired.
	geoip2 {
		# accountId         ooo
		databaseDirectory "/tmp/"
		# licenseKey        "ooo"
		lockFile          "/tmp/geoip2.lock"
		editionID         "GeoLite2-Country"
		# updateUrl         "https://updates.maxmind.com"
		# updateFrequency   86400   # in seconds
	}
}

(headers) {
	map {host} {src-policies} {
		abc.example.com       redacted
		def.example.com    redacted
		ghi.example.com  redacted
		default             "default-src 'self'; object-src 'none'; style-src 'self'; script-src 'self'; font-src 'self';"
	}
	map {host} {referrer-policy} {
		jkl.example.com   redacted
		default             "no-referrer"
	}

	# https://owasp.org/www-project-secure-headers/
	# https://infosec.mozilla.org/guidelines/web_security
	header {
		# https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP
		Content-Security-Policy "{src-policies} form-action 'self'; frame-ancestors 'self'; upgrade-insecure-requests;"

		# https://owasp.org/www-project-secure-headers/#referrer-policy
		Referrer-Policy {referrer-policy}

		# Enable HTTP Strict Transport Security (HSTS)
		Strict-Transport-Security "max-age=15768000; includeSubDomains;"

		# Disallow sniffing of X-Content-Type-Options
		X-Content-Type-Options "nosniff"

		X-Download-Options "noopen"

		# https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options
		# Disallow the site to be rendered within a frame (clickjacking protection)
		# If you want to use FIDO2 WebAuthn, set X-Frame-Options to "SAMEORIGIN" or the Browser will block those requests
		X-Frame-Options "SAMEORIGIN"

		X-Permitted-Cross-Domain-Policies "none"

		# Prevent search engines from indexing (optional)
		X-Robots-Tag "noindex, nofollow"

		# Disable cross-site filter (XSS)
		# Deprecated in favor of the Content-Security-Policy, see https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection
		# It is thus recommended to set the header as X-XSS-Protection: 0
		X-XSS-Protection "0"

		# Remove headers
		# Remove Server name
		-Server
		# Remove X-Powered-By though this shouldn t be an issue, better opsec to remove
		-X-Powered-By
		# Remove Last-Modified because etag is the same and is as effective
		-Last-Modified
	}
}
(geoip2) {
	# strict: Alway ignore 'X-Forwarded-For' header 
	# wild:   Trust 'X-Forwarded-For' header if existed
	# trusted_proxies: Trust 'X-Forwarded-For' header if trusted_proxies is also valid (see https://caddyserver.com/docs/caddyfile/options#trusted-proxies)
	# default: trusted_proxies
	geoip2_vars strict

	# Add country code to the header
	header geoip-country "{geoip2.country_code}"

	# Block any country not France, allowing private ranges
	# private_ranges: https://caddyserver.com/docs/caddyfile/matchers#client-ip
	@denied {
		not client_ip private_ranges
		not expression ({geoip2.country_code} == "FR")
	}
	respond @denied 444 {
		close
	}
}
(only_private) {
	# Block any non private ranges
	# private_ranges: https://caddyserver.com/docs/caddyfile/matchers#client-ip
	@denied {
		not client_ip private_ranges
	}
	respond @denied 444 {
		close
	}
}
(logs) {
	log {args[0]}_access {
		format json
		output file /logs/{args[0]}_access.log {
			roll_size 10MB
			roll_keep 10
		}
	}
}

mno.example.com {
	reverse_proxy http://10.2.6.2:80
	rewrite * /alexa{uri}
	uri strip_suffix /

	import headers
	import logs mno.example.com

	@denied {
		not header_regexp useragent_asksonic User-Agent "ghi"
	}
	respond @denied 444 {
		close
	}
}

def.example.com {
	reverse_proxy http://10.2.2.3:80

	import headers
	import logs def.example.com
	import geoip2
}

ghi.example.com {
	reverse_proxy http://10.2.7.2:80

	import headers
	import logs ghi.example.com
	import only_private
}

abc.example.com {
	reverse_proxy http://10.2.1.3:80

	import headers
	import logs abc.example.com

	geoip2_vars strict
	header geoip-country "{geoip2.country_code}"
	@denied {
		not client_ip private_ranges
		not expression ({geoip2.country_code} == "US")
		not header User-Agent "bcd"
		not header_regexp useragent_hasslambda User-Agent "abc"
	}
	respond @denied 444 {
		close
	}
}

jkl.example.com {
	reverse_proxy http://10.2.3.2:80

	import headers
	import logs jkl.example.com
	import only_private
}

5. Links to relevant resources:

2

Why are you trying to reuse an account? There’s no reason to do that. Just let Caddy make a new account. It’s not a problem to make new accounts as needed.

3

CertMagic actually supports specifying your ACME account key when you want to use a specific one.

This is exposed in the Caddy JSON here: JSON Config Structure - Caddy Documentation

This is very seldom used, so I don’t think we’ve exposed it in the Caddyfile yet. But you can use the JSON for now! (Or submit a pull request to add it to the Caddyfile)

4

Hello
thank you for your answer.
As I wrote, my account has been whitelisted by Let’s Encrypt.
That’s why I need to reuse this specific one.

5

Are you sure the whitelist is still necessary? I don’t think it is at this point.

6

Hi
thanks for the clue, I’ll look into this.

I guess I’l learn this in the docs, but I reckon once one has modified the JSON conf we can’t use the Caddyfile anymore ?

7

Correct, Caddyfile to JSON is a one-way conversion.

If you must, you can edit the account ID in /data/caddy/acme/acme-v02.api.letsencrypt.org-directory/users/default/default.json I thnk, and also copy over your account private key to /data/caddy/acme/acme-v02.api.letsencrypt.org-directory/users/default/default.key

8

I did replace the account ID in
/data/caddy/acme/acme-v02.api.letsencrypt.org-directory/users/default/default.json

but also in (since it was there also)
/data/caddy/acme/acme-v02.api.letsencrypt.org-directory/users/my@email.com/xxxxx.json

As for the private key, I had to convert it from JWK (its storage in certbot datafiles) to PEM and copied it there (there was no key in the defaultdir)
/data/caddy/acme/acme-v02.api.letsencrypt.org-directory/users/my@email.com/xxxxx.key

I don’t know which one did the trick (the account id in default or in my@email.com), but it worked.

The Let’sEnrcypt allowlist is still required in order to get a cert with full ECDSA chain from ISRG Root X2/ E1 as seen in their chain of trust doc: Chain of Trust - Let's Encrypt. Otherwise, you get a cert from ISRG Root X1/ R3 that are RSA CA and intermediate certs.

I rather choosed to change the datafile, provided it is not overridden upon next runs, as it allows me to keep using the Caddyfile.

I’ll see if I have the skills to push a PR to include the account key param in the Caddyfile as suggested by @matt .
But, as of now, I did not understand yet why there is only the key in the JSON Config file, in such a case, where is the Account ID specified to CertMagic?

1 Like
9

Because Caddy’s native config language is JSON, the Caddyfile is just an adapter that produces JSON config. That means that for any config property, we need to write additional code to have support in the Caddyfile. We just never did it for that option (cause we didn’t think to – I didn’t even realize it was a thing), you’re the first to need it so far.

Just look for AccountKey in the codebase, next to that there’s an Email field, you can trace where Email is set in the Caddyfile adapter code and you can add an account_key option next to that. Probably inside the issuer config.

1 Like
10

Hum I was probably not clear enough: this I understand very well.

What is not clear for me is: I expect the account private key to go with a 2nd parameter which should be the Account ID itself. (since I had to change both in the datafiles)

In the JSON Config file doc, I saw clearly the account key, but not the ID (I can only see the email).
How can the CertMagic’s magic happen without having the account ID?
I’m not an ACME expert, but I doubt the email is enough to lookup an account since it seems that one can create several accounts with the same email…
I’ll probably have to dig into the code to understand all the details.

edit: OK, I just read that the account lookup is performed with the key as parameter RFC 8555 ACME: 7.3.1. Finding an Account URL Given a Key
Nevertheless it would probably be more efficient if already knowing the AccountID (URL); wouldn’t it?

11

An account has to be looked up using the key, I guess I’m not quite sure what you’re asking with regards to efficiency?

12

The objective of the account lookup is to retrieve its directory URL, right?
Then it’d be more efficient to know directly this URL from the config file.

13

But what are you measuring with regards to “efficiency”? Is there throughput or something that is being bottlenecked? I guess I still don’t understand the question…

1 Like
14

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.