Short lived self signed certs and UTC time issue

1. Caddy version (caddy version):

caddy version
v2.5.1 h1:bAWwslD1jNeCzDa+jDCNwb8M3UJ2tPa8UZFFzPVmGKs=

2. How I run Caddy:

My issue can be reproduced by simply running caddy run and POSTing a config to it.

a. System environment:

Linux Mint, manual invocation of Caddy (doesn’t really matter in this case)

b. Command:

`caddy run`

c. Service/unit/compose file:

N/A

d. My complete Caddyfile or JSON config:

{
	"admin": {
		"config": {
			"persist": false
		}
	},
	"storage": {
		"module": "file_system",
		"root": "/tmp/tmp.lkyhcls60L/storage/"
	},
	"apps": {
		"pki": {
			"certificate_authorities": {
				"customca": {
					"name": "my custom CA name",
					"root_common_name": "custom CA - root",
					"intermediate_common_name": "custom CA - intermediate",
					"install_trust": false
				}
			}
		},
		"tls": {
			"certificates": {
				"automate": [
					"example.com",
					"example.home.arpa"
				]
			},
			"automation": {
				"policies": [
					{
						"subjects": [
							"example.com",
							"example.home.arpa"
						],
						"issuers": [{
							"module": "internal",
							"ca": "customca",
							"lifetime": 300000000000
						}],
						"disable_ocsp_stapling": true
					}
				]
			}
		}
	},
	"logging": {
		"logs": {
			"default": {
				"level": "DEBUG"
			}
		}
	}
}

3. The problem I’m having:

When playing with the PKI and reproducing something in my lab, I noticed caddy doing something I was not expecting it to do.

The UTC time is not only used for Caddys log but also for the internal PKI when creating local certs.
I had a hunch that this is the case after reading this thread: How to set log time to +0800 + another one I can’t find anymore where Matt was mentioning that he is on a mission to decrease the size of the Caddy binary and including the Golang timezone db etc would blow it up?!

This could cause issues for very short lived local testing certs - 5 minutes for example.
The default value for the lifetime is 24h, but this can be altered using the lifetime attribute.
https://caddyserver.com/docs/json/apps/tls/automation/policies/issuers/internal/

lifetime

The validity period of certificates.

Duration can be an integer or a string. An integer is interpreted as nanoseconds. If a string, it is a Go time.Duration value such as 300ms, 1.5h, or 2h45m; valid units are ns, us/µs, ms, s, m, h, and d.

So the config section of interest is this one:

"issuers": [{
    "module": "internal",
    "ca": "customca",
    "lifetime": 300000000000
}],

I noticed that you can’t have the lifetime be smaller than 300000000000 nanoseconds (5min).
The docs veeery kind of give you the idea you could have the lifetime be any valid value (though not longer than the root CA probably). This seems to be hard coded here: caddy/acmeserver.go at a1c41210d34c629c3b5cdbdca74c540fde752aad · caddyserver/caddy · GitHub

Although this is related to my original problem (I tried 1min as I needed it to be really short), the main problem is the usage of UTC globally throughout Caddy.

Here is the invocation of Caddy where I demonstrate the issue:

❯ timedatectl
               Local time: Sun 2022-05-22 20:47:06 CEST
           Universal time: Sun 2022-05-22 18:47:06 UTC
                 RTC time: Sun 2022-05-22 18:47:06
                Time zone: Europe/Berlin (CEST, +0200)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no

❯ caddy run
2022/05/22 18:47:13.147 INFO    admin   admin endpoint started  {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["//localhost:2019", "//[::1]:2019", "//127.0.0.1:2019"]}
2022/05/22 18:47:13.147 INFO    serving initial configuration
2022/05/22 18:48:06.764 INFO    admin.api       received request        {"method": "POST", "host": "localhost:2019", "uri": "/load", "remote_ip": "127.0.0.1", "remote_port": "44872", "headers": {"Accept":["*/*"],"Content-Length":["828"],"Content-Type":["application/json"],"User-Agent":["curl/7.68.0"]}}
2022/05/22 18:48:06.764 INFO    admin   admin endpoint started  {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["//localhost:2019", "//[::1]:2019", "//127.0.0.1:2019"]}
2022/05/22 18:48:06.765 INFO    pki.ca.customca root certificate trust store installation disabled; unconfigured clients may show warnings      {"path": "storage:pki/authorities/customca/root.crt"}
2022/05/22 18:48:06.765 INFO    tls.cache.maintenance   started background certificate maintenance      {"cache": "0xc0003cca10"}
2022/05/22 18:48:06.765 INFO    admin.api       load complete
2022/05/22 18:48:06.766 INFO    tls     cleaning storage unit   {"description": "FileStorage:/tmp/tmp.lkyhcls60L/storage/"}
2022/05/22 18:48:06.766 INFO    tls     finished cleaning storage units
2022/05/22 18:48:06.766 INFO    tls.obtain      acquiring lock  {"identifier": "example.com"}
2022/05/22 18:48:06.766 INFO    tls.obtain      acquiring lock  {"identifier": "example.home.arpa"}
2022/05/22 18:48:06.768 INFO    admin   stopped previous server {"address": "tcp/localhost:2019"}
2022/05/22 18:48:06.769 INFO    tls.obtain      lock acquired   {"identifier": "example.com"}
2022/05/22 18:48:06.769 INFO    tls.obtain      lock acquired   {"identifier": "example.home.arpa"}
2022/05/22 18:48:06.769 DEBUG   tls.obtain      trying issuer 1/1       {"issuer": "customca"}
2022/05/22 18:48:06.769 DEBUG   tls.obtain      trying issuer 1/1       {"issuer": "customca"}
2022/05/22 18:48:06.770 DEBUG   pki.ca.customca using intermediate signer       {"serial": "102486882956661856687520432921376090461", "not_before": "2022-05-22 18:48:06 +0000 UTC", "not_after": "2022-05-29 18:48:06 +0000 UTC"}
2022/05/22 18:48:06.770 DEBUG   pki.ca.customca using intermediate signer       {"serial": "102486882956661856687520432921376090461", "not_before": "2022-05-22 18:48:06 +0000 UTC", "not_after": "2022-05-29 18:48:06 +0000 UTC"}
2022/05/22 18:48:06.770 INFO    tls.obtain      certificate obtained successfully       {"identifier": "example.com"}
2022/05/22 18:48:06.770 INFO    tls.obtain      releasing lock  {"identifier": "example.com"}
2022/05/22 18:48:06.770 INFO    tls.obtain      certificate obtained successfully       {"identifier": "example.home.arpa"}
2022/05/22 18:48:06.770 INFO    tls.obtain      releasing lock  {"identifier": "example.home.arpa"}
2022/05/22 18:48:06.771 DEBUG   tls.cache       added certificate to cache      {"subjects": ["example.home.arpa"], "expiration": "2022/05/22 18:53:06.000", "managed": true, "issuer_key": "customca", "hash": "e1758808081cdd91ea2e947b934c5b8d5e48e4a7369da7028f109a9e275eb3a3", "cache_size": 1, "cache_capacity": 10000}
2022/05/22 18:48:06.771 DEBUG   tls.cache       added certificate to cache      {"subjects": ["example.com"], "expiration": "2022/05/22 18:53:06.000", "managed": true, "issuer_key": "customca", "hash": "e27ac2ff83476e1dbe5975d3b658cda7d0afec6fd8988e8a0226188fb9115b06", "cache_size": 2, "cache_capacity": 10000}
^C2022/05/22 18:48:10.906       INFO    shutting down   {"signal": "SIGINT"}
2022/05/22 18:48:10.906 WARN    exiting; byeee!! 👋     {"signal": "SIGINT"}
2022/05/22 18:48:10.906 INFO    tls.cache.maintenance   stopped background certificate maintenance      {"cache": "0xc0003cca10"}
2022/05/22 18:48:10.907 INFO    admin   stopped previous server {"address": "tcp/localhost:2019"}
2022/05/22 18:48:10.907 INFO    shutdown complete       {"signal": "SIGINT", "exit_code": 0}
❯ openssl x509 --enddate --startdate -in storage/certificates/customca/example.com/example.com.crt -noout
notAfter=May 22 18:53:06 2022 GMT
notBefore=May 22 18:48:06 2022 GMT

So you can see the certs have a timestamp which is in the past. This might not be an issue for certs with a default value of 24h but for shorter ones it could very well be.

So my question is: Is it intended behavior for the internal PKI to create its CA and its certs based on the UTC time even though the process could know the real timezone the server is running on?
For logs I get it, but for certs I really don’t. Maybe I just need some more words on why this is and why I can’t have there short lived certs on a Caddy instance which is not UTC.

4. Error messages and/or full log output:

5. What I already tried:

See above

6. Links to relevant resources:

Linked within the text

Which timestamp is in the past? May 22 18:48:06 2022 GMT and May 22 18:53:06 2022 GMT are not in the past. The cert was issued at precisely 18:48:06.

I don’t really understand the problem here?

Hey Matt :wave:t2:

That’s why I added this in the output section above:

❯ timedatectl
               Local time: Sun 2022-05-22 20:47:06 CEST
           Universal time: Sun 2022-05-22 18:47:06 UTC
                 RTC time: Sun 2022-05-22 18:47:06
                Time zone: Europe/Berlin (CEST, +0200)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no

The cert were valid for 5 min - which was 2 hours ago in Germany. So by the time they got generated they were invalid already.

Or am I missing something here?

Ok so it’s late here and I think I have a logic issue.
My thought was that when I create a cert, the dates for begin and end date should match my timezone.

The certs are in GMT timezone (Greenwich Mean Time) which is the same as UTC. When clients check the validity of a cert, it’ll use your system’s time converted to UTC to compare.

If every client compared using their local timezone, things would break because everyone’s using a different timezone around the world. Using UTC makes that constant/consistent.

There’s no issue here.

2 Likes

Thank you Francis! :slight_smile: I should’ve looked up more about the validation process in general and I bet the first three stackoverflow posts would’ve answered my question.

2 Likes

This topic was automatically closed after 30 days. New replies are no longer allowed.