Block sharing HTTP/HTTPS protocols with no redirect and uses custom cert fails to run

1. Caddy version (caddy version):

2.1.1

2. How I run Caddy:

a. System environment:

Linux - Manjaro KDE
Docker 19.03

b. Command:

docker run --rm -it \
--name caddytest \
-p 80:80 -p 443:443 \
-v $PWD/public:/usr/share/caddy/ \
-v $PWD/caddy/data:/data  \
-v $PWD/caddy/Caddyfile:/etc/caddy/Caddyfile \
-v $PWD/caddy/tls:/tls caddy

c. Service/unit/compose file:

N/A

d. My complete Caddyfile or JSON config:

http://pugs.localhost https://pugs.localhost {
  #tls internal
  tls /tls/pugs.localhost.pem /tls/pugs.localhost-key.pem
  root * /usr/share/caddy
  file_server
}

3. The problem I’m having:

Support HTTP and HTTPS without redirect, but providing local cert for TLS when using HTTPS. Fails.

4. Error messages and/or full log output:

Not necessary for localhost site address as Caddy would otherwise enable this implicitly for HTTPS, When explicit produces same problem:

tls internal:

run: adapting config using caddyfile: hostname appears in more than one automation policy, making certificate management ambiguous: pugs.localhost

Supplying my own certificate with tls cert.pem key.pem:

run: adapting config using caddyfile: server listening on [:80] is HTTP, but attempts to configure TLS connection policies

5. What I already tried:

  • Duplicating the inner config with the HTTPS block version having the tls line.
  • Creating a snippet for the shared block content and using import snippet_name in each block.

Both of those work, just curious if it can all be managed in a single block instead of one for each protocol + snippet.

Also tried using handle protocol https {} to wrap the tls directive in conditionally, but this failed with error:

run: adapting config using caddyfile: parsing caddyfile tokens for 'handle': directive 'tls' is not ordered, so it cannot be used here

I don’t think this is possible without adding more complex implicit behaviour to the Caddyfile adapter.

FYI handle protocol https {} is not valid, matchers must be either *, a path starting with /, or a named matcher starting with @. So you would need to do this:

@https protocol https
handle @https {
	...
}

But even then, it wouldn’t work here because tls is not a request handler, it’s essentially site-wide config, and you can’t wrap those in request handlers or use matchers.

If you caddy adapt this following Caddyfile config, you’ll notice that the :80 and :443 listeners are split and the config is duplicated in the JSON. This is because Caddy can’t serve HTTP and HTTPS with the same listeners because one expects TLS bytes and the other doesn’t.

http://pugs.localhost https://pugs.localhost {
  root * /usr/share/caddy
  file_server
}
{
	"apps": {
		"http": {
			"servers": {
				"srv0": {
					"listen": [
						":80"
					],
					"routes": [
						{
							"match": [
								{
									"host": [
										"pugs.localhost"
									]
								}
							],
							"handle": [
								{
									"handler": "subroute",
									"routes": [
										{
											"handle": [
												{
													"handler": "vars",
													"root": "/usr/share/caddy"
												},
												{
													"handler": "file_server",
													"hide": [
														"Caddyfile"
													]
												}
											]
										}
									]
								}
							],
							"terminal": true
						}
					],
					"automatic_https": {
						"skip": [
							"pugs.localhost"
						]
					}
				},
				"srv1": {
					"listen": [
						":443"
					],
					"routes": [
						{
							"match": [
								{
									"host": [
										"pugs.localhost"
									]
								}
							],
							"handle": [
								{
									"handler": "subroute",
									"routes": [
										{
											"handle": [
												{
													"handler": "vars",
													"root": "/usr/share/caddy"
												},
												{
													"handler": "file_server",
													"hide": [
														"Caddyfile"
													]
												}
											]
										}
									]
								}
							],
							"terminal": true
						}
					]
				}
			}
		}
	}
}

I think in Caddy 1, if a site block had both http:// and https:// addresses (implicitly or explicitly), it would still enable TLS for the non-http:// sites. Hmm. I will try to look into this case…

Hmm. I will try to look into this case…

It’s not super important to me. I just noticed it while working on that auto_https wiki doc contribution in my spare time. I have used the support for testing / verifying some features like Cache-Control immutable which Firefox says only is respected with HTTPS.

I used to avoid HTTPS locally due to the self-signed issues (Chrome especially makes it more difficult to bypass, especially on mobile). I have since learned that Caddy is awesome and can avoid that if I had been using the binary locally, instead of running via Docker (which I still prefer).

I am not sure if I can take the Caddy rootCA from trust store within Docker and use that locally, not sure how long it’d remain valid for that way. I’ve used mkcert which generates a 10 year valid rootCA(which is valid for private/internal CA certs) and generated my site leaf certificate after from that rootCA one, providing the leaf cert to Caddy.

Caddy becomes a little less convenient at handling this niche situation, but perhaps it could be argued that’s a good thing as there should rarely be any reason for it to be a good idea, so you could brush it off as Caddy discourages bad practice?

1 Like

It’ll remain valid as long as you keep the /data volume persisted somewhere.

Caddy basically does the same thing with tls internal, except it uses smallstep. End result is the same. The root cert installation is almost identical to mkcert’s.

2 Likes

That’s not what I was referring to. If I took data/caddy/pki/authorities/local/ intermediate and root certs and placed them on trusted stores for my devices, how well does that last maintenance wise? (Answer: Just the root cert should be fine, and it lasts a decade)

I’ve not seen the root cert validity duration, but the intermediate appears to be 1 week, and the leaf cert for a site block a single day. Do I just need the root cert in my trust stores and does that have to be changed frequently?

Caddy basically does the same thing with tls internal , except it uses smallstep. End result is the same. The root cert installation is almost identical to mkcert’s.

mkcert was simple enough to create my root cert and distribute, then generate the leaf cert. Not sure if I can supply Caddy that root cert to generate it’s intermediate/leaf certs for, would be better than having mkcert generate leaf certs for site addresses manually.


EDIT: I have inspected the root.crt with:

sudo openssl x509 -in caddy/data/caddy/pki/authorities/local/root.crt -text -noout

Which shows the validity period of ~10 years :

Validity
  Not Before: Sep 10 03:07:20 2020 GMT
  Not After : Jul 20 03:07:20 2030 GMT

I have added it to my local OS trust store via (Arch Linux, other linux distros may have different methods):

sudo trust anchor --store caddy/data/caddy/pki/authorities/local/root.crt -v

I don’t think I need to add the intermediate cert to my trust store, Firefox seems fine without doing so, Chrome refuses regardless, this has apparently been fixed in master in early July, but latest Docker 2.1.1 which was last updated 6 hours ago is still missing the fix. So for now I’ll continue to use mkcert.

EDIT: Received response on github issue, Chrome cert fix won’t be made available to 2.1, requires 2.2 release, so I’ll stick to mkcert for now.

Or just use the 2.2 release candidate that’s been out for a month?

I’m using Docker, which @francislavoie mentions in the Github issue:

We don’t publish beta and RC versions to Docker.

I have no interest in building from master atm, nor using a local binary for Caddy, I’m fine with mkcert in the meantime.

I have a similar issue. Basically with caddy 2.1.1 from docker:

example.com {
  reverse_proxy /_matrix/* http://localhost:8008
}

example.com:8448 {
  reverse_proxy http://localhost:8008
}

causes:

[ERROR]  Removing invalid block: hostname appears in more than one automation policy, making certificate management ambiguous:

(slightly modified from https://github.com/matrix-org/synapse/blob/master/docs/reverse_proxy.md#caddy-2 )

Any ideas for sharing the LE certificate on :443 and :8448 ?

PS: also, why does the reverse-proxy touch the Content-Type header

I think you would define both site addresses for the same site block, then need to conditionally handle them based on port? You can get the port from a placeholder, perhaps with an expression?

I haven’t tried, so no idea if this is valid:

example.com, example.com:8448 {
  @port_80 expression {port} == 80
  @port_8448 expression {port} == 8448

  handle @port_80 {
    reverse_proxy /_matrix/* http://localhost:8008
  }
  handle @port_8448 {
    reverse_proxy http://localhost:8008
  }
}

Any ideas for sharing the LE certificate on :443 and :8448 ?

It shouldn’t matter what port, unless it’s the HTTP port, it’s all treated implicitly as HTTPS.

I wasn’t able to reproduce the issue, but I was only running locally with local_certs/tls internal, did a test with example.com as site address and allowing LetsEncrypt to be used, but again no errors with my docker 2.1.1 container.

Thank you for your suggestion. Actually it seems that just a simple block like:

example.com, example.com:8448 {
...
}

triggers the same error

Use Caddyfile:

{
  local_certs
}

example.com, example.com:8448 {
  respond "Hello"
}

Run docker command:

docker run --rm -it --name caddy_test -p 80:80 -p 8448:8448 -p 443:443 -v $PWD/Caddyfile:/etc/caddy/Caddyfile caddy:2.1.1-alpine

Should be no error, I cannot reproduce your error. If the above does not have an error, replace example.com with your actual domains and try again. If still no problem, remove local_certs to use LetsEncrypt.

If no problem again, then your real Caddyfile may have a misconfiguration somewhere else, or your persisted state such as LetsEncrypt certificates might be contributing to the problem you’re experiencing.

1 Like

Thank you for your patience. It seems that my problem was the tls directive:

example.com, example.com:8448 {
    tls webmaster@example.com
}

gives the error

[ERROR]  Removing invalid block: hostname appears in more than one automation policy, making certificate management ambiguous:

while

{
    email webmaster@example.com
}

example.com, example.com:8448 {
...
}

does not

That still leaves me with the unrelated error that Caddy removes the content of the Content-Type header sent by the backend server

Ok, that was just me being unfamiliar with caddy 2 reverse proxy (/ vs /*)

I was able to reproduce your issue with local certs. Same behaviour:

local_certs:

{
  local_certs
}

# OK
example.com, example.com:8448 {
  respond "Hello"
}

tls internal:

# ERROR
example.com, example.com:8448 {
  tls internal
  respond "Hello"
}

Separate site blocks:

# ERROR
example.com {
  tls internal
  respond "Hello"
}

# ERROR
example.com:8448 {
  tls internal
  respond "Hello"
}

You should probably raise that issue on the github as a bug, or create a separate thread on the forum about it.

https://github.com/caddyserver/caddy/issues/3550 - seems to be fixed, waiting to be released

1 Like

seems to be fixed, waiting to be released

Can confirm, fixed in 2.2.0


Still my issue with having http:// and https:// failing when used on the same site-block but explicitly using the tls directive continues to fail. Since it works with implicit tls, I assume that’s a bug and Caddy should be ignoring the tls directive for http://?

@matt would you like me to raise this as an issue on Github?

No, 2.2.0 rc 2 is broken in exactly that way; already working on a hotfix.

1 Like

Try using RC 3, it should work.