Demo: run Caddy with socket activation + rootless Podman + Quadlet files

Showcase
1

I wrote some examples of how to run socket-activated Caddy with rootless Podman here:

A cool thing is that X-Forwarded-For is now supported when running Caddy with rootless Podman (using network driver Pasta). Caddy can act as a reverse proxy for a custom network (i.e. a network that was created with podman network create mynet or with a quadlet).

2 Likes
2

That’s awesome, thanks! With the official support in Caddy 2.9 I tried to get this working. While in principle it works, I could not get the default Caddy behaviour:
The ACME-HTTP-Challenge cannot be completed and shows errors in the log, but this should not be a problem as it reverts to the TLS-ALPN challenge and we only lose one option for verification.

Additionally, I could not get automatic redirects from HTTP to HTTPS. In the examples, per domain both protocols are defined in separate blocks and the action (Hello World) is duplicated there. For this definition I can get a response for http and https each and could even define different responses for each port.

If I try to include the binding for port 80 into a single domain block like for the config with standard ports, neither http nor an automatic redirect works, e.g.:

example.com {
        bind fd/3 {
		protocols h1
	}
	bind fd/4 {
		protocols h1 h2
	}
	bind fdgram/5 {
		protocols h3
	}
	log
	respond "Hello world"
}

Not a showstopper, but still a bit different from how it works without sockets, and I have no idea if I could change that via the config.

3

Thanks for the feedback! Right now I don’t have access to any linux computer with direct access to the internet so I’m not able to test letsencrypt. Unfortunately, I don’t have a good idea of what goes wrong. The other problem about redirects might be possible to to debug without direct access to the internet. I’ll try to learn more about Caddy and how redirects work. I’ll see if I can debug the problem.

4

Forgot to say: it would be interesting to know if it works without Podman. If the problems can be reproduced by just using Caddy in a systemd service then we could just create standard Caddy github issues. I reported one such issue in

5

For the record: We are running this setup now in production and it’s working fine. I could not reproduce the issue with the http-challenge, so I assume I botched the testing while following the examples :frowning: Seems like we are still losing automatic http->https redirect, but it would be good if someone could confirm this.

For now, we are explicitly defining the redirects as:

http://example.com {
  bind fd/3 {
    protocols h1
  }
  redir https://{host}{uri} permanent
}

https://example.com {
        bind fd/4 {
                protocols h1 h2
        }
        bind fdgram/5 {
                protocols h3
        }
    templates
    header Content-Type text/plain
    respond "{{.RemoteIP}}"
}
6

Thanks for the demo!

One thing I didn’t quite understand, do I need separate file descriptors for each subdomain I have in Caddy? Or is it possible to specify global file descriptors that get used for each subdomain/domain served?

7

To add more domains I think you only need to modify the file Caddyfile. There is no need to modify the file caddy.socket. The number of socket file descriptors will remain the same.

I did a test where I modified Caddyfile in Example1 so that it uses two domains test1.example.com and test2.example.com

Caddyfile

{
	admin off
}

http://test1.example.com {
	bind fd/3 {
		protocols h1
	}
	log
	respond "Hello world from test1.example.com
"
}
http://test2.example.com {
	bind fd/3 {
		protocols h1
	}
	log
	respond "Hello world from test2.example.com
"
}

It works:

$ curl -s --resolve test1.example.com:8080:127.0.0.1 http://test1.example.com:8080
Hello world from test1.example.com
$ curl -s --resolve test2.example.com:8080:127.0.0.1 http://test2.example.com:8080
Hello world from test2.example.com
$

It looks like there will be some duplication of information in Caddyfile. I don’t know if there is any way to avoid that.

1 Like
8

Thanks for your work :slight_smile: I was playing around with Example 4 and Debian testing to move my rootless nextcloud-fpm pod to quadlets and away from slirp4netns. Got everthing working except that after a short time period downloads started by a webbrowser got limited to ~2mb/s while mobile and desktop client are not affected. Restarting all containers fixed it again for a short time but I’m not sure what’s causing this problem.

9

I’ve noticed one problem. A container on the custom network can’t access the proxy, because the podman network driver pasta does not (by default) allow a container on the custom network to connect to the IP address of the host’s main network interface.

Luckily, it looks like there is a solution to the problem:
Let Caddy also listen on the custom network and add NetworkAlias= to caddy.container for every proxied domain.

I’ve implemented this in the PR

I’m half-way guessing how the Caddyfile syntax works. I hope I got it right.