Just a wild thought: to run on say a Raspberry Pi, would it be possible (and make sense) to replace process id 1 with the caddy executable? This would be just for serving static pages. The main reason for me would be to have maintenance-free https + Let’s Encrypt support. Updates would have to be done by updating the SD card on another system.
I think an Arduino would be more suitable as an embedded web server, maybe? I don’t think Pi is really geared around the idea of replacing PID 1, but it’s Linux, so you can do it with a bit of hacking, for sure.
I’d probably just strip back the services and packages until you’ve got only what you need, then set up auto-login to a service user and put Caddy in your rc.local.