Caddy Lab / Setup Examples

Yeah, basically. But there’s no better, easy, one-liner command I could find that would just do it correctly without piping back into less without the -S option. I hate journalctl's CLI interface. It’s so, so bad. You can do SYSTEMD_LESS=FRXMK journalctl -eu caddy which overrides the options passed to less to not have S, but explaining or remembering that is ridiculous.

I think you misunderstood what I was saying. I did suggest a way to do it without Let’s Encrypt.

You just never told Caddy which domain for which to issue certs for. If you just put :443 as your site address, Caddy has absolutely no information about what domains it might serve.

Either you need to configure Caddy with a site address that does have a domain/ip so that Caddy has something to issue, or use on_demand to make the issuance happen on-the-fly, during the TLS handshake.

Thanks again for the quick reply

I guess that’s a preference/application thing. – again depending on what I am looking at/for… if I can identify which line I want, or even if I know what to search for with /(string), then I have no problem with the default. I don’t know if I succeed in explaining it in my example… either way choice is better than no choice.

I’m going to circle back on this and try what you suggested for completeness.

After many hours of messing around with it, I finally found a connectivity problem… some Oracle could situation I’m having difficulty figuring out exactly how I fixed the problem well enough to document. For now I’m going to push forward with Caddy, and I’ll eventually go back and try to make a small piece on Oracle Cloud connectivity.

I hooked up a domain and it actually worked as described! I’d like to post my logs for comment/to explain what to look for, but before I do – what do I need to redact (if anything) so I don’t open up the site to hacking?

Next step is to demonstrate php – can you guide me as to which package a person would want for a typical WordPress install on Ubuntu?

I have a problem with the default, as someone who answers questions on the forums. Lots of users post truncated logs, which are entirely useless because they miss the important bits at the end, where the message typically ends up being, causing a bunch of extra back-and-forth to get on the same page.

Caddy avoids logging secrets, so probably nothing. Obviously give it a quick scan yourself if you spot anything you care about.

Just google “install php-fpm on ubuntu” and I’m sure you’ll find what you need. Then you just configure Caddy with php_fastcgi to connect to php-fpm, and that’s pretty much it.

A post was split to a new topic: Caddy + Cloudflare tunnel

A post was merged into an existing topic: Caddy + Cloudflare tunnel

@francislavoie I’ve got to say from what I’ve seen so far this is one of the best run foss software forums I have seen – you are really on top of things! Can you please move my “Thank You” post to Caddy + Cloudflare tunnel along with the post that you moved. – Thanks.

I understand completely - FYI typing ‘s’ in the default systemctl CLI gives a prompt ‘log file:’. Entering a path/file will save the command output untruncated.

Is there a good way to handle log files? They are beautifully structured for machine reading but there is unstructured text that causes jq to error out. The lines are so long that it’s really hard to read.

After sorting out the firewall issues and getting the domain working I tried to get the following to work without success:

:80   {                                                                         
    root * /usr/share/caddy                                                     
    file_server                                                                 
}                                                                               
                                                                                
                                                                                
:443   {                                                                        
    tls internal
    root * /usr/share/caddy                                                     
    file_server                                                                 
}    

http:// port 80 failed

$ curl -v --insecure -I http://132.145.103.78/
*   Trying 132.145.103.78:80...
* TCP_NODELAY set
* Connected to 132.145.103.78 (132.145.103.78) port 80 (#0)
> HEAD / HTTP/1.1
> Host: 132.145.103.78
> User-Agent: curl/7.68.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 308 Permanent Redirect
HTTP/1.1 308 Permanent Redirect
< Connection: close
Connection: close
< Location: https://132.145.103.78/
Location: https://132.145.103.78/
< Server: Caddy
Server: Caddy
< Date: Tue, 10 May 2022 10:14:30 GMT
Date: Tue, 10 May 2022 10:14:30 GMT
< 
* Closing connection 0

And so dit port 443:

$ curl -v --insecure -I https://132.145.103.78/
*   Trying 132.145.103.78:443...
* TCP_NODELAY set
* Connected to 132.145.103.78 (132.145.103.78) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS alert, internal error (592):
* error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
* Closing connection 0
curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error

Any ideas?

@francislavoie The forum won’t let me edit/add to my initial post - too old / can’t edit or delete.

I tried to add the following:

  1. When Caddy is configured to use a domain, https is served by default. For this example, I have registered the domain *caddytest.cf". To make sure that the DNS has propagated and that an A record has been properly configured the dig command can be run on the server:
    .
    dig caddytest.cf A
; <<>> DiG 9.16.1-Ubuntu <<>> caddytest.cf
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10382
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;caddytest.cf.			IN	A

;; ANSWER SECTION:
caddytest.cf.		300	IN	A	132.145.103.78

;; Query time: 144 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Tue May 10 08:40:19 UTC 2022
;; MSG SIZE  rcvd: 57

The line caddytest.cf. 300 IN A 132.145.103.78 confirms that the DNS has been correctly configured and has propagated.

  1. To make Caddy serve the domain, all that is necessary is to change the line :80 { to caddytest.cf: { and to reload Caddy with ‘sudo systemctl reload caddy
    .
    Visiting https://caddytest.cf/ should cause the Caddy “Congratulations” page to be displayed.
    .
    The command sudo journalctl -u caddy should display a log file. If you are having problems it is the first place to look for answers. Examining the log is also a good way to find the location of important directories used by Caddy.
-- Logs begin at Sun 2022-05-08 20:05:38 UTC, end at Mon 2022-05-09 21:58:17 UTC. --
May 09 00:52:59 caddytest-20220508-1602 systemd[1]: Starting Caddy...
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: caddy.HomeDir=/var/lib/caddy
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: caddy.AppDataDir=/var/lib/caddy/.local/share/caddy
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: caddy.AppConfigDir=/var/lib/caddy/.config/caddy
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: caddy.ConfigAutosavePath=/var/lib/caddy/.config/caddy/autosave.json
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: caddy.Version=v2.5.1 h1:bAWwslD1jNeCzDa+jDCNwb8M3UJ2tPa8UZFFzPVmGKs=
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: runtime.GOOS=linux
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: runtime.GOARCH=amd64
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: runtime.Compiler=gc
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: runtime.NumCPU=2
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: runtime.GOMAXPROCS=2
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: runtime.Version=go1.18.1
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: os.Getwd=/
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: LANG=C.UTF-8
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: NOTIFY_SOCKET=/run/systemd/notify
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: HOME=/var/lib/caddy
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: LOGNAME=caddy
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: USER=caddy
May 09 00:52:59 caddytest-20220508-1602 caddy[2010]: INVOCATION_ID=b5bed0f1f71b42929c755713c179bd29

I initially had a firewall issue, and the following lines appeared in the log:

May 09 20:44:28 caddytest-20220508-1602 caddy[2010]: {"level":"error","ts":1652129068.489746,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"caddytest.cf","challenge_type":"http-01","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"132.145.103.78: Fetching http://caddytest.cf/.well-known/acme-challenge/5vtpluZ1BYU62rwnV-erenpIUoyWx0-jTaZ6lXlDaiU: Timeout during connect (likely firewall problem)","instance":"","subproblems":[]}}
May 09 20:44:28 caddytest-20220508-1602 caddy[2010]: {"level":"error","ts":1652129068.4897828,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"caddytest.cf","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"132.145.103.78: Fetching http://caddytest.cf/.well-known/acme-challenge/5vtpluZ1BYU62rwnV-erenpIUoyWx0-jTaZ6lXlDaiU: Timeout during connect (likely firewall problem)","instance":"","subproblems":[]},"order":"https://acme-v02.api.letsencrypt.org/acme/order/536049396/87239794716","attempt":2,"max_attempts":3}
May 09 20:44:28 caddytest-20220508-1602 caddy[2010]: {"level":"error","ts":1652129068.4898076,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"caddytest.cf","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:connection - 132.145.103.78: Fetching http://caddytest.cf/.well-known/acme-challenge/5vtpluZ1BYU62rwnV-erenpIUoyWx0-jTaZ6lXlDaiU: Timeout during connect (likely firewall problem)"}

Typing ‘/error’ while in the log viewer will find lines with an error. You may need to use the arrow keys to scroll across the line to see the text located.

If a certificate was successfully obtained, typiing ‘/success’ should find the line(s) confirming successful certificate creation/acquisition.

May 09 21:42:29 caddytest-20220508-1602 caddy[5508]: {"level":"info","ts":1652132549.736588,"logger":"tls.issuance.acme.acme_client","msg":"successfully downloaded available certificate chains","count":1,"first_url":"https://acme.zerossl.com/v2/DV90/cert/ZNlj7N2rtt9Z_Zo-gLHXmQ"}
May 09 21:42:29 caddytest-20220508-1602 caddy[5508]: {"level":"info","ts":1652132549.736983,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"caddytest.cf"}

I guess just use pipe it through grep to keep only lines starting with { then jq should be more useful.

Again – you haven’t given it a domain name to use!!

Either give it a domain name, or turn on on_demand.

tls internal {
	on_demand
}

Thanks… I didn’t understand without the complete syntax. Now I have it.

OBSERVATION

An observation (NOT a complaint, but hopefully useful feedback for helpers/documentaton writers)
For me “lack of context” or mixing back and when documentation switches between JSON and Caddyfile format is a real barrier to understanding. I get that the JSON config is super cool for automation and very necessary especially with cloud technology - my question is if I’m just trying to deploy something simple, how do I integrate a JSON example into my Caddyfile?

Example Getting Started — Caddy Documentation … It starts with some simple examples of downloading the binary and serving files on localhost. (A good use case – however if you do the Install as per the installation instructions, systemctl should be used instead of direct caddy commands). Then we jump right into an API/JSON example and that’s supposed to be my “First Config” - What??? I tried to use Caddy when V1 came out and that documentaton scared me away.

And this Testing Section Automatic HTTPS — Caddy Documentation - I had no idea what to do with this statement https://acme-staging-v02.api.letsencrypt.org/directory - I think it took me about a half an hour to find an example that put it into context.

All this to say I think more relevant examples in the documentation would go a long way to making things more understandable (and separate the API from the Caddyfile examples (or link them when the same issue is solved both ways–which in most cases should be done.)
Comments welcome - is this just me, or are others having the same issue?

Again, much love for all – not a complaint – just doing this example has shown me how hard writing documentation is!

My goal is to create a good clear example that can be worked though and leave the user with enough understanding to build static sites.

QUESTIONS:

Since you do a lot of helping maybe you could tell me if there are any common problems that you are seeing that my example would not address so far?

This successfully tests port 80 redirection to port 443 with a self-signed certificate.

:80   {
    root * /usr/share/caddy
    file_server
}


:443   {
    tls internal {
        on_demand
    }
    root * /usr/share/caddy
    file_server
}

If I wanted :80 not to reroute, what would I change?

Once I get answers I integrate them into the example and clean it up some more.

Understanding that the underlying config is JSON is pretty fundamental to understanding how Caddy works.

The point of the Getting Started pages is to give you a quick tour of the fundamental concepts behind Caddy so then you can make an informed choice of whether you want to use a Caddyfile, or JSON directly.

This isn’t a high barrier to entry IMO, you need to at least understand how to use your computer and a terminal.

Do you mean v2? Caddy v1 didn’t have JSON config.

Yeah, that’s fair. We could clarify there that for the Caddyfile, the acme_ca global option is what you should use when testing.

Generally though, we try to avoid specific config examples in pages/articles that explain concepts.

We have examples all over the reference docs for each Caddyfile directive.

Where are you seeing this, exactly? If you’re talking about Getting Started, well… that’s the entire point. Under the hood, every time you use caddy run or caddy reload, you are using the API, just wrapped up behind a CLI command.

For sure :+1: that’s our goal too, but every user has different needs, and we’re trying to cater to everyone at the same time.

You’re not doing any redirection here, because you’re overriding :80 with that config to serve static files.

I strongly suggest not to override redirects, you should only allow HTTPS (and HTTP only for ACME HTTP challenges and HTTP->HTTPS redirects, both of which Caddy’s Automatic HTTPS sets up for you).

I really cannot recommend using this kind of config other than for toying around. This opens you up to DDoS, because someone could just point a wildcard domain to your server then make requests like https://a.badguy.com, then https://b.badguy.com and so on to infinity, until you hit rate limits, or worse, your server’s disk storage is filled up with garbage TLS certs and keys.

You must configure an ask endpoint to safely use on_demand to avoid this, by only allowing domains you know to trigger fetching a cert.

Like I said before, you really should configure actual names in your config, instead of :443.

1 Like

Thanks again for taking the time to reply.

I think you may be a bit too close to see it–once you :“get it” - it’s easy - a bit like the Rubic’s Cube!. Solve in under a minute if you are an expert! Struggle for hours until you do. A well written tutorial will greatly speed up the journey to expert.

I have been almost exclusively using Linux since about 2018, and dabbled with some shell accounts and Linux VMs since 2014. I’ve set up FreeNAS (now TrueNAS) and pfSense and interact on the shell with both of them to automate custom processes. I was able to get going in Docker more easily than Caddy due to the presence of good examples that were close to my use cases. A block of code can often be worth a 1000 words (or more!)

I think that should qualify me as knowing how to use my computer – and having said that I’m brand new to cloud dev ops. First experience with a cloud server was about 2 weeks ago. I am maybe going a bit further with my than I should because I can remember the day when it would have made a big difference,

Having said that though, the documentation that is there is very well written and an excellent reference once you have enough context. The reason I am spending so much time on Caddy is that I can see the potential… once you know what you are doing you can get a lot done very very quickly, easily and with less resources. I know elegant design when I see it even if I can’t yet figure it all out. @matt has really thought this out well.

That’s sort of the problem… the guy who wants to set up an automated K8S cluster and small business who wants to set up a VPS to serve the company website are going to use very different approaches. The K8S dev isn’t likely going to care too much about the Caddyfile, and the small website builder doesn’t likely want to have to deal with the JSON/API.

“Getting started” is the small website / small proxy server. From what I have seen so far, I think 85-90% of the website on small servers would be better off with Caddy rather than Nginx or Apache.
The path of least resistance is likely still Apache/Nginx because there are many step-by-step tutorials on how to set it up! I found myself looking at a very detailed “Set up WordPress / PHP on Nginx” tutorial - showed exactly what packages to install – which should work for Caddy.

Then maybe a “Getting Started with the API” for an Advanced User. I did eventually trip over something like that on the site, but it took me awhile to discover it.

Anyway thanks for engaging, let me see what I can create with your help and how the community responds to it. I’m most grateful for everything so I’m not going to press this point any further.

I want to make it 100% clear the only reason I am asking about this setup is for TROUBLESHOOTING or testing behind the firewall. All 4 of the following must work for https pages to be served:

  1. Traffic getting to ports 80/443
  2. Is Caddy running properly on the server / Install worked - right package / directories correct/writeable.
  3. Caddy configuration done correctly
  4. Let’s Encrypt working properly / Correct certificates available.

On my first attempts to set up a server I had a problem with step 1 (port 443 was being blocked). I thought my problem was with 3/4. If I had a quick way to verify the steps in order I would have saved several hours. Since this was my first time setting things up on a cloud environment that I was using for the first time, all 4 steps were unfamiliar ( I am still looking for a quick way to verify #1 that doesn’t depend on the later steps. ) Because of the redirection the output from curl didn’t really show what the true problem was. The :80{ :443 { configs are an attempt to test #1/#2.

This Caddyfile does redirect to https as can be seen with curl.

$ curl -v http://132.145.103.78
*   Trying 132.145.103.78:80...
* TCP_NODELAY set
* Connected to 132.145.103.78 (132.145.103.78) port 80 (#0)
> GET / HTTP/1.1
> Host: 132.145.103.78
> User-Agent: curl/7.68.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://132.145.103.78/
< Server: Caddy
< Date: Wed, 11 May 2022 00:54:36 GMT
< Content-Length: 0
< 
* Closing connection 0

and it it correctly produces a NET::ERR_CERT_AUTHORITY_INVALID which can be bypassed on a web browser to display the sample site.

I would like to have an example that serves port 80 in plain text and 443 as https – again just for testing. That will tick off #1/#2 and sort of touch on #3/#4. Once this all works everything should be in the domain of #3.

Thanks for mentioning this… I will be sure to highlight … Again my intent is either for troubleshooting or quick proof of concept in test environments behind a firewall.

Just to clarify it doesn’t fetch a cert-it creaties self-singed cert internally (without going to the internet?).

I did my best to imitate the scenario you described (I have 5 caddytest domains registered from FreeNOM which are pointed at my test VPS.) . I hit the site with http://132.145.103.78 and from 4 different domains. I was surprised that this is the only log traffic I got:
(I expected to see the other domain names in the log file-why not?)

May 11 01:24:16 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652232256.5573242,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"132.145.103.78:443"}
May 11 01:24:16 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652232256.5573611,"logger":"tls.obtain","msg":"releasing lock","identifier":"132.145.103.78:443"}
May 11 01:24:16 caddytest-20220508-1602 caddy[7091]: {"level":"warn","ts":1652232256.5577471,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [132.145.103.78:443]: no OCSP server specified in certificate","identifiers":["132.145.103.78:443"]}
May 11 01:27:52 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652232472.2923038,"logger":"tls.on_demand","msg":"obtaining new certificate","server_name":"132.145.103.78"}
May 11 01:27:52 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652232472.2925432,"logger":"tls.obtain","msg":"acquiring lock","identifier":"132.145.103.78"}
May 11 01:27:52 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652232472.2961135,"logger":"tls.obtain","msg":"lock acquired","identifier":"132.145.103.78"}
May 11 01:27:52 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652232472.2973058,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"132.145.103.78"}
May 11 01:27:52 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652232472.2973285,"logger":"tls.obtain","msg":"releasing lock","identifier":"132.145.103.78"}
May 11 01:27:52 caddytest-20220508-1602 caddy[7091]: {"level":"warn","ts":1652232472.2976415,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [132.145.103.78]: no OCSP server specified in certificate","identifiers":["132.145.103.78"]}
May 11 01:32:09 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652232729.2993627,"logger":"tls.on_demand","msg":"attempting certificate renewal","server_name":"10.0.0.158","subjects":["10.0.0.158"],"expiration":1652247051,"remaining":14321.700641914,"revoked":false}
May 11 01:32:09 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652232729.2998455,"logger":"tls.renew","msg":"acquiring lock","identifier":"10.0.0.158"}
May 11 01:32:09 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652232729.3019702,"logger":"tls.renew","msg":"lock acquired","identifier":"10.0.0.158"}
May 11 01:32:09 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652232729.3022192,"logger":"tls.renew","msg":"renewing certificate","identifier":"10.0.0.158","remaining":14321.697781807}
May 11 01:32:09 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652232729.3033087,"logger":"tls.renew","msg":"certificate renewed successfully","identifier":"10.0.0.158"}
May 11 01:32:09 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652232729.3033264,"logger":"tls.renew","msg":"releasing lock","identifier":"10.0.0.158"}
May 11 01:32:09 caddytest-20220508-1602 caddy[7091]: {"level":"warn","ts":1652232729.303651,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [10.0.0.158]: no OCSP server specified in certificate","identifiers":["10.0.0.158"]}
May 11 01:32:09 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652232729.303676,"logger":"tls.cache","msg":"replaced certificate in cache","subjects":["10.0.0.158"],"new_expiration":1652275929}
May 11 02:25:31 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652235931.350404,"logger":"tls.on_demand","msg":"obtaining new certificate","server_name":"www.caddytest.ga"}
May 11 02:25:31 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652235931.3506382,"logger":"tls.obtain","msg":"acquiring lock","identifier":"www.caddytest.ga"}
May 11 02:25:31 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652235931.3540866,"logger":"tls.obtain","msg":"lock acquired","identifier":"www.caddytest.ga"}
May 11 02:25:31 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652235931.3553138,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"www.caddytest.ga"}
May 11 02:25:31 caddytest-20220508-1602 caddy[7091]: {"level":"info","ts":1652235931.3553362,"logger":"tls.obtain","msg":"releasing lock","identifier":"www.caddytest.ga"}
May 11 02:25:31 caddytest-20220508-1602 caddy[7091]: {"level":"warn","ts":1652235931.3556552,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [www.caddytest.ga]: no OCSP server specified in certificate","identifiers":["www.caddytest.ga"]}

As you said certificates we generated for each domain that were signed by Caddy’s internal cert.

Is there anything wrong with using tls internal { on_demand } behind a reverse proxy that has no direct internet connection? I would think that might be better than open http.
(I’m talking in a SOHO environment where everything is on 1 or at most 2 computers and it’s not really practical to setup a PKI server (which would be the way to do things in a larger environment.)

IIUC that would still create an encrypted connection and unless a machine was compromised there wouldn’t be anyone listening anyway.

Is there a better way to handle this “behind the reverse-proxy/no domain name” for a small installation?

@francislavoie Are there any other places in the setup that you find people getting stuck/lost that I should highlight?

That exists, though!

There’s a difference between “Getting Started” and “Tutorial”.

The Getting Started page is to help you decide “what kind of user am I?”.

I don’t see the problem at all with that page. It’s very simple. Shows you a super quick tour of the fundamental concepts. Start Caddy, see there’s an API, do a JSON hello world, do a Caddyfile hello world. Cool, now you know how to configure Caddy at a high level, so which one’s for you?

I think every new user needs to see this, to at least have a chance to choose for themselves what path they want to go with. It doesn’t make sense to split this up, because this is the entrypoint for new users.

From that point on, you then move onto the relevant tutorial for the way you want to use it.

What I’m trying to say is I’m not really hearing actionable feedback right now. If you have any specific suggestions to improve the flow, you can propose changes on the website repo (the docs site is open source):

Oh right – that’s cause we sort http:// or :80 routes (i.e. ones with no host matcher) to the end, and put the redirect route before it. Point is, your :80 block does nothing at all in this case. You can remove it.

If you actually want to serve content over http, then you’ll need to either specify an actual host, or configure the global option auto_https disable_redirects to get rid of the redirect.

But I don’t think you really need that. If you can see Server: Caddy and you see a redirect when connecting over HTTP, then everything’s good to go.

Yes. Caddy generates a local CA, using GitHub - smallstep/certificates: 🛡️ A private certificate authority (X.509 & SSH) & ACME server for secure automated certificate management, so you can use TLS everywhere & SSO for SSH. under the hood, and then uses that CA to generate your leaf certs.

I’m not sure I understand. I see domains:

That’s fine for testing as long as nobody from the internet can hit it and abuse it.

I’m not sure I understand the question

1 Like

I get you… let me think about it… let me see if I can get this tutorial sorted out - if I do it right it should address the basic “getting strated” problems.

Let me try asking a differnt way. How would you attach a couple of sites running on a mchine inside the firewall to a reverse proxy located at the edge? Assume that the route between the two machines was a dedicated 802.1Q tagged VLAN in a small office (actually it’s my home, but for illustration I’d like to raise the threat potential just slightly.)

Have a look at this curl output from earlier in the thread… would you interpret this as a problem with port 443 not being open. I didn’t… and I wasted a heap of time looking for the problem in the wrong place.

$ curl -v --insecure -I https://132.145.103.78/
*   Trying 132.145.103.78:443...
* TCP_NODELAY set
* Connected to 132.145.103.78 (132.145.103.78) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS alert, internal error (592):
* error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
* Closing connection 0
curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error

I just went back to the first server I tried to set up, and there were some issues. Having this bit of code:

:80   {
    root * /usr/share/caddy
    file_server
}


:443   {
    tls internal {
        on_demand
    }
    root * /usr/share/caddy
    file_server
}

helped me get the problem solved in about 2 minutes. I’ve got a few firewalls to get through, and it’s easy to make a mistake and miss something.

Next step redirecting http(s)://www.caddytest.cf → https://caddytest.cf

This is the Caddyfile I tried - I think I know why it didn’t work, but I have no clue how to fix it?
It looks like {host} includes the www. - have I got that right? If so, how do I get rid of it.
I notice from the logs that I got a cert for www.caddytest.cf, which is a good start.

caddytest.cf:   {                                                              
   # Set this path to your site's directory.                                   
   root * /usr/share/caddy                                                     
                                                                               
   # Enable the static file server.                                            
   file_server                                                                 
                                                                               
   log {
       output file /var/lib/caddy/logs/caddytest.cf.log
   }                                    
}                                                                              

www.caddytest.cf:   {                                                              
   # Redirect www
   redir https://{host}{uri}
}

This is the output I got when I hit the url with curl.

$ curl -v https://www.caddytest.cf/
*   Trying 132.145.103.78:443...
* TCP_NODELAY set
* Connected to www.caddytest.cf (132.145.103.78) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=www.caddytest.cf
*  start date: May 11 08:08:21 2022 GMT
*  expire date: Aug  9 08:08:20 2022 GMT
*  subjectAltName: host "www.caddytest.cf" matched cert's "www.caddytest.cf"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x5562167122f0)
> GET / HTTP/2
> Host: www.caddytest.cf
> user-agent: curl/7.68.0
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 302 
< location: https://www.caddytest.cf/
< server: Caddy
< content-length: 0
< date: Wed, 11 May 2022 09:14:00 GMT
< 
* Connection #0 to host www.caddytest.cf left intact

Any other directive you think I should include for a production static?

Is it possible to proxy port 443 on a subdomain directly without caddy handling the https?

The use case would be a web server in the cloud to serve my web site, and a subdomain to proxy a connection to my home office through a tunnel. I would likely do some geographic firewalling on the external server, but then route the traffic through an ssh tunnel to a reverse proxy on my pfSense firewall which has access control. That way if the cloud server gets broken into, it’s not really any greater exposure than having the reverse proxy open to the internet. The advantage of doing that is that I don’t draw attention to the fact that I’m running a server on my home internet connection. All the ISP sees is the tunnel to the cloud. I’m thinking that I would likely use s client cert and a password for access control.

Like this:

app1.example.com {
	reverse_proxy 192.168.1.10:8001
}

app2.example.com {
	reverse_proxy 192.168.1.10:8002
}

I’ve never heard of this term before :man_shrugging:

It says “connected”, so no. Was your firewall allowing the connection but preventing traffic, or something? :astonished:

You have : in there after the domains. Remove that.

Yes, this uses the incoming request’s Host there.

You can either just explicitly write https://caddytest.cf{uri} or use labels placeholders like https://{labels.1}.{labels.0}{uri} where a label is the segments of the hostname starting from the right.

We have an example in the docs that shows how to do this (but while typing this up I realized there was a small mistake, the . was missing between the label placeholders – fixing it here docs: Changes for v2.5.1 release by francislavoie · Pull Request #228 · caddyserver/website · GitHub):

Depends. You might need try_files if your site is JS with a frontend router (react-router or something), and probably should enable encode gzip for compression. You might need CORS headers depending on what you’re doing, etc. It’s all very usecase-specific.

You’re talking about a TCP proxy – Caddy’s default build doesn’t do this, because it’s an HTTP layer proxy.

You can use GitHub - mholt/caddy-l4: Layer 4 (TCP/UDP) app for Caddy if you need a TCP/UDP proxy.

@francislavoie Thanks again…

OOPS… I didn’t ask the question clearly… I meant on the back end receiving proxy connections (maybe from Caddy, but could be any reverse-proxy running on another machine). I’m assuming the machines are connected by a VLAN with no other tenants using the VLAN.

I found this writeup:

Can Caddy be on multiple IP networks simultaneously - One for the public side, and one for the private side?

Imagine the router being 192.168.1.1, and the public side of the frontend being 192.168.1.2. The private side of the front end would be 192.168.0.1 and 192.168.0.2 which would connect to the front ends on 192.168.0.3 and 192.168.0.4. The 192.168.0.0/24 subnet would be in a VLAN, so the likelyhood of anyone being able to snoop would be negligivle, but even if it was possible to snoop (by taking over the switch or compromising the front end), the traffic would still be encrypted by using:

:443 {
tls internal {
on_demand
}

wouldn’t it? I can see a more hardened environment for generating certificates if the environment was a cloud platform with multiple tenants, but in a home environment? Am I missing something?

Thanks… your solution was perfect!

802.1Q tagged VLAN is what most managed switches use. They add a tag to the beginning of the frames to segregate traffic. If the network is configured properly it allows multiple broadcast domains on the same physical network with complete isolation as if they were separate networks. If someone is able to break out of a VLAN, they likey own your whole network. If that happens I don’t think a more robust certificate generation strategy will make much difference

:100: :+1:

I’ll see if I can figure these out… I must say that I’m a bit confused about try_files. I looked at the docs but without more robust examples I don’t get it.

I guess that would mean it would be necessary to build/compile Caddy?

END of reply

I managed to find this:
How to Install and Configure Caddy Web Server with PHP and MariaDB on Ubuntu 20.04

and get PHP/MariaDB working – at least the test page displayed properly and it was possible to connect to the database. This is an example of a great tutorial! Easy to follow and it worked!

I went though php_fastcgi (Caddyfile directive) — Caddy Documentation and I must say that I was totally confused by the references to port 9000. Is this an obsolete method of installing PHP-FPM? When I installed PHP using apt, the resulting install used UNIX sockets. There are examples scattered though the Caddy site using :9000, and some using the UNIX sockets method and no background or explanation of when to use what.

I’m just about ready to clean this thread up and move it to the wiki as a tutorial. I assume it uses the same sort of markdown as the forum? Is there an easy way for me to get all the text our, or do I have to go in small bits and reformat. I know some of the posts are locked from editing so I cant open an edit window and scrape.

I’m wondering if you have had any experience with client side certificates? For private self hosted sites they seem like a good idea for access control. IIUC the site won’t even connect unless the client has a valid cert. If the CA is inside your firewall it would be impossible for anyone to generate a cert unless they had already been inside and stolen a cert. Back that up with passwords and you should have pretty good security. Any idea if Androd handles client certs properly?

BTW… do you use any kind of private CA for your home machines? How do you manage the security of your private keys?

I still don’t really know what you mean.

Yes – by default Caddy binds to all interfaces by default, but you can configure that with the bind directive to serve different sites from different interfaces.

Basically it’s “try to see if a file exists for this pattern; if it does, rewrite the URL to that; if not, try the next one (until none are left)”.

I think the best explanation I’ve written about it is in the php_fastcgi docs, since it’s a usecase specific to modern PHP apps:

For JS apps, SPAs (Single-Page Apps), it’s often necessary to do something like try_files {path} /index.html so any request to a path that doesn’t exist on disk instead just serves the index.

Yes. But it’s very easy. Either use the Download Caddy page to grab a custom build, or install Go (super easy, works on every OS) and xcaddy (single static binary, just like Caddy) which automates the process of producing a build Build from source — Caddy Documentation

Note that caddy-l4 does not work with Caddyfile config at this time, you’d need to use JSON config.

It’s not obsolete. It’s done that way a lot of the time. For example in Docker, it’s the easiest way to connect a web server container to the separate FPM container.

What you should use entirely depends on how PHP-FPM was installed. Some distros will use TCP by default, others will use unix sockets. Caddy supports both, you just need to figure out which one to use according to your environment.

The wiki is literally just another topic/thread, but in the Wiki category, and with a flag turned on so anyone can edit the post.

You should be able to just copy (Ctrl+C) the content you wrote and paste it into the editor box in the other topic. It should preserve formatting.

Yep, Caddy supports client certs, see the tls directive.

Caddy doesn’t manage issuance of client certs though, so that’s up to you to manage. Just plug in your root cert into Caddy’s config and it’ll restrict access to clients who present a trusted client cert.

But client certs can be annoying to deal with cause it requires manual steps to install on devices. It makes sense if you’re tech savvy enough and don’t mind the rigamarole.

Well, when you use tls internal, you are using a “private CA”, managed by Caddy, which it uses to issue server certs as needed. Just make sure nobody else has access Caddy’s storage location.

Let me try to explain with this crude Ascii Diagram (forget the other example). BackEnd1/2 are 2 Caddy servers, and FrontEnd is a third server to handle guarding the edge.

 BackEnd 1      |         Front End            |  Router    |
192.168.0.3------192.168.0.2-|
                             |
                             |--192.168.1.2-----192.168.1.1
 BackEnd 2                   |
192.168.0.4------192.168.0.1-|
    VLAN NO OTHER DEVICES 

Front end is bound to 2 interfaces - 1 on the front end and 1 on the back end .

I don’t see any reason for BACKEND1 or BACKEND2 (2 Separate machines or containers) to use anything more complicated than:

:443 { tls internal {
on_demand
}

BackEnd1/2 don’t need direct access to the internet, and 192.168.0.0/24 has only the two backend machines and the back end of the FrontEnd to worry about. Traffic is still encrypted with Caddy’s self signed certs so casual traffic snooping isn’t possible. Am I missing something, or does adding the complexity of Let’s Encrypt add any value? I think it just creates additional attack vectors if something isn’t set up correctly or otherwise goes wrong. Am I missing something?

Another question just occurred to me. How many certs are required by the front end? Can the same cert be used for 192.168.0.1, 192.168.0.2, and 192.168.1.2?

Thnaks for clarifying… If setting up on an unfamiliar distro any easy way to figure out what method is being used other than hoping it’s easy to find in the docs? (I’m sure I’m showing my ignorance here, but the relevent Caddy example is to just run journalctl -u caddy right after startup and all the file paths are right there.

Agreed – just wondering if you’ve ever done it with Android? Does android work properly/

I’ve used certs with OpenVPN on pfSense and they are great. With a UDP server and a TLS Auth cert the server ignores any probe attempts. You need the right key to the get the opportunity to provide a USER key, and then you enter your user password. pfSense has a nice program that exports a cert bundle that is very easy to import into OpenVPN. It would be nice to be able to do that for private web sites run on Caddy.

Okay – what’s running on the frontend though? Another Caddy instance? That’s what I was missing from your explanation.

Probably not for you in this case. And it wouldn’t add any attack vectors, I don’t see how it could.

But a better approach is probably to have the frontend act as the ACME CA for your backends, and do mutual TLS (mTLS) between them. This wiki explains it well, as I think you’ve seen:

The benefit of this is trust, and you’re only managing a single CA instead of a CA per backend, of which you need to copy the root cert to the frontend.

Caddy doesn’t support multi-SAN certs (i.e. multiple names in one cert), so it’ll be a cert per name you’re using.

Check the php-fpm config. It’ll have a listen = line which tells you how it’s running. Or check the systemd service for php-fpm and it should probably tell you as well in its logs.

Yes it works properly.