Same subdomain-name on A LOT of domains

Help
1

1. The problem I’m having:

Hello everyone,
I am trying to match certain names to several internal domains. For example:

  • test.domain1.lan, test.domain2.lan, test.domain3.lan, test.domain4.lan , … → goto reverseproxy1
  • foo.domain1.lan, foo.domain2.lan, foo.domain3.lan, foo.domain4.lan, … → goto reverseproxy2
  • bar.domain1.lan, bar.domain2.lan, bar.domain3.lan, bar.domain4.lan, … → goto reverseproxy3

I tried a wildcard matcher like this in the caddyfile, but it does not work unfortunately:

test.* {
	tls internal
	reverse_proxy http://censored:1234
}

foo.* {
	tls internal
	reverse_proxy http://censored:1235
}

bar.* {
	tls internal
	reverse_proxy http://censored:1236
}

2. Error messages and/or full log output:

Logs say there is no certificate …

{"level":"debug","ts":1707185587.665784,"logger":"events","msg":"event","name":"tls_get_certificate","id":"0017cedd-6ebd-45dd-becc-cb4ca8220a82","origin":"tls","data":{"client_hello":{"CipherSuites":[4865,4867,4866,49195,49199,52393,52392,49196,49200,49162,49161,49171,49172,156,157,47,53],"ServerName":"test.server.domain1.lan","SupportedCurves":[29,23,24,25,256,257],"SupportedPoints":"AA==","SignatureSchemes":[1027,1283,1539,2052,2053,2054,1025,1281,1537,515,513],"SupportedProtos":["h2","http/1.1"],"SupportedVersions":[772,771],"RemoteAddr":{"IP":"192.168.1.101","Port":63019,"Zone":""},"LocalAddr":{"IP":"172.17.0.22","Port":443,"Zone":""}}}}
{"level":"debug","ts":1707185587.6659157,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"test.server.domain1.lan"}
{"level":"debug","ts":1707185587.6659262,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.server.domain1.lan"}
{"level":"debug","ts":1707185587.6659327,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.*.domain1.lan"}
{"level":"debug","ts":1707185587.6659381,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.*.*.lan"}
{"level":"debug","ts":1707185587.6659439,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"*.*.*.*"}
{"level":"debug","ts":1707185587.6659596,"logger":"tls.handshake","msg":"no certificate matching TLS ClientHello","remote_ip":"192.168.1.101","remote_port":"63019","server_name":"test.server.domain1.lan","remote":"192.168.1.101:63019","identifier":"test.server.domain1.lan","cipher_suites":[4865,4867,4866,49195,49199,52393,52392,49196,49200,49162,49161,49171,49172,156,157,47,53],"cert_cache_fill":0,"load_or_obtain_if_necessary":true,"on_demand":false}
{"level":"debug","ts":1707185587.6661525,"logger":"http.stdlib","msg":"http: TLS handshake error from 192.168.1.101:63019: no certificate available for 'test.server.domain1.lan'"}

… but i used tls internal … and it does work, when i use the full qualifier in the caddyfile like so:

test1.server.domain1.lan {
	tls internal
	reverse_proxy http://censored:1234
}

3. Caddy version:

v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A=

4. How I installed and ran Caddy:

a. System environment:

Linux SERVER 6.1.38-Unraid

b. Command:

I run caddy as docker. Repository: caddy:alpine

c. Service/unit/compose file:

//

d. My complete Caddy config:

// see above

5. Something else?

I know i could do something like this …

test1.server.domain1.lan, test1.server.domain2.lan, test1.server.domain3.lan, test1.server.domain4.lan, test1.server.domain5.lan  {
	tls internal
	reverse_proxy http://censored:1234
}

foo.server.domain1.lan, foo.server.domain2.lan, foo.server.domain3.lan, foo.server.domain4.lan, foo.server.domain5.lan   {
	tls internal
	reverse_proxy http://censored:1235
}

bar.server.domain1.lan, bar.server.domain2.lan, bar.server.domain3.lan, bar.server.domain4.lan, bar.server.domain5.lan {
	tls internal
	reverse_proxy http://censored:1236
}

… but this gets cumbersome. There has to be a more generic way of doing that, right? I want to match several “subnames” to several “domains”, but every “subname” is resolving to the same reverseproxy. I hope it is clear what i mean.

Hope someone can help. Best regards.

2

This should do:

https:// {
	tls internal {
		on_demand
	}

	@test header_regexp domain Host ^test\.(.*)$
	handle @test {
		reverse_proxy censored:1234
	}

	@foo header_regexp domain Host ^foo\.(.*)$
	handle @foo {
		reverse_proxy censored:1235
	}

	@bar header_regexp domain Host ^bar\.(.*)$
	handle @bar {
		reverse_proxy censored:1236
	}
}
1 Like
3

Wow, thank you. I would have never been able to come up with this.

Now, this does work flawlessly (big thank you!!!). However I would have some questions about this setup, as it is confusing me a bit, and I would have to extend on it:

  • this does now match on ALL domains. → is there a way to get foo only on certain domains like domainX.lan, domainY.lan? I would like do define three sets of domains and certain names go with certain domain-sets but not others, so to speak.

  • I would also have some external domains (that would exactly match with the whole fully qualified dns name) which are not tls internal. can this be done since in this setup the whole https:// is tls internal if i get this correctly. i can not make a second https:// block, can i?

  • does the https:// block mean that as soon as a client requests https traffic, it goes into this block? I would need some exact dns-name matches, that are not tls internal

  • what happens if a user is asking http traffic on this domains? does it automatically redirect to https? i tested it and it seems so, but i am not quite sure how this works.

  • on hostnames not matched with one of those, caddy is returning a blank page. can it be configured to ignore the request?

4

Yeah, just change the regexp to only match those domains. Something like:

Host ^foo\.(domain1\.com|domain2\.com)$

There’s no “simple” way to do that.

You’d have to have two separate site blocks, and at least one of them listing all the domains you want to allow. It’s necessary so that Caddy can match which TLS connection policy to use etc.

It’s a catch-all/fallback. If you define a more specific site block with specific domains, those will take priority if an incoming TLS-SNI matches.

Yes, the feature called “Automatic HTTPS” adds an HTTP server and routes for HTTP->HTTPS redirects, unless you disable that explicitly or provide your own http://<domain> site block to do whatever routing.

Yeah, just add a handle { } block at the end with no matcher, which acts as a fallback. See Common Caddyfile Patterns — Caddy Documentation for example

1 Like
5

Is there a way to save those in a variable and isert it there?
I am thinking in bash terms like: DOMAINS="(domain1\.com|domain2\.com)" and later doing something like @foo header_regexp domain Host ^foo\.$DOMAINS$

That would be awesome for my usecase!

Thank you again for all your input.
When I’m combining all the information then I assume something like this could work, no?

# External Domains
a.realdomain.com {
    reverse_proxy 10.0.10.254:80
}

b.realdomain.com {
    reverse_proxy 10.0.10.254:8080
}


# Internal Domains
https:// {
	tls internal {
		on_demand
	}

	@test header_regexp domain Host ^test\.(domain1.lan|domain2.lan|domain3.lan)$
	handle @test {
		reverse_proxy 10.0.10.222:1234
	}

	@foo header_regexp domain Host ^foo\.(domain1.lan|domain5.lan|domain6.lan)$
	handle @foo {
		reverse_proxy 10.0.10.222:1235
	}

	@bar header_regexp domain Host ^bar\.(domain1.lan|domain2.lan|domain3.lan)$
	handle @bar {
		reverse_proxy 10.0.10.222:1236
	}
}

# Fallback
handle {
}

If I understand correctly:

  • Caddy would go for the more specific domains first. In this case a.realdomain.com and b.realdomain.com
  • If the connection is not requested via the first two domains, it would match with the https:// {} block.
  • If no regex is matching it would → fallback handler ?

Given that my internal domain all end on lan, i could change the generic https:// site block to *.lan, no?

This would be perfect, if it would work like that.

6

Yes, you can use env vars. Make sure to read this whole page:

It would look more like @foo header_regexp domain Host ^foo\.{$DOMAINS}$

The handle must go within a site block. All directives must go within site blocks. Otherwise, Caddy will think handle is a domain name to serve.

Yes :+1: (except for the fallback handle having to go within a site block)

Yes that would be much better.

The “problem” with on_demand is that it’s vulnerable to DDoS unless protections are put in place. That’s why we require configuring the ask option, normally. But we loosen that restriction for tls internal because typically people don’t mix public and private domains on the same server.

To clarify, On-Demand TLS issues a new cert every time Caddy sees a new domain in TLS-SNI (as long as it’s allowed by the ask endpoint). This means that if it’s too permissive, an attacker could point a wildcard to your server, then keep making TLS handshakes with different domains every time (like 1.domain-evil.com then 1.domain-evil.com to infinity) and that would fill up your server with useless TLS certs until you run out of storage space or hit rate limits causing other domains to fail issuance.

So basically, I don’t recommend using https:// for internal domains + on_demand, because it’s open to DDoS. It’s definitely better to list the internal domains so that on_demand is only enabled for a specific subset of names.

7

Thank you for your awesome help with all of this. I think I am set to get startet and make my hands dirty, and that would have not been the case without you. I appreciate it a lot.

I think i might also go this route. Not with two servers but having two caddies on the same machine will work in this case and limits the risk of exposing something internal for external users by me misconfigering something.

That makes sense. But i don’t think this would be a problem in this deployment. But it is something to keep in mind.

1 Like
8

:blush:

Keep in mind that you’d need to use different ports (or at least different a different IP/interface with the bind directive) because two instances can’t listen on overlapping addresses.

1 Like
9

Yes I know that. :blush: :+1:

10

Hi, it’s me again with two follow up questions.

The setup untill now was great and without a lot of hassle. However there are some limitations I am faced with.

(A) Redirect to http and skip certificate
Is it possible to skip the ssl certificate when redirecting? (example foo below)
The issue is, that this is a http ressource anyway and no encryption should be involved. However when a browser contacts foo.server.lan/ it first gets greeted with an untrusted certificate. Moving this to the http:// block isn’t an solution either because the browser defaults to https and the default handler would be served.

(B) Redirect to http without SSL certificate
I use two services that have to serve via http:// (represented by bar).
My solution like below is working fortunatelly, but is there a simpler way of doing that?
(Now that I am writing … This methode of redirecting to http:// is also a solution for (A) but I’d hope there is a simpler way.)

http:// {
	
	@bar header_regexp domain Host ^ bar\.(.*)$
	handle @ bar {
		reverse_proxy http://bar:1234
	}
	
}


https:// {

	@foo header_regexp domain Host ^ foo\.(.*)$
	handle @foo {
		redir http://192.168.1.254:9999 permanent
	}

	@bar header_regexp domain Host ^bar\.(.*)$
	handle @bar {
		# redirect to http variant
		redir http://{host} permanent
	}

}

(C) Allowing only one certain uri / path and then reverseproxying it
Then there is this service that needs a redirect to a certain file and parameters at the very first call. I managed to do it like this:

    # if it is on path / redirect to vnc.html?autoconnect?true
    @blubb {
		header_regexp domain Host ^blubb\.(.*)$
		path /
	}
	handle @blubb {
		redir {host}/vnc.html?autoconnect=true&resize=remote permanent
	}

	# if it is not path / then do the reverse proxy
	@blubbReverse {
		header_regexp domain Host ^blubb\.(.*)$
		not path /
	}
	handle @blubbReverse {
		reverse_proxy http://blubb:6080
	}

This is working okayish, but not really what i wanted to do.
I want it that every call gets changed to {host}/vnc.html?autoconnect=true&resize=remote and then reverse_proxied.

But when i delete the path / in @blubb it will result in a recursive loop that always adds {host} in front of the uri every time, so that I end up with my.domain/my.domain/my.domain/my.domain/...

I then tried something like …

@blubb {
		header_regexp domain Host ^krusader\.(.*)$
		not expression `{uri}.equals("{host}/vnc.html?autoconnect=true&resize=remote")`
	}

But that did surely not work out.

The docs could definitely need improvements regarding expressions and the offerings of CEL with some real world examples in a caddyfile.
Also examples next to the placeholders in the caddyfile concepts page would be very nice. :slight_smile:

11

A site block like http://foo.server.lan will always take precedence over built-in HTTP->HTTPS redirects.

Or you can turn off built-in HTTP->HTTPS redirects with the auto_https disable_redirects global option, then redirects are fully in your control via the http:// site block.

I’m not really sure I understand the goal, but refer to my answer to A, might answer both?

Move the redirect into your handle like this:

	@blubb header_regexp domain Host ^blubb\.(.*)$
	handle @blubb {
		redir / /vnc.html?autoconnect=true&resize=remote permanent
		reverse_proxy http://blubb:6080
	}

redir matches exactly / and redirects to /vnc...

You don’t need {host}, redirects can contain only a path. This is cleaner for this usecase.

For CEL, it’s a difficult balance. We don’t want to restate all of the CEL language syntax, that’s not the Caddy docs’ job. We can’t reasonably maintain that.

But I agree, we need more CEL examples. The problem is that matcher examples are always extremely specific to a certain usecase, and having to explain what the usecase is for the given example is often a chore.

We don’t want to give examples to patterns that are not actually good things to do – many of the examples in the matcher docs are already somewhat nebulous… and being at the point where that complex matcher is needed is likely a sign that a rework is needed to avoid it being necessary.

If you’d like to open an issue on the website repo, we can discuss specific ideas for examples if you’d be willing to help us craft those.

I agree. I’m not sure how we’d present them though. There’s not really any room in that table to add another column with value examples. We’d have to totally re-do how placeholders are presented.

I’ve been thinking we need a dedicated Placeholders page elsewhere which attempts to collect all the known placeholders from all the Caddy apps (keep in mind that there’s HTTP placeholders, and global placeholders, and other “apps” and plugins may have their own placeholders they define).

13

I tried removing the @bar handler from the https:// block. Then I opened bar.my.domain/ in chromium and i got the https:// default handler. Therefore I need the redir to http in the https:// block for the same service. (see caddyfile above)

Note, that I can’t use regular site block, but instead have to match the domain with regex handlers, because they should work with every domain pointing to this device. Too bad that full site blocks like subdomain.* { ... } would not work in caddy.

I wanted to know if the described construct of having two handlers for each site, where one redirects to http (if only http should be available) could be made simpler.

Thanks, this is better. Is there a way I could send all paths (not only root /) to this path? replacing / with * does not work well with browsers.

That is true. The CEL is in my humble opinion a really difficult topic to research online. There was not much to find online regarding this topic, unfortunatelly. I couldn’t even find if there is .endsWith() or .includes() available as String methodes. This is of course not your responsability.

Thank you for beeing open for improvement. I consider doing that. I think for newcommers, like me, it would be great to have several common examples of “real-world” caddyfiles with explanations of what it does. Preferably at one page.
All the powerful opportunities caddy can provide via the caddyfile is imho held back by the docs, because they present the information only bit by bit. Additionally they are presented as one line of code most of the time, which are probably not most often used as such. This holding me back a little from understanding the syntax exactly and also the “big picture”. I catched myself often thinking: “okay, but where does this feature go exactly?”
I think having an example caddyfile at the end of every page would help tremendously newbies understand and therefore you spending less time on the forums.
I could be wrong though.

That would be very helpful. :+1:
Also it did take some hours, that I understood that environment placeholders can not be set at the global section of the caddy file, but are actually environment variables at os level. Would be awesome though, to have some variables inside the caddyfile that I can set values to myself.

14

I’m not sure I follow. Please use curl -v to test things, browsers sometimes have their own behaviour like skipping HTTP and trying HTTPS directly (especially if you ever set an HSTS header).

Technically it does, but it only matches a single label (so sub.* only matches sub.localhost but not sub.example.com). I don’t recommend it though, best to just use host matchers, because then it doesn’t fight with the Caddyfile site block sorting stuff which could cause other problems.

I’m still not sure I understand. You want to redirect HTTPS to HTTP? That seems silly. Why would you ever have a valid HTTPS cert but want to force users to use HTTP? That’s very backwards.

I don’t understand why you’d want that. That means any request path the user uses would get completely lost. Also means paths to any assets (CSS/JS/etc) on the same domain would totally fail to load because they’d also get redirected to /vnc.html.

But I guess you could do this:

@notvnc not path /vnc.html
redir @notvnc /vnc.html?autoconnect=true&resize=remote permanent

They have their functions documented here cel-spec/doc/langdef.md at master · google/cel-spec · GitHub and there is endsWith() in there. But yeah, I wish the CEL docs were better. I’ve asked for improvements in Add examples in the "standard definitions" table · Issue #213 · google/cel-spec · GitHub but unfortunately it hasn’t been worked on yet :frowning:

We do have that already:

What constitutes a “real world example” is extremely nebulous. The needs of every user is so different.

We keep getting requests for “more examples”, but honestly, the problem is we can’t reasonably imagine all the things users might want to do with Caddy. The possibilities are literally infinite.

What’s most important it to learn the concepts so that you can write things as you need it, yourself. It’s very important to read and understand Caddyfile Concepts — Caddy Documentation so that you know where things go.

We already do that. Every docs page for directives has examples at the bottom.

Where are you missing examples, exactly?

They can be set in global options. What do you mean exactly?

Keep in mind that there’s two syntaxes for env vars. There’s {$ENV} which can be used anywhere (literally string replacement before the Caddyfile is parsed) and there’s {env.*} which can only be used in specific places where the underlying module actually attempts to do placeholder replacement.

There’s the vars directive that can define variables for the current request context, then can be referenced with {vars.*} placeholders.

But remember – the Caddyfile is NOT Caddy’s native configuration language. JSON is. That distinction is very important to understand, because it explains why things are designed the way they are.

Anything that the Caddyfile does must map to a valid JSON config. You can see the adapted JSON config with caddy adapt -p.

If we had “global variables”, then they would have to map to something that works in JSON config.

15

Intersting now its working. Might have been a cache issue. I’m sorry for bothering you with this.

Would be cool to have an option to also match with the second example. Then I could make use of site-blocks inside the caddyfile instead of having a full https:// block that chatches everything and then matching the domain via regex, which work well but feels like a hack.

I can only guess at this point, but I think the reason is the same why you’re recommending using http in the backend instead of skipping the tls verfy. As your Docs states:

proxying over HTTP in private networks is preferred if possible, because it avoids the false sense of security.

Those services android apps outright refuse to work with a self signed certificate. There is no option to allow or add the certificate because it is self signed, and therefore “can not be trusted”. But the android apps work with using http, so I’m using that. :man_shrugging:

You’re right, that has not worked out. The idea was to redirect everything to this site initially, because this is the entry point. The user should not be able to see anything else but the vnc.html.
Because now the user is able to surf up to /some/folder and could see things that he should not see. But the things are required to function for the app, so I think it might not be possible.

Ah yes, at least that would have helped. I could not find that. Might be worth linking in your Doc.

I think you’re getting the requests because of a reason. This is not by any means ment to be an offense, so please take this as what it is: A report as how an average computer guy (me) felt, when first comming in contact with caddy.
The docs are indeed very useful and nicely put together and I think the most important page to get started is the Caddyfile Concepts Page answell as the Common Caddyfile Patterns Page.
However I always had questions like: “Can i mix that together?”, “Where exactly is that possible?”, “Where do I have to put that?”, …

So in my opionion (and this is debateable for sure) this would be some welcome improvements:

(A) Going from an overview into detail, instead of list every detail and let the user find out how to put them together
So e.g. on the Common Patterns Page it is stated, that the listed examples are fully caddyfiles, but this can be easily overlooked. I think the more ideal approach would be to give newcomers a basic but solid caddyfile with multiple different blocks and explain the different sections, instead of letting the newbies throw together different sections at their own. Let them see at first glance how a real world caddyfile looks like for like 3 domains, where one is a file_server, one is a redirect and one is a reverse proxy.

(B) Use blocks all the time
It is cool, that caddy is able to work without a block when only there is only one site “block”, but please use blocks all the time in the examples. Especially in the Caddyfile Tutorial, after you have explained Multiple sites. Without block everything seems so confusing.

(C) Where am I?
Especially the matchers page and the handle page lacks a bigger context in my opinion. I could not and still can’t figure out, where in the caddyfile this can be applied. Full caddy files where the written examples are applied would be so much easier to read and easier to understand. I know it is already a big page, but please embed your examples into a full caddyfile with one or more site blocks. Like in Point (A) seeing the big picture is essential for understanding. If you think there is no space for a full caddyfile at every point, then at least bring one big at the end.

More caddyfile examples also applies to the conecepts page. Why wouldn’t you bring a full caddyfile example e.g. in the addresses section, matchers section, named routes section and environment variables section?
I imagine people having a hard time learning new stuff, when they’re not seeing the knowledge applied.

(D) Quickstart? More like how not to do it
Ok this title is mean and I am sorry about that. But the reverse proxy quickstart page is not very helpful. How do people use caddy? I imagine 85% spin up a docker countainer and editing the caddyfile. Ok… i need a reverse proxy and go to its quick start guide … and then I am seeing this as the holy example:

:2080

reverse_proxy :9000

This config as minimalistic as it is, is NOT simple to understand because it is raising more questions than answering them. You know, you’re dealing with IT guys right? They want to spin up their reverse proxies and they want it fast. Why not give them a more useful example like this, where I think the concepts are more intuitive, more understandable, explaind with comments and would help people to get to their goal a lot faster, because they can adapt what they are seeing to their usecase.

# General Options Block
{
	# http and https port caddy operates at
	http_port 80
	https_port 443

	# email for requesting TLS certificates
	email myemail@sample.org
}


# reverse proxy for the domain "help.sample.org"
help.sample.org {
	reverse_proxy https://10.0.133.55:443
}

# it is also possible for one block to handle multiple domains
share.sample.org, send.sample.org {
	reverse_proxy http://10.0.133.56:3000
}

# caddy can also handle hostnames e.g. for use in a docker environment
cloud.sample.org {
	reverse_proxy https://cloud:443 { 
		transport http {
			# the tls verification can be skipped like that
			# WE DO NOT RECOMMEND DOING THAT, THIS IS NOT SECURE! FOR INTERNAL USE ONLY!
			tls_insecure_skip_verify
		}
}

(E) Real world examples
I would also love to see a dedicated page for real world examples of caddyfiles. Like the most common applications. I don’t know, but what are the most common questions people have? I could imagine: How do i reverse proxy to a nextcloud instance? … Make it a point at this page and give them this:

nextcloud.sample.org {
	redir /.well-known/carddav /remote.php/dav 301
	redir /.well-known/caldav /remote.php/dav 301
	
	header Strict-Transport-Security max-age=31536000;
	
	reverse_proxy http://10.0.15.186:443
}

How can I reverse proxy to a path? Give them this:

service.sample.org {
	rewrite * /new/path
	reverse_proxy https://service.backend.lan
}

How can i match my domains with regex? etc. etc.

Please do not read this as a rant, allegation or something like that. You have fantastic software and the docs are good, but the latter is really lacking examples. Getting into caddy was a frustrating and more often then not confusing process, but I dont think it has to be this way. I would have been completely lost without your help in the forums.

It is indeed not possible to write down every possibiliity, but that is not necessary either. But there are common examples that a lot of people use for sure.

That is true,and this page is one of the most valueable ones getting into caddy. But the learning process could - in my oppionion - be supported by more, and more suitable examples.

In the the matchers page and the handle page. There are a lot examples, but I am not able to see where in the caddyfile they belong to. Examples with a full domainblock would be more understandable.

They can? Ok am apperently still confused about this. I could not find any information about how to set {$MYVAR} to a value i would like. Like {MYVAR="SomethingSomething"} in the global block and use it later in the caddyfile. I don’t think that is possible. Would be a great feature request.

1 Like
16

It’s intended.

Site blocks are supposed to map to valid domains, because top-level host matchers trigger the Automatic HTTPS behaviour of automatically managing certs for domains.

This means that site addresses must conform to the same format as what appears in TLS certs, so that we can automate it. This means * only matches one label.

Using a header_regexp Host matcher is the best way to handle edgecases.

Okay but then you shouldn’t be running HTTPS at all for that domain, only HTTP. It doesn’t make sense to redirect HTTPS to HTTP. Just choose to only serve HTTP for that domain, by using http:// as a prefix to your site address.

You can add a matcher which blocks access to specific sensitive paths, then (with the error directive). That requires an understanding of the site structure etc. So that’s up to you to figure out.

We link to GitHub - google/cel-spec: Common Expression Language -- specification and binary representation, and at the bottom there’s a link to the Language Specification, which gets you there.

The problem is the language spec is kinda dense. I wish there was a more user friendly CEL guide somewhere.

We might just need to bite the bullet and write one ourselves at some point. Time consuming to do.

Hmm, so you’re suggesting an example on the patterns page that shows multiple sites?

I’m not sure there’s much value in that. I think there’s enough places that show that’s possible already. There’s even the “Redirect www. domains” one which does this as well.

That’s more a “concepts” thing than a “pattern” thing. And the concepts page explains that right at the start, both with the structure diagram & the part right below that explaining site blocks.

I agree with you. But that’s something @matt would disagree with. IMO there should be one way, but the goal is to have the argument that a Caddyfile can be super short with 2 lines or text or whatever.

Yeah, it’s a tricky balance. We want the docs to stay focused on the matcher at hand, showing other unrelated config can get unwieldy/misleading.

One possible approach is we make all (or most) of the examples be wrapped in a tab box like this which would by default just be the basic matcher example, and a 2nd tab for “full config” so it can be toggled to see where it would show up in context.

Yeah we should add an Examples section at the end of the matcher docs :+1: The tricky thing is finding sensible examples that don’t feel contrived. A confusing example is worse than nothing I think.

Yeah, we’ll probably do that. Again, a balance of being “focused” on the concepts, and finding realistic examples.

I agree with you. The quick start was written in v2.0 betas and hasn’t significantly changed since. I agree it’s really lacking in terms of covering usage with Docker etc. Matt wrote those with the thought that majority of readers would want to be using Caddy directly using the CLI, and that’s what it walks you through. But in reality most users install Caddy either to run as a systemd service or in Docker, and way less use the CLI directly. We do plan to rewrite all those soon.

FWIW, we have some major improvements to the docs coming down the pipeline, but people got busy and we haven’t been able to finish it up yet. For example, we’ll have this fancy “choose your own adventure” style thing up front and center. We just need to write the content for it:

https://twitter.com/mholt6/status/1704365700620693712/video/1

It’s a deliberate decision that we don’t call this out in an obvious way. It’s a anti-pattern, so we don’t want to bring attention to it. We rather point users to the right way to do things instead. (And in this case, proxying over HTTP and not HTTPS if the upstream is in your local network, or having your upstream serve a CA-signed cert which can be trusted explicitly).

That’s what the wiki is for: Wiki - Caddy Community

We can’t spend our time documenting config for apps like this. We have to stay focused on documenting how to use Caddy, not how to use other apps. Instead, those apps should show the config they recommend for Caddy users to use.

And if we document one thing over another alternative, it might be seen as preferentialism. (e.g. documenting NextCloud but not OwnCloud, etc).

We (the maintainers) are often not users of most of those applications, so we’re not really in a position to test and confirm that configs actually work either.

Yeah no worries, you’re good. We need the feedback :+1:

Ah, yeah no, nothing like that exists. Setting env vars from config doesn’t make sense, they have to come from the environment.

I thought you confusion was about where the env vars can be used in the Caddyfile, which is anywhere for the {$ENV} syntax.

I’ll do some more thinking about it though. I understand what you mean.

2 Likes
17

FYI I’ve been working on a big update of the Caddyfile docs.

2 Likes
18

Sorry for the late answer, i was having a busy week.

Yes, that is what I am doing now. While testing this, I thought it was not possible otherwise, because of Firefoxes chashing algorithm. (Which is not even emptied when manually clicking refresh. One have to explicity delete cache for this to work.)

Thanks for the hint. :wink:

Like I said, everything I mention or suggest is highly debatable, because of course you have the clearer picture and an overview of everything caddy can do, which I do lack. Additionally you of course decide approach and philosophy of the docs, which I am not aware of. And I agree with you after revisiting the pages, that is more of a concepts than a pattern thing.
What I wanted to say is, that sometimes (and I think this is the case here), it is better to have the user look at the big picture (whole config) where concepts can learned implicitly and inuitively, maybe explaint what is going on there in the differenc sections, instead of explainging every point in detail one after another. This maybe do not apply to the pattern page, because I think it is good and understandable. Maybe more to the matchers page.

I honestly do not see any value in a 2-line config vs. a 4-line but more accessable (read: easily expandable) config. But yeah people do stuff differently i guess.

I think this would be a very helpful extension to the current docs. :slight_smile:

I don’t agree with that. Contrived examples do not neccessarily have to be confusing. I think the key point here is to have something people can copy and adapt to their needs. Of course highly complex matchers and patterns are out of scope for the docs, but some basic things seen applied in a caddy file would be helpful indeed.

That is a good idea. Speaking of docker - but this is again only suggestions and looks through the eyes of a new user … Is there any way to use the “reload” feature without restarting caddy without modifying the container itself? It would be great to have that accessable through docker environment variables.
It is also unfortunate that we can not have plugins/addons without recompiling the binary and therefore using the official docker container. I would have loved to have the Geo-Block feature available, but found it to be too complicated and not worth the hassle. But thats a software architecture decision and way out of scope here, but I wanted to mention it.

Wow, that seems handy!

It was only a suggestion, because I thought people sometimes need it. I understand and fully agree with the decision you made.

Yes i see that and it is true. Documenting uscases for several products is not your job. What i thought was: “What do people ask very frequency?” And document that, like a more indepth FAQ. At a first glance over the topics in the Help section, I thought that I am seeing “reverse proxy to path” very often, so maybe include that.

This would be handy to transfer the dockerfile to another system if you save your $DOMAINS in a variable e.g.

Cool! I will have a look. :slight_smile:

1 Like
19

No worries :blush:

I’ve added another example to the patterns page in the PR for “Caddy proxying to another Caddy” which has multiple sites. Should be sufficient I think.

Yup, explained here Keep Caddy Running — Caddy Documentation

I understand. I wish it were easier/possible, but there’s a lot of practical issues with Go shared libraries that are difficult to overcome.

Here’s a long discussion about it Support Go plugin mechanism for Caddy plugins/modules · Issue #5183 · caddyserver/caddy · GitHub

I’m pretty sure I added something like that in the docs PR :+1:

2 Likes
20

Nono… The reload itself you have to do manually and can be archieved via docker file / terminal.
I am talking about the --watch feature, that can not be turned on via docker environment variables. So it is basically not accessable for docker users, despite making our own docker image. This is an issue. Or am I mistaken here?
By the way: This feature is IMHO worth mentioning also in the “Keep Caddy Running” Docs, despite the fact that it is only meant for non-production installations.

21

You can override the default container command to add that flag if you want it.

Unfortunately --watch is kinda inefficient as-is, so I don’t recommend it outside of while actively working on changing your config. We’re reloading the config from disk every couple seconds, so it doesn’t really let your server go idle.

1 Like