I’m assuming that this is impossible because I have not found any example in the docs, but maybe I’m wrong and there is a workaround
According to this:
The system first check if there is a snippet that match in the same file. If not then it considers it is a file and looks for a match.
I would like to see if for example I have a snippet in the file rules/example like:
(snippet) {
...
}
And I could use this snippet for example in another file by importing it.
Couldn’t you simply use a normal import as in the example?
import rules/*.rule
example.com {
import snippet
}
But is the idea that if (snippet) is specifically used in the Caddyfile, then it should not import it from rules/*?
Say I have a file called redir-options
Here I set redirs for some domains from www to non-www and the other way around: two snippets (strip-www) and (add-www)
Now I have this file:
example.com {
...
}
According to your idea, it would be
import rules/*
example.com {
import add-www
}
And it should work. But for some reason, it was throwing me a ton of errors like add-www file doesnt exist This is why I got to my first conclusion
Hmm, not sure why it wouldnt work. In my setup there is the main Caddyfile which sets up some generic stuff, then it has a section of varios snippets, and at last a line import vhosts/*.caddy. I have each domain or subdomain as an individual domain.com.caddy. Inside these caddy files I am using the snippets defined in the main Caddyfile.
It may be that using snippets from an imported file doesn’t work? I didn’t actually try that way around.
I have not tried putting the snippets in the main Caddyfile and then using them in the subfolders files, its not what I was looking for, but it could do the trick.
Imports are processed first, turning everything into a single long list of tokens, and then snippets are parsed and loaded, then sites are parsed and loaded. So it doesn’t matter which file snippets came from or which file they’re used in, it’s all just one big config in the end.
Sure, that attends the classic importing logic of most programming languages.
But something is not working this way under certain circumstances in Caddy importing. I started receiving errors like “that file doesn’t exist” when a file with a snippet was imported, like:
import /etc/caddy/snippet/example
And then using a snippet within that file say, called sample-snippet
Errors like: sample-snippet file doesnt exist
Where /etc/caddy/snippet/example having this format
(sample-snippet) {
...
}
I will set up a Docker and Caddyfile to demonstrate this.
Maybe nested importing has some caveats? Like doing an import of an import of another import may have restrictions on which can be loaded?
There are no caveats. Perhaps you need to be careful with the importing chain. Here’s a test I’ve just done, and it works fine without issues.
File: ./nested-snippet
(snippet-one) {
respond "Hello"
}
File: ./imported-caddy
import nested-snippet
File: ./Caddyfile
import imported-caddy
localhost {
import snippet-one
}
Copy/paste this compose.yaml, it should show you that it works correctly.
services:
reverse-proxy:
image: caddy:2.8
container_name: caddy
configs:
- source: caddyfile
target: /etc/caddy/Caddyfile
- source: snippet-outer
target: /etc/caddy/snippets/outer
- source: snippet-inner
target: /etc/caddy/snippets/inner
ports:
- "80:80"
- "443:443"
# Normally you'd use `volumes` to provide these files
# Used only for easy example via a single copy/paste `compose.yaml` file,
# All config files embedded into `compose.yaml`
configs:
snippet-inner:
content: |
(respond-snippet) {
respond "Hello"
}
snippet-outer:
content: |
import inner
caddyfile:
content: |
import snippets/outer
# This will fail since inner is imported twice
# (direct via * and also indirectly via outer)
#import snippets/*
:80 {
import respond-snippet
}
If you ran into your issue while using Caddy in a container, it’s important to remember that depending on what you’re doing a container may be started (or rather resumed from a stop) or restarted, which can sometimes cause problems as changes you make after are not always going to be applied.
For example with the Docker Compose example above, I’m using the configs feature. If you Ctrl + C + docker compose up or docker compose restart, even if you had made a change to that respond snippet text in compose.yaml, you’ll notice that the change is not applied. This is because the original container created was not removed.
To avoid that, you can force recreate a container like so:
docker compose down && docker compose up
docker compose up --force-recreate
You can run into similar problems with volumes, especially if the Dockerfile itself has the VOLUME directive. That will create an anonymous volume for every new container instance (it will be removed after disposing the container if you start it with docker run --rm), while Docker Compose works a little differently and makes extra effort to preserve the volume to the Compose project service it was created for, you need to find it (docker volumes ls isn’t too helpful there) and remove it explicitly, otherwise even when you create a new container for that same compose service like shown above, that anonymous volume retains the old content.
For bind mount volumes, you can run into a similar problem if you are bind mounting files directly instead of directories. In this scenario depending how you modify a file the inode might change on the host while the bind mount is tied to the old inode (this is not dockers fault, its how bind mounts work on linux AFAIK), recreating the container should update the inode. To avoid this problem bind mount directories only if file content will change (the container itself cannot write changes to a file bind mount IIRC, which is another caveat).
Likewise, even without volumes a container will persist it’s internal filesystem state until destroyed. That can be another source of confusion if you expected it to be reset when you stopped/restarted a container but didn’t recreate it when bringing it back up
I tested your solutions and yes, they work pretty well. So I went back to my last caddy implementation to review how I made up everything and I tried to replicate it here in a simpler way to illustrate better, what do I mean when nesting:
I’ve tried multiple options:
If the file with the snippets is imported in the same file where the snippet is working, everything goes well. But if the snippet is called alone (without any file imports, because the import has been done in another previous file), it throws an error (missing file, File to import not found)
But in this case, if we have two files calling the same file where the snippet it, it also throws an error (duplicate entries, redeclaration of previously declared snippet)
Here is the code I used for the example
That shows the same problem my previous compose.yaml example above mentioned in a comment. Skip to the end of this reply for a clearer explanation of the problem you’re hitting and how to avoid it.
Here is another adapted from your git repo link (since the linked github repo may be removed in future it’s better for future readers to have something to reference):
services:
reverse-proxy:
image: caddy:2.8
container_name: caddy
ports:
- "80:80"
- "443:443"
configs:
- source: caddyfile
target: /etc/caddy/Caddyfile
- source: snippet-configs
target: /etc/caddy/configs/site1
- source: snippet-configs
target: /etc/caddy/configs/site2
- source: snippet-rules-redir
target: /etc/caddy/rules/redir
- source: snippet-sites
target: /etc/caddy/sites/sites
# Normally you'd use `volumes` to provide these files
# Used only for easy example via a single copy/paste `compose.yaml` file,
# All config files embedded into `compose.yaml`
configs:
snippet-configs:
content: |
# Introduces conflict by importing twice:
#import /etc/caddy/rules/*
{args[0]}:80 {
import add-www {args[0]}
}
snippet-rules-redir:
content: |
(add-www) {
@{args[0]} host {args[0]}
redir @{args[0]} http://www.{args[0]}{uri}
}
snippet-sites:
content: |
# Import at a higher level so it is only imported once:
import /etc/caddy/rules/*
import /etc/caddy/configs/site1 a.localhost
import /etc/caddy/configs/site2 b.localhost
caddyfile:
content: |
import /etc/caddy/sites/*
www.a.localhost {
respond "Hello A"
}
www.b.localhost {
respond "Hello B"
}
The error with your original approach is quite clear:
Error: adapting config using caddyfile: redeclaration of previously declared snippet add-www, at /etc/caddy/rules/redir:1 import chain ['/etc/caddy/configs/site1:1 (import)']
So if you think about it as having the same named snippet declared in the Caddyfile (with the same or different content), that’d be an obvious conflict right?
# No imports, a single Caddyfile with two snippets
# as if they had already been imported:
(conflict) {
respond "A"
}
(conflict) {
respond "B"
}
Error: adapting config using caddyfile: redeclaration of previously declared snippet conflict, at /etc/caddy/Caddyfile
Yeah… so avoid that and just import the file content without the named snippet wrapping it, now it works like you want?
configs:
snippet-configs:
content: |
{args[0]}:80 {
# Refer to snippet by filename instead:
import ../rules/redir {args[0]}
}
# Avoid wrapping into a named snippet:
snippet-rules-redir:
content: |
@{args[0]} host {args[0]}
redir @{args[0]} http://www.{args[0]}{uri}
# Just like how you were doing already elsewhere:
snippet-sites:
content: |
import /etc/caddy/configs/site1 a.localhost
import /etc/caddy/configs/site2 b.localhost
caddyfile:
content: |
import /etc/caddy/sites/*
www.a.localhost {
respond "Hello A"
}
www.b.localhost {
respond "Hello B"
}
@polarathene yeah, this is what I’m doing right now. A naked file without a snippets wrap.
So, basically from your answer, I can conclude that snippets imports are very intricate (not meant to be imported very well, except for very straightforward examples).
I think that maybe for a single monolithic-form file, the idea of snippets could work, but once you move into a multiple file structure, things start to shake a little bit.
It’s not really the import directive at fault, that’s effectively a copy/paste of the file content when it’s not a named reference (the actual snippet syntax feature).
As I showed above with the conflict example, you can declare two snippets with the same snippet name in a single Caddyfile and you’ll have a problem. That’s all that is happening with import as snippets are duplicated.
You could raise a bug report about that. While it should error if the snippet were actually a real conflict in naming due to content, there could be logic that compares a hash of the snippet to detect dupes that aren’t real conflicts, then avoid the redundant snippet import. @francislavoie may have input on how viable that approach is.
In the meantime, you may want to organize your snippets in a global scope. Place them in a common top-level folder and while you can use subdirectories to organize them further, consider an index file that does import on all the actual named snippets (and subdirectory index files). Now your main Caddyfile does a single import on that and it should bring them all into scope.
There’s not a functionality difference AFAIK between the snippet feature vs using separate files for each snippet and then import? So you could just leverage the file structure import snippet/rules/redir as was done above and forget about the named references entirely? You only need to ensure relative paths are adjusted properly.
Yeah, probably possible to do, to ignore duplicate snippets. PRs welcome if it’s simple enough.
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.