1. Output of caddy version
:
v2.5.2 h1:eCJdLyEyAGzuQTa5Mh3gETnYWDClo1LjtQm2q9RNZrs=
2. How I run Caddy:
docker run -d -p 80:80 -v caddy_data:/data caddy
a. System environment:
Latest docker.
b. Command:
n/a
c. Service/unit/compose file:
n/a
d. My complete Caddy config:
:80 {
root * /usr/share/caddy
file_server
}
3. The problem I’m having:
We are producing static websites on a central CICD server and hosting each site on multiple CDN nodes.
The CDN is in-house: Each node is a Caddy server and this is a “push CDN” such that we cannot rely upon a typical origin pull or TTL cache invalidation strategy.
Our static site generator is Hugo and Hugo does not support partial builds (unless you enable persistent build caches which is impractical in most ephemeral, container-based, cloud CICD environments).
Builds/updates are high traffic and hundreds of builds are running at any one time.
As such, when we push the latest build from the CICD server out to all CDN nodes, if we wish to do this in a bandwidth-efficient fashion, we must select a file sync diffing strategy that simultaneously:
- Ignores file ‘created’ timestamps; and
- Can distinguish between changed and unchanged files of the same size and name.
We’re heading into checksum territory.
We are thus exploring various local/remote strategies that are not related to Caddy (and I won’t waste the forum’s time with those here) and one that might work with Caddy’s help.
So far I have not found a way to configure this strategy, it may not be possible, but I thought I’d try my luck here with the awesome Caddy community! Cannot tell you enough how much we love Caddy !!!
The strategy proposed is:
- During CICD add a checksum of the file’s contents (md4, etc) to the filename of every file:
/index.<checksum>.html
/admin/index.<checksum>.html
/assets/app.<checksum>.js
/assets/brand.<checksum>.css
etc
-
Push/sync these files to each CDN node using filename diffs.
-
Configure Caddy to try_file for (eg.):
/index.<regex-or-wildcard>.html
....when the request is to....
/
/admin/index.<regex-or-wildcard>.html
....when the request is to....
/admin
…and so forth.
This would only be required for the html files. The /assets can all remain fingerprinted in the html that calls them.
4. Error messages and/or full log output:
n/a
5. What I already tried:
I have tried permutations of file_server, try_files, named matchers (esp. file), url, path_regexp and so on.
The common challenge is that all of these accommodate a regex/wildcard for requests/paths, but do not seem to accommodate regex/wildcards for file names on the caddy file system.
I got a little excited when I saw the configuration for…
‘try_policy’: ‘first_exist|smallest_size|largest_size|most_recent_modified’
…because that hints at rules to select from multiple matching files on the filesystem, but it doesn’t seem to be the filesystem wildcard feature that I’m looking for.
I’m completely expecting to be told that this cannot, or should not, be done.
These are not the droids you are looking for.
But…just maybe…
Any advice greatly appreciated!
6. Links to relevant resources:
Thx in advance for your advice Caddy community.