Using "@id"s from Caddyfiles

1. The problem I’m having:

I have a simple Caddyfile that reverse-proxies two different servers:

example {
	reverse_proxy localhost:3000
}
api.example {
	reverse_proxy localhost:4000
}
www.example {
	# strip "www" prefix
	redir https://example{uri} permanent
}

Now, I’d like to be able to update the upstream for the frontend server
(on example) from localhost:3000 to localhost:3001. Of course,
I can rewrite the whole config and reload it, but I’d like to use
Caddy’s object ID (@id) functionality to patch it in-place.

As written, I can issue a PATCH update to a long path under the admin server:

localhost:2019/config/apps/http/servers/srv0/routes/2/handle/0/routes/0/handle/0/upstreams/0/dial

but not only is this unwieldy, it also contains indices that are not
obvious from the Caddyfile: specifically, the index in routes/2/.

Ideally, I would like to add a bit of syntax to my Caddyfile like this:

example {
	#                    new!
	#             +----------------+
	reverse_proxy $frontend-upstream localhost:3000
}
api.example {
	reverse_proxy localhost:4000
}
www.example {
	# strip "www" prefix
	redir https://example{uri} permanent
}

…which would adapt to JSON like this:

{
  "apps": {
    "http": {
      "servers": {
        "srv0": {
          "listen": [":443"],
          "routes": [
            {"match": [{"host": ["api.example"]}], "handle": [/* ... */]}, "terminal": true},
            {"match": [{"host": ["www.example"]}], "handle": [/* ... */]}, "terminal": true},
            {
              "match": [{"host": ["example"]}],
              "handle": [{
                "handler": "subroute",
                "routes": [{
                  "handle": [{
                    "handler": "reverse_proxy",
                    "upstreams": [{
                      "@id": "frontend-upstream",  // <-- new!
                      "dial": "localhost:3000"
                    }]
                  }]
                }]
              }],
              "terminal": true
            }
          ]
        }
      }
    }
  }
}

Then, I could PATCH localhost:2019/id/frontend-upstream/dial with the
new string "localhost:3001".

It’s okay with me if the @id doesn’t attach at the leafmost node, as
long as I can unambiguously navigate down from there to the actual
upstream definition.

Unfortunately, I wasn’t able to find documentation on how to write this
in the Caddyfile. I see docs about @id and about Caddyfiles but not
about using them together. I tried @myid, $myid, [myid], etc. in
various positions, but those variously were parse errors or resolved to
the wrong config.

2. Error messages and/or full log output:

Shooting in the dark, with this Caddyfile:

example {
	reverse_proxy @frontend-upstream localhost:3000
}
api.example {
	reverse_proxy localhost:4000
}
www.example {
	# strip "www" prefix
	redir https://example{uri} permanent
}

caddy adapt emits this error:

Error: parsing caddyfile tokens for 'reverse_proxy': unrecognized matcher name: @frontend-upstream

…which of course makes sense, since this is the syntax for matchers, not
@ids. I don’t know what the right syntax is.

3. Caddy version:

v2.7.5 h1:HoysvZkLcN2xJExEepaFHK92Qgs7xAiCFydN5x5Hs6Q=

4. How I installed and ran Caddy:

Installed per the Debian, Ubuntu, Raspbian docs, using the
Cloudsmith stable repos.

a. System environment:

Ubuntu 22.04.3, Linux 6.2.0, x86-64

b. Command:

caddy adapt

c. Service/unit/compose file:

Not relevant.

d. My complete Caddy config:

example {
	# how can I attach an "@id" to this upstream?
	#             +------------+
	reverse_proxy localhost:3000
}
api.example {
	reverse_proxy localhost:4000
}
www.example {
	# strip "www" prefix
	redir https://example{uri} permanent
}

5. Links to relevant resources:

https://caddyserver.com/docs/api#using-id-in-json

They’re not compatible. @id is exclusive to JSON config and the API.

If you use a Caddyfile you should not use the API. Any changes performed via the API cannot be persisted back to your Caddyfile, so it doesn’t make sense to do that.

You should either edit your Caddyfile and reload Caddy, or use caddy adapt to get a JSON config to start with and then manipulate that from here-on.

That said, you might want to consider using a dynamic upstream module instead of configuring a static upstream. It would let you update which upstream is used out-of-band. You could even write your own plugin if you want.

Another option if you need to automate is you can put a # comment line just above the proxy you want to update, then use sed or awk or whatever to update your Caddyfile, using the comment as a marker to find the line you want to edit, then sudo systemctl reload caddy as normal.

1 Like

Thanks for the quick response! That’s what I feared, but makes sense.

I saw the dynamic upstreams functionality, which looks tempting, but it
seems like it’d be more complexity overall. In my use case, I really am
just reverse-proxying from different ports on localhost (this is
actually my prod config except for domain names). Setting up a local DNS
server to serve SRV records specifying the port and dealing with the
resulting complexity (e.g., caching) seems not worth it, unless I’m
misunderstanding a piece.

I’ll stick with making my /etc/caddy/Caddyfile user-writable, updating
it with a script, and systemctl reload caddying. Thank you!

2 Likes

To clarify, you MAY use the API but probably just the /load endpoint makes the most sense. Other endpoints return JSON or specifically manipulate JSON so you may want to stick to just /load.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.