Show both JSON and Caddyfile configs in all wiki pages

This is a suggestion to show both JSON and Caddyfile formats in the examples in all wiki pages. Will help to grasp ideas quicker, especially for those who happen to write their first config files in JSON format from scratch for the first time. Many thanks.

I’d be ok with this. But I think 99% of users prefer Caddyfile, so it’ll be quite the time investment to go back and retrofit existing content with JSON for just a few users, which is probably why it hasn’t been done in most cases.

Is there an easy and quick way to convert from one file to another? Maybe that technique could be employed.

Wait, what, really? Am I wasting my time by learning the JSON format? The documentation clearly says that it is the most flexible and complete and gives the most power, so I though, why would I bother learning Caddyfile format if I can learn its JSON format straight away. More to that, it has a JSON schema, which is also quite useful.

So,… does it still make sense to learn JSON format? If not, why not? (considering it’s advertised as the most “native” and most complete).

Also, my thinking behind my suggestion to give examples in both formats was that JSON is the “primary” / “native” format, so it would definitely make sense to have all examples with JSON first, and with Caddyfile format next. Like e.g. it should not be optional.

It’s up to you of course, but I just wanted to share the experience I’m getting with learning JSON format while being a new Caddy user. It can definitely be improved, as currently with all my many years experience in software development, my experience so far has been such that it was not trivial to configure a reverse proxy just by looking at the JSON documentation format and at the Wiki. Hence sharing these suggestions.

1 Like

No. Caddyfile to JSON is a one-way conversion. It’s not 1:1, the Caddyfile implements all kinds of magic to make Caddy much easier to use.

I always recommend to write your config in Caddyfile first, then run caddy adapt --pretty to see the JSON it adapts to, if you want to better learn how Caddy works.

Not at all. I wish more people would learn it. It is a little more tedious to write by hand, though, which is why people prefer the Caddyfile. (But the code that converts Caddyfile to JSON is a rat’s nest, not gonna lie. And there is no conversion the other direction.)

Yep, I do usually recommend learning the JSON structure since you can exact all of Caddy’s flexibility from it, whereas the Caddyfile has more gotchas and is more limited in its expressive power.

Yes it does, definitely learn it. That puts you in the top 99 percentile of Caddy users in terms of expertise. And you don’t have to use JSON all the time. Usually it’s used by people who are automating their Caddy configs (because JSON is very easily programmable), or people who need to use Caddy in advanced ways that the Caddyfile doesn’t expose.

I agree there needs to be more documentation explaining practical uses of JSON. That’s kind of what the wiki is for, as well as my Expert Caddy series for sponsors. The docs are the reference that explain “what” and the wiki can be more community-edited to explain “how”.

I wish there was a way to inject bits of JSON content into the Caddyfile to get the best of both worlds :thinking:

What you can do is run caddy adapt then pipe the output of that into a CLI tool like jq to apply transformations to the JSON (or write your own little script with PHP or whatever scripting language of your choice), and then run with that JSON output. That way you can still maintain the Caddyfile while adding in JSON bits when necessary.

2 Likes

Compose the file in one format, then convert it with a tool, then apply yet another tool, then write a script… @basil and @francislavoie don’t you think you’re making very simple things at least two magnitudes more complex. Let’s instead improve the core thing so that there is no need to do all these extra steps. Why not improve JSON documentation, give more examples, add a live JSON editor that validates schema, add JSON schema files to public catalogues, etc. Let’s improve the foundation and then you don’t need to even spend time trying to figure out all those extra steps.

We are open to this.

We hope users will contribute examples to the wiki. Wouldn’t that be sufficient?

Most modern editors/IDEs do this, I believe.

This sounds complicated, since everyone’s schema may be different.

But to be clear, I support the spirit of what you’re saying. We just need to work through some hurdles first. And more community contributions are sorely need too.

Like we’ve said, 99% of users will never need to do this. The Caddyfile covers the vast majority of things users want to do. If it can’t do it, we try to make it possible when we can.

It’s just due to the nature of how the Caddyfile works. Caddy’s core is very simple because essentially all it does is provide support for provisioning and running modules, and loading/reloading config from a JSON document. Everything else is on a per module basis. The concept of sites in the Caddyfile does not map 1:1 with servers in the JSON, so we need to do a bunch of trickery to assemble a config that matches the user’s intent. This means that sometimes we don’t cover every possibility. We’re working on it. If you find something you can’t do, ask us and we’ll help.

2 Likes

Thanks @francislavoie, but this diverges from the original matter I tried to convey. I was not (still not) going to use Caddyfile, I wanted to use JSON file. But thanks for your patience anyway!

This does not sound like based on real numbers. If 99% of users use Caddyfile format, why do you advertise JSON as the most powerful and the “primary” format in the documentation, and then here say “almost nobody uses it”? Something is not right here - it’s illogical.

My opinion differs from Matt’s on this sightly – he wrote the docs for that part. If I had my way I’d put the Caddyfile front and center. But the argument that Matt puts forward is that we would ideally like users to understand at the very least that the Caddyfile is an adapter and is not actually Caddy’s underlying config language. The more users understand how Caddy works under the hood, the better they can use it and explore it. That’s why the getting started guide goes with an approach like: “look at how verbose this is… but wait, using the Caddyfile it’s way simpler!”

Sure “99%” is probably inaccurate, but it gets our point across. It’s by far the vast majority of users. The amount of questions on the forums that are specific to JSON configuration are quite few, in comparison to Caddyfile questions. And not because the Caddyfile is any harder to use, but just because there’s way more users of the Caddyfile.

1 Like

I see. My point is that why would I learn two formats (JSON and Caddyfile), if I can learn just one and know for sure that it can solve all my needs.

My personal preference is to use JSON exclusively - because every IDE can highlight the JSON, and we now know that we can get JSON schema as well.

So if I learn Caddyfile, one day I might be forced to switch to JSON to solve some configuration issue that only JSON can solve. If that’s true, then what’s the long-term benefit for me to learn Caddyfile? There isn’t any. It’s just two things to learn and maintain and keep in mind instead of one.

Just like “go” language has just one “official” way to format code, I’d advocate to have just one way to write a configuration file - be it JSON or Caddyfile. That would eliminate lots of needless things - extra documentation, extra converters, extra discussions in the forums, and extra doubt that people have when they just start with Caddy and don’t know which format they need.

You see, while being a newbie, in the very beginning, I’m afraid to pick Caddyfile format - because I don’t know if it is going to be a limiting factor or not, so I choose JSON. But now it turns out that it is advised against the use of the primary format. So I feel very confused, to be fair.

1 Like

If it were not for Caddyfile, I doubt very much if I would be using Caddy today. I’d probably be using a VPN into my private network rather than offering services from behind a reverse proxy with encrypted communication. Caddyfile trivialises the setup. While I’m starting to find my way around JSON, Caddyfile will always be far more human-readable. It’s like reading a bit of prose. One Caddyfile directive can map to dozens of JSON lines. Why would I want to do the latter? Everything I need to do, I can do in Caddyfile. There are just one or two edge cases of interest to me e.g. GitHub - mholt/caddy-dynamicdns: Caddy app that keeps your DNS records (A/AAAA) pointed at itself. and https://twitter.com/mholt6/status/1284250606673080321. There are alternatives to using Caddy for these.

3 Likes

Yes, it’s as Francis explained (though he and I might disagree as to whether it’s best): I believe that everyone should understand that Caddy’s native config format is JSON, and that the Caddyfile is just sugar to make it easier to configure. Our goal with official docs and tutorials is to teach people to fish, not give them already-caught fish. Toward that end of learning to explore the software for oneself, it is crucial to understand that Caddy’s native config is JSON, not Caddyfile.

Despite the fact that JSON is Caddy’s native config format – and thus the most flexible and expressive – almost nobody uses it. Most people who do are advanced users who are automating things, combining multiple Caddy apps, or doing other complex/unusual things.

Do you know that for sure? Our Getting Started guide expresses the pros and cons of each. Unless you can perfectly predict your feature, you will not know for sure that just one can solve all your needs. Both exist because there is no one perfect format. That’s why config adapters are pluggable – you can implement yet others if you prefer.

A lot of people find it faster to write configs with the Caddyfile, but I agree with you that with proper editor tooling, writing JSON can be just about as fast and is even more flexible and expressive.

If you don’t think you’ll need the advanced flexibility and automation that JSON gives you, then the Caddyfile might be a good thing to learn. But even then, it’s optional. Just use what you want.

I do not understand what you mean by this. Can you please re-explain?

1 Like

It failed with a new user like me, I gave up and switched to Caddyfile.

Being a developer for 15+ years, I could not figure JSON out based on the current documentation. Something to think about.

1 Like

You’re right! The documentation is terse and needs to be unravelled, However, you’ve probably noticed, unlike many other forums, Caddy core maintainers are very accessible and are extremely helpful. I think they’re doing the best they can with very limited resources. As you’ve come to realise, there’s still plenty of work to be done and community contributions are sorely needed.

3 Likes

JSON config’s goals are not to be easy, it’s to be flexible to support doing anything. This means that UX takes a hit. The Caddyfile actually has the goal of being easy to use, but we try to find a balance where it can do as much as possible.

2 Likes

This topic was automatically closed after 90 days. New replies are no longer allowed.