Is Caddy appropriate for my setup?

akhu · November 16, 2024, 10:30pm

1. The problem I’m having:

Currently using Apache to serve static pages for ~40 different domains, from the same server, with moderate traffic, and, when a pattern is detected in the url the request is rather forwarded to a separate (Java) process (REST) to handle and respond.

The question is will this work with Caddy and if so, is it rather straight forward or very tricky (;.

Is there some better way to setup?

Thank you.

2. Error messages and/or full log output:

n/a

3. Caddy version:

Latest

4. How I installed and ran Caddy:

Not yet. API/JSON.

a. System environment:

Ubuntu 24.04 - Plasma.

b. Command:

n/a

c. Service/unit/compose file:

n/a

d. My complete Caddy config:

n/a

5. Links to relevant resources:

n/a

skip2networks · November 18, 2024, 7:51pm

Hello Akhu

Yes, Caddy will certainly work well for this. Writing a Caddy config file (Caddyfile) is really easy, I’d like to refer you to the official docs: Caddyfile Concepts — Caddy Documentation

Unlike Apache, you don’t have to have lots of configuration files - just one Caddyfile will do.

Since this is so easy to do with Caddy, I wrote a snippet you could use for each of your sites:

www.your-webapp.com {
    handle /your-pattern-for-matching-java-rest-requests/* {
        reverse_proxy localhost:3000 # (wherever your Java REST api is listening)
    }
    handle {
        root /var/www/your-static-file-location/
        file_server
    }
}

This snippet implements two handle directives - the first to route Java REST api requests to your backend and the second, a ‘fallback’ route, to serve static files.

Let me know if it works for you!

akhu · November 19, 2024, 12:51am

Hi Alex,
This sound great for me to work with now. Thank you.

From my early understanding, JSON config seems more flexible than Caddfile. Is that right?

I do a lot of work with xslt and can generate JSON … endlessly …,
So, although I typically prefer simple, I am willing to consider JSON config. That may well be a mistake, as I am trying to consider medium/long term set up with Caddy. Can Caddyfile be exported to a JSON equivalent?

In the mean time, I can try the Caddyfile first and report here in a little while.

I appreciate your support.
Thank you.

francislavoie · November 19, 2024, 1:00am

Caddyfile lets you do 99% of everything, but has an easier to use and read syntax. The Caddyfile is an adapter that produces JSON config, and Caddy actually runs on the JSON config. But that’s largely just ran implementation detail.

Yes, with the caddy adapt -p command.

Using JSON config most often makes sense in automated/scripted setups where you plan to use Caddy’s config API to push config changes etc. Otherwise, you should certainly start with the Caddyfile.

akhu · November 19, 2024, 1:00am

Hi Alex,

can that be something like /abc/def/ghi/, where * is a wild card (protocol, domain, path, suffix, …), or similar?
That must be in the documentation … I need to look it up.

Thank you.

francislavoie · November 19, 2024, 1:03am

See the docs. Make sure to read this whole page, but that part you quoted is a request matcher (specifically a path matcher there).

akhu · November 20, 2024, 9:18am

Hi Alex, Francis,

With Apache we have virtual hosts like this one, for example, without certs:

# example.com

<VirtualHost *:80>
    ServerAdmin webmaster@example.com
    DocumentRoot /var/www/html/library/hyperlib/com01
    ServerName www.example.com
    ServerAlias example.com *.example.com
    ErrorLog /var/www/logs/example.com-error_log
    CustomLog /var/www/logs/example.com-access_log combined
</VirtualHost>

<VirtualHost *:443>
    ServerAdmin webmaster@example.com
    DocumentRoot /var/www/html/library/hyperlib/com01
    ServerName www.example.com
    ServerAlias example.com *.example.com
    ErrorLog /var/www/logs/example.com-error_log
    CustomLog /var/www/logs/example.com-access_log combined
</VirtualHost>

From your suggestion and some reading so far, I could have a Caddyfile like :

{
DEBUG
}

example.com www.example.com {
    handle */xml/* {
        reverse_proxy localhost:3000 # (wherever your Java REST api is listening)
    }
    handle {
        root /var/www/html/library/hyperlib/com01/
        file_server
    }
}

In this initial attempt at conversion I left out ports (http/https), aliases, subdomains and log files. How does it seem?
I like the idea of integrated structured logs. currently, with Apache, each domain has its own logs, for access and error.
It can be a single file/database and JSON should work fine. Consumption processing will be through xml, BaseX (XML DB) is an option.
How should I set it up?
For subdomains, can I use a wild card like *.example.com in the site address list (instead of www.example.com in the example) if all subdomains would map to same directory, …?
Since I have about 40 domains to manage and they are not all the same, I am not looking for a structured way to do this. Any help or guidance is appreciated.
I did not find documentation on cross-porting (ex. from Apache). Please tell me if there is.
I am quite sure that snippets and placeholders are handy, but unsure how/when.
For the reverse_proxy, I did not change it yet and I am wondering if I am right in assuming that the token is a url that could correspond to a different computer/IP address?

The good news is that I am looking forward to diving into Caddy. after almost 30 years of Apache …
I still have work to do to setup the server/bridge/firewall, to get Caddy running, but I will, especially with your support.

Thank you.
Regards.

francislavoie · November 21, 2024, 4:22pm

This isn’t right – inline path matchers must start with /, otherwise you need to use a named matcher.

Seems fine I guess. I’d add a comma between your domains to make it visually stand out more that you have two domains there. But also do you really need www.? Should you just redirect one to the other so you don’t have two identical sites loaded from different subdomains?

Just add the log directive. With no options it writes them to stderr along with the rest of your runtime logs. Or configure it to write to a file. The docs cover all this.

Yes, but then you would need a wildcard cert which means you need to build Caddy with a DNS plugin to solve the ACME DNS challenge. This adds some complexity to your setup.

What do you mean by “structrured” in this case? I don’t understand this point.

Forget everything you know from other servers. Learn Caddy from scratch. The docs have everything you should need.

Yeah, only reach for them when you feel you need them. Depends what you want to do. Snippets can reduce config duplication, placeholders can allow dynamic behaviour (pulling bits of data from the request etc).

You mean the upstream address? Yes it’s the host:port of your upstream app at which Caddy will connect to send the request and get the response.

akhu · November 24, 2024, 4:31am

Hi Jean-François,

I appreciate you comments and I am learning.

Here is my current Caddyfile, which I still can’t test bc of the work left to configure the system. I would very much appreciate your comments, especially when pointing my weaknesses, as I will work to fix them.

As for getting a better grip and structuring my setup, what I have done is build an application that reads the site properties and builds the Caddyfile.

The application can also obfuscate domain names and things.

The included Caddyfile is a result example (that takes less than 1/10 of a second to load, compile, obfuscate the config, as well as build the Caddyfile and save it.

It is easier to build, maintain, and evolve.

The domains in the obfuscated version (include here) exactly correspond to the domains I need to manage, only the names have changed.

As you recommended, all www. references are now redirections.
I have used some snippets, to try to ease reading, but now it is all generated anyway … I could have used another (stat), for static file service but the library and folder for the root directory change for each site. My app can handle it easy, but I was wandering how to do this with Caddy snippets.

Thank you for your time and kind support.
Regards.

global {
       debug
       (dynamic) {
               handle /xml/* {
                       reverse_proxy localhost:3000
               }
       }
       (logging) {
               log {
                       output file /var/www/log/caddy.log
                       format json
               }
       }
}
site01a.com {
       import dyn
       stat {
               root /var/www/html/library/lib/site01a/
       }
       import logging
}
site01b.com {
       redir https://site01a.com/
}
site01c.com {
       import dyn
       stat {
               root /var/www/html/library/form/site01c/
       }
       import logging
}
site02a.com {
       import dyn
       stat {
               root /var/www/html/library/lib/site02a/
       }
       import logging
}
site02b.com {
       redir https://site02a.com/
}
site03a.solutions {
       redir https://site02a.com/
}
site03b.solutions {
       redir https://site02a.com/
}
site04a.com {
       import dyn
       stat {
               root /var/www/html/library/lib/site04a/
       }
       import logging
}
site04b.com {
       redir https://site04a.com/
}
site05a.org {
       redir https://site04a.com/
}
site05b.org {
       redir https://site04a.com/
}
site06a.com {
       import dyn
       stat {
               root /var/www/html/library/media/site06a/
       }
       import logging
}
site06b.com {
       redir https://site06a.com/
}
site07a.com {
       redir https://site06a.com/
}
site07b.com {
       redir https://site06a.com/
}
site08a.com {
       redir https://site06a.com/
}
site08b.com {
       redir https://site06a.com/
}
site09a.com {
       redir https://site06a.com/
}
site09b.com {
       redir https://site06a.com/
}
site10a.com {
       import dyn
       stat {
               root /var/www/html/library/media/site10a/
       }
       import logging
}
site10b.com {
       redir https://site10a.com/
}
site11a.com {
       import dyn
       stat {
               root /var/www/html/library/mall/site11a/
       }
       import logging
}
site11b.com {
       redir https://site11a.com/
}
site12a.com {
       redir https://site11a.com/
}
site12b.com {
       redir https://site11a.com/
}
site13a.com {
       import dyn
       stat {
               root /var/www/html/library/media/site13a/
       }
       import logging
}
site13b.com {
       redir https://site13a.com/
}
site14a.com {
       redir https://site13a.com/
}
site14b.com {
       redir https://site13a.com/
}
site15a.com {
       import dyn
       stat {
               root /var/www/html/library/media/site15a/
       }
       import logging
}
site15b.com {
       redir https://site15a.com/
}
site16a.com {
       redir https://site15a.com/
}
site16b.com {
       redir https://site15a.com/
}
site17a.com {
       import dyn
       stat {
               root /var/www/html/library/media/site17a/
       }
       import logging
}
site17b.com {
       redir https://site17a.com/
}
site18a.com {
       redir https://site17a.com/
}
site18b.com {
       redir https://site17a.com/
}
site19a.com {
       import dyn
       stat {
               root /var/www/html/library/media/site19a/
       }
       import logging
}
site19b.com {
       redir https://site19a.com/
}
site20a.com {
       import dyn
       stat {
               root /var/www/html/library/media/site20a/
       }
       import logging
}
site20b.com {
       redir https://site20a.com/
}
site21a.org {
       import dyn
       stat {
               root /var/www/html/library/lib/site21a/
       }
       import logging
}
site21b.org {
       redir https://site21a.org
}
site22a.solutions {
       redir https://site21a.org/
}
site22b.solutions {
       redir https://site21a.org/
}
site23a.org {
       import dyn
       stat {
               root /var/www/html/library/form/site23a/
       }
       import logging
}
site23b.org {
       redir https://site23a.org
}
site24a.org {
       redir https://site23a.org/
}
site24b.org {
       redir https://site23a.org/
}
site25a.org {
       import dyn
       stat {
               root /var/www/html/library/form/site25a/
       }
       import logging
}
site25b.org {
       redir https://site25a.org/
}
site26a.com {
       import dyn
       stat {
               root /var/www/html/library/form/site26a/
       }
       import logging
}
site26b.com {
       redir https://site26a.com
}
site27a.com {
       import dyn
       stat {
               root /var/www/html/library/mall/site27a/
       }
       import logging
}
site27b.com {
       redir https://site27a.com
}
site28a.com {
       import dyn
       stat {
               root /var/www/html/library/mall/site28a/
       }
       import logging
}
site28b.com {
       redir https://site28a.com
}
site29a.com {
       import dyn
       stat {
               root /var/www/html/library/mall/site29a/
       }
       import logging
}
site29b.com {
       redir https://site29a.com
}

akhu · November 24, 2024, 4:36am

PS: a sites, like site29a.com, are the main sites while 29b … sites correspond to subdomains of site 29a.com.

francislavoie · November 24, 2024, 10:20am

That’s not my name.

That’s wrong, there’s no such thing as global. A global options block is a block with nothing in front of it, and must appear at the start. See the docs. Caddyfile Concepts — Caddy Documentation

Snippets don’t go inside the global options block, they’re separate top-level blocks.

You declared the snippet as dynamic, not dyn.

What’s this? That’s not a Caddy directive, not built-in anyway. You didn’t show you were building with plugins.

akhu · November 24, 2024, 6:03pm

Hi Francis,

I am sorry for my mistake with your name.
It must have been too late for me.

I will fix the other mistakes in the Caddyfile (and generator).
I am learning Caddy, enjoying it so far.

When fixed I could include the xml property sheet that provides the parameters for generating the Caddyfile. There is a “static” snippet and it is invoking an inner “snippet” (stat) in the generator and there is probably a bug where I show the wrong name. Sorry for the confusion. Thanks for pointing it out. Similar mis-coding/understanding for the other items you highlighted.

Thank you, this is great.
Regards.

akhu · November 25, 2024, 4:40am

Hi Francis,

Here is a new obfuscated Caddyfile version, hopefully reducing issues.
How does it?

{
       debug
}
(dynamic)  {
       handle /xml/* {
               reverse_proxy localhost:3000
       }
}
(logging)  {
       log {
               output file /var/www/log/caddy.log
               format json
       }
}
site01a.com {
       import dynamic
       handle {
               root /var/www/html/library/base/
               file_server
       }
       import logging
}
site01b.com {
       redir https://site01a.com/
}
site01c.com {
       import dynamic
       handle {
               root /var/www/html/library/form/stratml/
               file_server
       }
       import logging
}
site02a.com {
       import dynamic
       handle {
               root /var/www/html/library/lib/dnaos/
               file_server
       }
       import logging
}
site02b.com {
       redir https://site02a.com/
}
site03a.solutions {
       redir https://site02a.com/
}
site03b.solutions {
       redir https://site02a.com/
}
site04a.com {
       import dynamic
       handle {
               root /var/www/html/library/lib/com01/
               file_server
       }
       import logging
}
site04b.com {
       redir https://site04a.com/
}
site05a.org {
       redir https://site04a.com/
}
site05b.org {
       redir https://site04a.com/
}
site06a.com {
       import dynamic
       handle {
               root /var/www/html/library/media/c2/
               file_server
       }
       import logging
}
site06b.com {
       redir https://site06a.com/
}
site07a.com {
       redir https://site06a.com/
}
site07b.com {
       redir https://site06a.com/
}
site08a.com {
       redir https://site06a.com/
}
site08b.com {
       redir https://site06a.com/
}
site09a.com {
       redir https://site06a.com/
}
site09b.com {
       redir https://site06a.com/
}
site10a.com {
       import dynamic
       handle {
               root /var/www/html/library/media/telepath/
               file_server
       }
       import logging
}
site10b.com {
       redir https://site10a.com/
}
site11a.com {
       import dynamic
       handle {
               root /var/www/html/library/mall/consonance/
               file_server
       }
       import logging
}
site11b.com {
       redir https://site11a.com/
}
site12a.com {
       redir https://site11a.com/
}
site12b.com {
       redir https://site11a.com/
}
site13a.com {
       import dynamic
       handle {
               root /var/www/html/library/media/accordeon/
               file_server
       }
       import logging
}
site13b.com {
       redir https://site13a.com/
}
site14a.com {
       redir https://site13a.com/
}
site14b.com {
       redir https://site13a.com/
}
site15a.com {
       import dynamic
       handle {
               root /var/www/html/library/media/musnov/
               file_server
       }
       import logging
}
site15b.com {
       redir https://site15a.com/
}
site16a.com {
       redir https://site15a.com/
}
site16b.com {
       redir https://site15a.com/
}
site17a.com {
       import dynamic
       handle {
               root /var/www/html/library/media/mup-en/
               file_server
       }
       import logging
}
site17b.com {
       redir https://site17a.com/
}
site18a.com {
       redir https://site17a.com/
}
site18b.com {
       redir https://site17a.com/
}
site19a.com {
       import dynamic
       handle {
               root /var/www/html/library/media/mup-fr/
               file_server
       }
       import logging
}
site19b.com {
       redir https://site19a.com/
}
site20a.com {
       import dynamic
       handle {
               root /var/www/html/library/media/gbc/
               file_server
       }
       import logging
}
site20b.com {
       redir https://site20a.com/
}
site21a.org {
       import dynamic
       handle {
               root /var/www/html/library/lib/ka/
               file_server
       }
       import logging
}
site21b.org {
       redir https://site21a.org
}
site22a.solutions {
       redir https://site21a.org/
}
site22b.solutions {
       redir https://site21a.org/
}
site23a.org {
       import dynamic
       handle {
               root /var/www/html/library/form/kebek
               file_server
       }
       import logging
}
site23b.org {
       redir https://site23a.org
}
site24a.org {
       redir https://site23a.org/
}
site24b.org {
       redir https://site23a.org/
}
site25a.org {
       import dynamic
       handle {
               root /var/www/html/library/form/sps/
               file_server
       }
       import logging
}
site25b.org {
       redir https://site25a.org/
}
site26a.com {
       import dynamic
       handle {
               root /var/www/html/library/form/akhu/
               file_server
       }
       import logging
}
site26b.com {
       redir https://site26a.com
}
site27a.com {
       import dynamic
       handle {
               root /var/www/html/library/mall/pero/
               file_server
       }
       import logging
}
site27b.com {
       redir https://site27a.com
}
site28a.com {
       import dynamic
       handle {
               root /var/www/html/library/mall/letendre/
               file_server
       }
       import logging
}
site28b.com {
       redir https://site28a.com
}
site29a.com {
       import dynamic
       handle {
               root /var/www/html/library/mall/chatillon/
               file_server
       }
       import logging
}
site29b.com {
       redir https://site29a.com
}

Thank you.
Regards.

Whitestrake · November 25, 2024, 6:53am

This pattern can be consolidated.

site02b.com, site03a.solutions, site03b.solutions {
  redir https://site02a.com/
}

Likewise the redirs for https://site04a.com/, https://site06a.com/, https://site11a.com/, https://site13a.com/, etc…

Every single individual site block that has identical routes to another site block can be combined with its counterparts into one site block with all of the applicable site labels.

akhu · November 26, 2024, 7:13am

Hi Matthew,

Thank you for your attention and suggestion which I followed.
I can also change the configuration and just regenerate the Caddyfile.

To get Caddy operational I first need to resolve some firewall issues, but I am looking forward to Caddy.

Thank you.
Regards.

The Caddyfile, here with modified obfuscation and numbering now looks like:

{
       debug
}
(dynamic)  {
       handle /xml/* {
               reverse_proxy localhost:3000
       }
}
(logging)  {
       log {
               output file /var/www/log/caddy.log
               format json
       }
}
site01.com www.site01.com {
       import dynamic
       handle {
               root /var/www/html/library/base/root01/
               file_server
       }
       import logging
}
subdo1.site02.com {
       import dynamic
       handle {
               root /var/www/html/library/form/root02/
               file_server
       }
       import logging
}
site03.com www.site03.com site04.solutions www.site04.solutions {
       import dynamic
       handle {
               root /var/www/html/library/lib/root03/
               file_server
       }
       import logging
}
site05.com www.site05.com site06.org www.site06.org {
       import dynamic
       handle {
               root /var/www/html/library/lib/root05/
               file_server
       }
       import logging
}
site07.com www.site07.com site08.com www.site08.com site09.com www.site09.com site10.com www.site10.com {
       import dynamic
       handle {
               root /var/www/html/library/media/root07/
               file_server
       }
       import logging
}
site11.com www.site11.com {
       import dynamic
       handle {
               root /var/www/html/library/media/root11/
               file_server
       }
       import logging
}
site12.com www.site12.com site13.com www.site13.com {
       import dynamic
       handle {
               root /var/www/html/library/mall/root12/
               file_server
       }
       import logging
}
site14.com www.site14.com site15.com www.site15.com {
       import dynamic
       handle {
               root /var/www/html/library/media/root14/
               file_server
       }
       import logging
}
site16.com www.site16.com site17.com www.site17.com {
       import dynamic
       handle {
               root /var/www/html/library/media/root16/
               file_server
       }
       import logging
}
site18.com www.site18.com site19.com www.site19.com {
       import dynamic
       handle {
               root /var/www/html/library/media/root18/
               file_server
       }
       import logging
}
site20.com www.site20.com {
       import dynamic
       handle {
               root /var/www/html/library/media/root20/
               file_server
       }
       import logging
}
site21.com www.site21.com {
       import dynamic
       handle {
               root /var/www/html/library/media/root21/
               file_server
       }
       import logging
}
site22.org www.site22.org site23.solutions www.site23.solutions {
       import dynamic
       handle {
               root /var/www/html/library/lib/root22/
               file_server
       }
       import logging
}
site24.org www.site24.org site25.org www.site25.org {
       import dynamic
       handle {
               root /var/www/html/library/form/root24/
               file_server
       }
       import logging
}
site26.org www.site26.org {
       import dynamic
       handle {
               root /var/www/html/library/form/root26/
               file_server
       }
       import logging
}
site27.com www.site27.com {
       import dynamic
       handle {
               root /var/www/html/library/form/root27/
               file_server
       }
       import logging
}
site28.com www.site28.com {
       import dynamic
       handle {
               root /var/www/html/library/mall/root28/
               file_server
       }
       import logging
}
site29.com www.site29.com {
       import dynamic
       handle {
               root /var/www/html/library/mall/root29/
               file_server
       }
       import logging
}
site30.com www.site30.com {
       import dynamic
       handle {
               root /var/www/html/library/mall/root30/
               file_server
       }
       import logging
}

Whitestrake · November 26, 2024, 8:24am

Not bad.

The last bits of advice I have are:

Firstly, it can help readability to use commas between site addresses. site18.com, www.site18.com, site19.com, www.site19.com is easier on the eyes in my opinion than site18.com www.site18.com site19.com www.site19.com. This is of course entirely to personal preference and if you prefer it without them, it should function fine.

Secondly, handle serves two main functions: mutual exclusivity with other adjacent handle blocks, and to neatly contain a set of configuration under one matcher. Therefore if you have only one handle in a site block and you are not using it with a matcher, this stucture:

       handle {
               root /var/www/html/library/media/root18/
               file_server
       }

Can be functionally simplified to remove the redundant handle:

       root /var/www/html/library/media/root18/
       file_server

Saving some more lines in your Caddyfile.

akhu · November 26, 2024, 7:14pm

Hi Matthew,

Thanks again as this his truly helping me get a grasp.

You are right about the human readability of adding the commas,
but I may refrain from that, mostly bc parsing is simpler, I process tons of token sequences so I am comfortable, nevertheless, you are right.

I have fixed the redundant “handle” with blank matcher. I am still unsure about the effective scope of “functionally simplified”.

In any case, I do feel much more comfortable with Caddyfile and configuring the Caddy webserver,

As I noted before, I first need to fix the network and firewall config …
I’ll be back.

In the mean time, Take care.
Thank you.
Regards.