Can I send "ASK" request to local file instead of URL?

Today I’m sending the ASK request like that:

on_demand_tls {
    ask      https://www.example.com/isDomainValid.php
}

And sometimes it send us a lot of traffic to the URL. Instead of that I thought it will be better to send the “ASK” into a local file in the server.
This file will include an update list of all the valid domains in the system. Maybe Caddy can also save the file in is memory for some seconds so it will prevent easy DDOS attack.

Can we do something like that?

You can run a thin HTTP server on the same server if you’d like, and make your ask endpoint http://localhost:8001/ or something.

But no, Caddy won’t directly read from a file. It expects it to be an HTTP server that it can hit.

1 Like

File isn’t a bad idea though. Maybe with the file:// scheme

3 Likes

I like this idea, and if you could also do something similar for the caddyfile root, make root file:// that points to a file that maps domains to directories, then I could easily dump apache for one of my applications.

You can already map domains to directories. If I understand you properly. Can you elaborate?

The thing I don’t like about the file:// idea is that we need to come up with a particular format the file needs to be in for us to read. More code in Caddy to maintain. More ways for things to go wrong. People will complain about file permission issues, inevitably.

Setting up a simple HTTP server microservice is dead-simple. You can do it in any programming language you prefer. Standard request format with the ?domain= query param. You can implement your own caching logic as you see fit (whereas we’d have to be opinionated with the file approach).

Yes, but if domains are being added dynamically, you need to rewrite the caddyfile and restart caddy or use the json api which is complicated. A file format is simpler assuming changes to the file would be reflected in caddy without a restart.

In Apache they have a rewritemap function where you can create a file with domain to root mappings.

example format:

www.domain1.com /home/directorya
www.domain2.com /home/directoryb

Then you could define a caddyfile like:

https://
root file://mapfilename

and depending on the domain, it would map to the correct directory

This file format could also be used for the ask function that this topic is about.

The format is simple… one domain per line. Setting up a simple http server microservice is a lot more effort than creating a file for the user of Caddy and you are adding more complexity to the infrastructure that people need to run a caddy service. The elegance of Caddy is it is so simple and it works.

Honestly I was just thinking of one domain per line.

We have one-liners in Caddy too:

map {host} {folder} {
    example.com dirA
    example.net dirB
    default dirC
}

Or, more simply, name you folders after the domain name, and just do: root * /{host}

or something like that.

You can do it in your Caddyfile like this:

{
	on_demand_tls {
		ask http://localhost:8001/
	}
}

https:// {
	tls {
		on_demand
	}
	# your actual on-demand site
}

:8001 {
	bind 127.0.0.1 ::1

	map {query.domain} {allowed} {
		foo.com 1
		bar.com 1
		default 0
	}

	@allowed `{allowed} == "1"`
	respond @allowed 200
	respond 400
}

You can use a simple bash script to update the map part of your config with new sites, then reload Caddy.

root * /{host} works great if the directories are named after the domain. Great solution and I have used this and it works great.

But if the directories don’t match the domain, then a map is nice when it is static, but if you are dynamically adding domains, you need to restart caddy. And if the domain update is issued from a client request using caddy/php, php can’t restart caddy as caddy won’t restart until the php request is completed. It sort of seems that restarting caddy to add or remove a domain isn’t ideal. It is sort of a catch 22. Plus rewriting caddyfile on the fly doesn’t feel right to me.

Well that is pretty slick! Thanks for sharing that!!!

However, still the issue with updating the caddyfile each time you add/remove a domain. And while a bash script is great, the issue of when the update is initiated from php invoked from caddy, there is a deadlock of caddy waiting for the php to complete before it reloads and php waiting for caddy to restart.

No, you never need to restart Caddy.

Only reload the config, which is graceful, zero downtime.

The only time you ever need to restart Caddy is if the program itself changed (new version, added plugins, etc)

1 Like

@Mohammed90 pointed out on our Slack that this could also be done via filenames and the file matcher, instead of map and such.

:8001 {
	bind 127.0.0.1 ::1

	root * /allowed-domains

	@allowed file {query.domain}.txt
	respond @allowed 200
	respond 400
}

The file matcher checks if the file /allowed-domains/foo.com.txt exists, and if it does, returns 200 status. No config reload necessary.

3 Likes

Sorry, I used restart and reload interchangeably. I will try and be more precise in the future.

Can I send a https request to caddy that runs a php program that executes a bash script that will update the caddyfile and reload caddy while the php program is running once the bash script completes, the php program can continue executing statements? It is my understanding that caddy won’t reload while the php program is running.

1 Like

This seems like a great solution for the ask.

You can respond to the request and continue to run PHP code after the response, for example with PHP: fastcgi_finish_request - Manual

1 Like

You teach me so much! Thanks! But once I issue the fastcgi_finish_request I can no longer communicate with the client from the php program. It does not seem like a clean approach, but I think it is a workable solution.