Efficiency of matchers

[Dante]

Hi,

I have 2 matchers opposing each other. Can I make this more efficient by merging them and using with and with not in routes?

  	@cache {
		not header_regexp Cookie "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in"
		not path_regexp "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
		not method POST
		not expression {query} != ''
    }
    
    route @cache {
        try_files /wp-content/cache/cache-enabler/{host}{uri}/https-index.html /wp-content/cache/cache-enabler/{host}{uri}/index.html {path} {path}/index.php?{query}
    }

    @nottocache {
	    header_regexp Cookie "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in"
	    path_regexp "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
	    method POST
	    expression {query} != ''
    }

    route @nottocache {
        header {
            Cache-Control "no-cache, no-store, must-revalidate"
        }
    }	  

Thanks!

[matt (Matt Holt)]

I don’t think that is particularly inefficient in terms of performance, but a bit unwieldy in terms of maintenance…

I would do this:

@nocache {
	header_regexp Cookie "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in"
	path_regexp "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
	method POST
	expression {query} != ''
}

handle @nocache {
    header Cache-Control "no-cache, no-store, must-revalidate"
}
handle {
    try_files /wp-content/cache/cache-enabler/{host}{uri}/https-index.html /wp-content/cache/cache-enabler/{host}{uri}/index.html {path} {path}/index.php?{query}
}

[Dante]

Thanks, @matt.

I have one more query and would appreciate any help with it. I noticed that in the next version of Caddy, the “ask” endpoint is now mandatory for on-demand TLS. Previously, @francislavoie had suggested a solution to me, but now I have a text file that contains all the domain names, each on a new line.

{
	on_demand_tls {
		ask http://localhost:8000
	}
}

:8000 {
	root * /var/www
	@domain-exists file /{query.domain}
	respond @domain-exists 200
	respond 400
}

How do I make this compatible with reading domain names from a .txt file called “domains.txt”? Would something like the following work?

:8000 {
  @domain-exists file domains.txt
  respond @domain-exists 200
  respond 400
}

[francislavoie]

I can’t think of a way to have Caddy open and read a text file like that. The only thing I could think of would be to import the text file as a config file (could be imported into a map handler) but this would require config reloads on every change.

You really should probably just write a super simple HTTP server that does what you need. It could be a single file program if you write it in Go.

@matt

I would do this:

I just want to say, be careful with that handle approach because if you have any other handle blocks in the same level of the config, only the first matching handle will run and every other one will be ignored. It depends what else is in your config.

[matt (Matt Holt)]

We leave the “how to determine if a domain is approved” logic up to you: you just have to provide the endpoint that produces the right result.

If you have a static list you maintain, you might just move it into your config file directly, something like:

:8000 {
  map {query.domain} {status} {
    example.com 200
    default 404
  }
  respond {status}
}

[Dante]

@francislavoie

I can’t think of a way to have Caddy open and read a text file like that.

I’ll use what you had suggested in the past. I just wrote a php script with a cron job which will fetch all domains from a WP network installation and create empty folders with the name. Caddy will refer to them for approval.

@francislavoie

I just want to say, be careful with that handle approach because if you have any other handle blocks in the same level of the config, only the first matching handle will run and every other one will be ignored. It depends what else is in your config.

I don’t have any other handle but I am posting my caddyfile here for inputs. Let me know if you see any improvement.

{
    order coraza_waf first
    storage file_system /var/lib/caddy
    admin off
    email redacted
    on_demand_tls {
        interval 2m
        burst    5
        ask http://localhost:8000
    }
}

:8000 {
	root * /var/www/domains
	@domain-exists file /{query.domain}
	respond @domain-exists 200
	respond 400
}

https:// {
    tls {
        on_demand
    }

    coraza_waf { // wip 
		directives `
		    SecAction "id:1,pass,log"
		`
	}

    root * /var/www/html/htdocs/
    php_fastcgi unix//run/php/php-fpm.sock
    encode gzip
    file_server

    header {
        Referrer-Policy "strict-origin-when-cross-origin"
        Content-Security-Policy "default-src https: 'unsafe-eval' 'unsafe-inline'; font-src 'self' data:; img-src 'self' *.mycdn.com https://secure.gravatar.com/ https://ps.w.org/ data:; object-src 'none'"
        X-Content-Type-Options "nosniff"
        Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
    }
    
    @nocache {
	    header_regexp Cookie "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in"
	    path_regexp "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(index)?.xml|[a-z0-9-]+-sitemap([0-9]+)?.xml)"
	    method POST
	    expression {query} != ''
    }

    handle @nocache {
        header Cache-Control "no-cache, no-store, must-revalidate"
    }
    
    handle {
        try_files /wp-content/cache/cache-enabler/{host}{uri}/https-index.html /wp-content/cache/cache-enabler/{host}{uri}/index.html {path} {path}/index.php?{query}
    }

    @disallowed {
        path /wp-config.php
        path /xmlrpc.php
        path *.sql
        path *.zip
        path *.tar
        path *.bak
        path /wp-content/uploads/*.php
    }

    rewrite @disallowed '/index.php'
}

Thanks @matt and @francislavoie! Really enjoying the caddy experience. It’s been the perfect nginx replacement for me with less moving parts.

[francislavoie]

@Dante

I’ll use what you had suggested in the past. I just wrote a php script with a cron job which will fetch all domains from a WP network installation and create empty folders with the name. Caddy will refer to them for approval.

If you’re using PHP, why not just have Caddy make HTTP requests to your PHP app to lookup the domains? It’s really weird to do it this way.

[Dante]

You are correct. I am thinking of something like this with the help of wp cli tool. Yet to test though.

on_demand_tls {
    ask http://localhost:8000
}

:8000 {
    @domain-exists {
        exec wp site list --field=url | grep --regex "^http(s)?://(www\.)?{query.domain}$" >/dev/null
    }
    respond @domain-exists 200
    respond 400
}

[francislavoie]

Why CLI? That’s overcomplicating things. Just use php_fastcgi:

{
	on_demand_tls {
		ask http://localhost:8000/ask-domain.php
	}
}

:8000 {
	root * /var/www/html/htdocs
	php_fastcgi unix//run/php/php-fpm.sock
	file_server
}

Just write a script /ask-domain.php to do the work.

[Dante]

Thanks for your reply. I am already using WP Cli for a lot of stuff. So I thought why not use it here too. But yours work too.

ask-domain.php

<?php
define('WP_USE_THEMES', false);
require_once('/var/www/html/htdocs/wp-load.php');

// Get the domain from the request host
$domain = $_SERVER['HTTP_HOST'];

// Check if the domain exists in the multisite network
if (domain_exists($domain)) {
    header('Content-Type: text/plain');
    http_response_code(200);
    echo "Domain exists!";
} else {
    header('Content-Type: text/plain', true, 400);
    http_response_code(400);
    echo "Domain does not exist!";
}