Automatic redirect to HTTPS only if on_demand ask gives 200 OK

1. Caddy version (caddy version):

v2.0.0 h1:pQSaIJGFluFvu8KDGDODV8u4/QRED/OPyIR+MWYYse8=

2. How I run Caddy:

/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile

a. System environment:

Debian 9.12, PHP 7.2.31 on a KVM machine.
Caddy installed from official deb package from https://apt.fury.io/caddy

b. Command:

/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile

c. Service/unit/compose file:

I use default caddy.service file, this one dist/caddy.service at master · caddyserver/dist · GitHub

d. My complete Caddyfile or JSON config:

{
    email mymail@example.com
    on_demand_tls {
        ask https://cfl.man-at-work.it/api/allowed-domain.php
    }
}

(global-encode) {
  encode zstd gzip
}

cfl.man-at-work.it {
    import global-encode
    root * /srv/www/cfl/current/public/

    # keep this line to avoid on_demand webhook call on itself
    tls mymail@example.com

    file_server
    php_fastcgi unix//run/php/php7.2-fpm.sock

    log {
        output file /var/log/caddy/cfl.log
    }
}

*.*, *.*.*, *.*.*.* {
    import global-encode
    tls {
        on_demand
    }

    root * /srv/www/sat/current/public/

    @urlblock {
        path_regexp wpattack /(wp-admin|wp-login|wp-content|xmlrpc|wp|wordpress)
    }
    respond @urlblock 410

    file_server
    php_fastcgi unix//run/php/php7.2-fpm.sock

    log {
        output file /var/log/caddy/sat.log
    }
}

3. The problem I’m having:

I want to configure Caddy2 to serve 1 well known site address with automatic HTTPS (cfl.man-at-work.it in this example, but this list can grow to 5 or 6 domains) and a multitenant software on a catchall (every customer has is own domain pointed to the server ip).
This config works ok (i.e.: HTTP to HTTPS redirect, and certificate generated and served on the fly) with domains that get a 200 OK response from the ask https://cfl.man-at-work.it/api/allowed-domain.php webhook, but I’d like to serve HTTP to those domains that are receiving a non-200 response (saas software manages this situation), or a static html page that says “domain not enabled” (or something similar).

My questions are:

  1. I don’t like the catchall pattern, is there a better way to catch any other connection to the server that are not listed as known site addresses (think about nginx server_name _;)?
  2. Can Caddy2 skip on_demand tls and automatic HTTPS redirect if webhook responds with 404 error and serve sites on HTTP?
  3. Can I catch tls { on_demand } errors and manage them?

4. Error messages and/or full log output:

Known site address:

$ curl -I http://cfl.man-at-work.it/
HTTP/1.1 308 Permanent Redirect
Connection: close
Location: https://cfl.man-at-work.it/
Server: Caddy
Date: Wed, 24 Jun 2020 08:13:17 GMT

curl -I https://cfl.man-at-work.it/
HTTP/2 200
content-type: text/html; charset=UTF-8
server: Caddy
date: Wed, 24 Jun 2020 08:12:17 GMT

catchall domain with 200 OK in allow-domain webhook

$ curl -I http://sat1.man-at-work.it
HTTP/1.1 308 Permanent Redirect
Connection: close
Location: https://sat1.man-at-work.it/
Server: Caddy
Date: Wed, 24 Jun 2020 08:15:02 GMT

$ curl -I https://sat1.man-at-work.it
HTTP/2 200
content-type: text/html; charset=UTF-8
server: Caddy
date: Wed, 24 Jun 2020 08:15:06 GMT

catchall domain with 404 Not Found in allow-domain webhook

$ curl -I http://sat4.man-at-work.it
HTTP/1.1 308 Permanent Redirect
Connection: close
Location: https://sat4.man-at-work.it/
Server: Caddy
Date: Wed, 24 Jun 2020 08:16:03 GMT

$ curl -I https://sat4.man-at-work.it
curl: (35) error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error

on console Caddy log this:

2020/06/24 10:17:19 http: TLS handshake error from 176.206.133.64:57171: certificate for hostname 'sat4.man-at-work.it' not allowed; non-2xx status code 404 returned from https://cfl.man-at-work.it/api/allowed-domain.php

5. What I already tried:

To write a better looking catch-all I’ve tried to set site address to http://, https:// but I cannot use tls directive

run: adapting config using caddyfile: server listening on [:80] is HTTP, but attempts to configure TLS connection policies

if I set site address to * Caddy responds with this:

$ curl -I https://sat1.man-at-work.it
curl: (35) error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error

Caddy console
2020/06/24 10:21:12 http: TLS handshake error from 176.206.133.64:57412: no certificate available for 'sat1.man-at-work.it'

Actual catch all pattern works, but on console I get this waring:

2020/06/24 08:29:05.661	WARN	http	most clients do not trust second-level wildcard certificates (*.tld)	{"domain": "*.*"}

6. Links to relevant resources:

Simplified allowed-domain.php webhook that I’m currently using on test machine, real webhook will do a series of checks on domain before returning 200 OK or 404 KO

<?php

$domain = $_GET['domain'] ?? null;

$goodDomains = [
    'cfl.man-at-work.it',
    'sat1.man-at-work.it',
    'sat2.man-at-work.it',
    'sat3.man-at-work.it',
];

header('Content-Type: application/json; charset=utf-8');
header('Access-Control-Allow-Origin: *');

if (in_array($domain, $goodDomains, true)) {
    http_response_code(200);
    echo json_encode(['OK']);
} else {
    http_response_code(404);
    echo json_encode(['KO']);
}

You can do :443 or https://, and a separate server block for :80 or http:// where you can do your own handling of redirects and such. This will override the default http:// catch-all that is automatically provisioned via automatic HTTPS.

Caddy currently only uses ask when it encounters a TLS handshake for a domain it hasn’t encountered yet. This means it only happens on HTTPS requests for a domain that Caddy doesn’t have a certificate for yet. This happens before any handlers are executed because it happens at the TLS handshake which is basically the first thing after accepting the connection.

If Caddy doesn’t have a valid certificate (or refuses to issue one because ask said no), then there’s no way for it to respond to the request with anything but a failed TLS handshake. Those errors would only happen if ask says no, so that means you can log the domains you rejected on your backend if you need.


I have a couple additional comments.

  • In your PHP script, I recommend using isset() instead of in_array() because isset() is O(1) whereas in_array() is O(n). With a lot of domains, this could make a significant performance difference. You’d just need to make sure the keys are the domains instead of the values (i.e. make your array like 'cfl.man-at-work.it' => true and so on)

  • Since it seems like you still want to serve sites on HTTP if the domain isn’t in your whitelist, I recommend you disable automatic HTTPS redirects (this is possible in the Caddyfile starting in Caddy v2.1 beta 1 with the global option auto_https disable_redirects, but having a http:// block will override this anyways) and have your PHP backend make the decision whether a redirect should occur rather than having Caddy do it.
    This essentially means that you’ll make an http:// server block in Caddy which you have handled by php_fastcgi, and in your backend, if the request was for HTTP, you can make a decision whether to respond with a Location header to trigger a redirect, or just serve whatever you want to serve if you don’t want to trigger a redirect.

Hopefully that answers your questions.

2 Likes

@francislavoie thank you for your precise and professional response!

I’ve implemented your suggestion and I have a working config:

{
    email myemail@example.com
    on_demand_tls {
        ask https://cfl.man-at-work.it/api/allowed-domain.php
    }
}

(global-encode) {
    encode zstd gzip
}

# add here any known site addresses that need HTTP -> HTTPS redirect
http://cfl.man-at-work.it,
http://other.example.com,
http://anotherone.example.net, {
  redir https://{host}{uri}
}

https://cfl.man-at-work.it {
    import global-encode
    root * /srv/www/cfl/current/public/

    # keep this line to avoid on_demand webhook call on itself
    tls myemail@example.com

    file_server
    php_fastcgi unix//run/php/php7.2-fpm.sock

    log {
        output file /var/log/caddy/cfl.log
    }
}

https://other.example.com,
https://anotherone.example.net {
    import global-encode
    root * /srv/www/static_site/

    # keep this line to avoid on_demand webhook call
    tls myemail@example.com

    file_server
}

# HTTP catch-all that manages HTTPS redirect for good SAAS domains and a generic 404 for all other
http:// {
    import global-encode
    root * /srv/www/sat_check/
    header X-Sat-Checker "Here I am, rock you like a hurricane"
    file_server
    php_fastcgi unix//run/php/php7.2-fpm.sock
}

# HTTPS catch-all with on-demand SSL cert
https:// {
    import global-encode
    tls {
        on_demand
    }

    root * /srv/www/sat/current/public/

    @urlblock {
        path_regexp wpattack /(wp-admin|wp-login|wp-content|xmlrpc|wp|wordpress)
    }
    respond @urlblock 410

    file_server
    php_fastcgi unix//run/php/php7.2-fpm.sock

    log {
        output file /var/log/caddy/sat.log
    }
}

http:// catch all will responds in rpoduction server with a symfony application that convert http request to https after looking up and validating domain name, this is the test file I wrote to outline the process logic:

<?php

$goodDomains = [
    'cfl.man-at-work.it',
    'sat1.man-at-work.it',
    'sat2.man-at-work.it',
    'sat3.man-at-work.it',
];

$domain = $_SERVER['HTTP_HOST'];

if (in_array($domain, $goodDomains, true)) {
    //redirect to https
    $redirectUrl = 'https://' . $domain;
    //rebuild orginal request
    if (strlen($_SERVER['REQUEST_URI']) > 0) {
        $redirectUrl .= $_SERVER['REQUEST_URI'];
    }
    if (strlen($_SERVER['QUERY_STRING']) > 0) {
        $redirectUrl .= '?' . $_SERVER['QUERY_STRING'];
    }

    http_response_code(301);
    header('Location: ' . $redirectUrl);
    die();
}

echo '<html><body>Site not enabled</body></html>';

Both above script and allowed-domain.php in previous post are not actual production code, they both are part of a Symfony application that lookup domain names on a Redis instance, but I agree with you on O(1) vs O(n) algorithms.

I’ve tried this option because I’ve found it in documentation, but Caddy reported an error: I find very strange reading official documentation for an unreleased version of the software, without clear indication that a particular option is available only on a future version.

Apart from this inconvenience with the documentation, I’m glad I started working with Caddy2!

2 Likes

This topic was automatically closed after 30 days. New replies are no longer allowed.