Making sense of `auto_https` and why disabling it still serves HTTPS instead of HTTP

Wiki
1

When your site addresses adhere to the requirements for automatic HTTPS activation, Caddy implicitly listens on the HTTPS port (default 443).

Automatic HTTPS can be configured via the Caddyfile using the global option auto_https, or via per-server JSON configuration. auto_https can be set to either of the following:

  • disable_redirects — Disables the implicit HTTP->HTTPS redirect.
  • off — In addition to removing the implicit redirect, disables automatic TLS provisioning for site addresses meeting automatic HTTPS activation requirements. Continues to implicitly listen on the HTTPS port.

Disabling automatic HTTPS (Only use HTTP)

The auto_https setting should probably be thought of as if it were named auto_tls instead, which more clearly describes what it does, unrelated to the implicit HTTPS by default behavior.

You might consider auto_https off would do as the name implies:

{
  auto_https off
}

localhost {
  root * /usr/share/caddy
  file_server
}

auto_https off only disables the HTTP->HTTPS redirects and the automatic enablement of TLS (automated certificate creation and renewals). The site address still listens implicitly only for HTTPS, however connecting will now result in the following outcomes (URL requests):

  • https://localhost :
    Server logs a TLS handshake error, stating no certificate is available — Caddy is not automating the creation of a TLS certificate.
  • localhost:443 (http://localhost:443) :
    Clients receive a response: “Client sent an HTTP request to an HTTPS server.” — Caddy is only allowing HTTPS connections. The port is correct, but the protocol scheme is not.
  • localhost / localhost:80 / http://localhost :
    Clients fail to connect with a connection refused error — Caddy is no longer applying the implicit redirect to HTTPS.

The activation rules surprisingly still apply, Caddy will unexpectedly continue to implicitly listen on the HTTPS port, but fail to establish a secure connection as you would expect with auto_https off.

Caddy’s error log will point out why, the TLS certificate is unavailable as it was not generated, thus the handshake will be unsuccessful. If a certificate is found (eg explicitly provided via tls directive) the handshake is successful and HTTPS works as normal.

For Caddy to allow insecure connections, you must be a little more explicit and have the site address fail to qualify for HTTPS activation.

You can opt out of implicit HTTPS by having your site address declared as:

  • http://localhost or localhost:80 - Exclusively listening on the HTTP port (80 by default).
  • :80 - Just the port, no hostname or IP address provided. This does not need to be the HTTP port.

Each site address in the below Caddyfile achieves the same outcome. auto_https off is not needed as implicit HTTPS no longer applies:

http://localhost, localhost:80, :80, :9000 {
  root * /usr/share/caddy
  file_server
}

Allowing HTTP and HTTPS without the implicit redirect

If you wish to serve content through both HTTP and HTTPS, without HTTP->HTTPS redirects, your first thought when new to Caddy might be to use auto_https disable_redirects. Nope, the redirect that gets disabled is also what made the site address listen on the HTTP port.

You actually need your site address to explicitly declare the intent to listen on both ports. To do so, specify multiple site labels, separated by a comma or whitespace.

One way to do this is by specifying the protocol scheme, which infers the standard ports (which can be changed via the http_port and https_port global options):

http://localhost, https://localhost {
  root * /usr/share/caddy
  file_server
}

Adding just the HTTP port also works (any other port would be treated as https://):

localhost:80, localhost {
  root * /usr/share/caddy
  file_server
}
  • The explicit https:// enables automatic HTTPS for that specific site address, enabling the automated management of TLS.
  • The explicit http:// (or any other rule that prevents automatic HTTPS activation) prevents an implicit redirect to HTTPS (for that site address), establishing an insecure connection over HTTP.

You’ll notice that auto_https disable_redirects is not in the Caddyfile global options, it serves no purpose when allowing insecure HTTP connections.

If you use an HTTPS only directive such as tls, the above approach with a single site block will not work. Instead you’ll need one site block for HTTP and another site block for HTTPS. To avoid duplicating the majority of the site block, you can use a third section as an import snippet:

(site-example) {
  root * /usr/share/caddy
  file_server
}

http://example.com {
  import site-example
}

https://example.com {
  tls internal
  import site-example
}

The above example uses a self-signed certificate generated by Caddy, instead of provisioning from LetsEncrypt. This is what Caddy does for a localhost site address automatically.

Disabling implicit HTTP->HTTPS redirects (HTTPS only)

HTTP->HTTPS redirects are good, but if for whatever reason you prefer to opt-out, you’ll want to useauto_https disable_redirects. Only HTTPS will be available unless you explicitly have a site address listen on HTTP.

{
  auto_https disable_redirects
}

https://example.com {
  root * /usr/share/caddy
  file_server
}

Note that since disable_redirects is a global option, there is no way to selectively disable implicit redirects for a specific site block in a Caddyfile. You can add explicit redirects if needed via the redir directive:

{
  auto_https disable_redirects
}

http://example.localhost {
  redir https://{host}{uri} permanent
}

https://example.localhost {
  root * /usr/share/caddy
  file_server
}

Misc

Testing

If experimenting. be careful of browsers caching 301 redirects or TLS certificates. It can be easy to make a configuration change and get confused why the behaviour is not matching expectations. Firefox private windows will still remember 301 redirects for example. Chrome Incognito windows are fairly reliable, be sure to close the browser session (all incognito window instances) inbetween tests.

Tools like curl behaves differently when a request omits the protocol scheme: https://localhost vs localhost:443 (equivalent to http://localhost:443, not https://localhost:443).

Docker - Using internal certificates

When Caddy creates self-signed certificates (eg for localhost site address, or when using tls internal), it will automatically add the root certificate to your systems trust stores for you. When running in Docker however, it is doing so within the running container, not the host system.

To avoid the insecure warnings from web browsers, you’ll need to manually add /data/caddy/pki/authorities/local/root.crt to your systems trust stores yourself. The root certificate has a validity of 10 years, while the intermediate (1 week) and leafs (1 day) will be renewed from that root certificate.

You can add the root cert to your other devices trust stores and all the internal certificates Caddy generates from the root certificate will be considered valid. Do not share the root.key file.

Misleading tls docs:

The docs are a bit misleading for tls when providing your own certificate file to use as they imply that automatic HTTPS will be disabled:

<cert_file> and <key_file> are the paths to the certificate and private key PEM files. Specifying just one is invalid; specifying both will disable automatic HTTPS.

That makes sense for automated TLS provisioning, but implicit redirects surprisingly are still applied when qualifying for automatic HTTPS activation. If the implicit redirect is unwanted, use disable_redirects.

{
  auto_https disable_redirects
}

example.localhost {
  tls /my-certs/localhost-cert.pem /my-certs/localhost-key.pem
  root * /usr/share/caddy
  file_server
}

TL;DR:

You probably don’t want to disable auto_https.

  • Serve content via HTTP, be explicit with http:// for your site address (eg http://example.com).
  • Serving both HTTP and HTTPS, declare the site address with both http:// and https://, or repeat the site address with one specifying the HTTP port (eg example.com, example.com:80 { ... }).
  • localhost automatically issues self-signed certificates instead of the usual LetsEncrypt default. If you need the same for for a FQDN site address, use the global option local_certs, or per site block tls internal.
    • If using Docker, add Caddy’s root certificate to your host systems trust stores to avoid the insecure warnings from web browsers.
5 Likes