Cloudflare Proxy + Caddy AutoSSL causes 301 redirect loop

1. Caddy version (caddy version):

[developer@code-server-fs bin]$ /opt/bin/caddy version
v2.2.1 h1:Q62GWHMtztnvyRU+KPOpw6fNfeCD3SkwH7SfT1Tgt2c=
[developer@code-server-fs bin]$ /opt/bin/caddy list-modules
admin.api.load
admin.api.metrics
caddy.adapters.caddyfile
caddy.listeners.tls
caddy.logging.encoders.console
caddy.logging.encoders.filter
caddy.logging.encoders.filter.delete
caddy.logging.encoders.filter.ip_mask
caddy.logging.encoders.json
caddy.logging.encoders.logfmt
caddy.logging.encoders.single_field
caddy.logging.writers.discard
caddy.logging.writers.file
caddy.logging.writers.net
caddy.logging.writers.stderr
caddy.logging.writers.stdout
caddy.storage.file_system
dns.providers.cloudflare
http
http.authentication.hashes.bcrypt
http.authentication.hashes.scrypt
http.authentication.providers.http_basic
http.encoders.gzip
http.encoders.zstd
http.handlers.acme_server
http.handlers.authentication
http.handlers.encode
http.handlers.error
http.handlers.file_server
http.handlers.headers
http.handlers.map
http.handlers.metrics
http.handlers.push
http.handlers.request_body
http.handlers.reverse_proxy
http.handlers.rewrite
http.handlers.static_response
http.handlers.subroute
http.handlers.templates
http.handlers.vars
http.matchers.expression
http.matchers.file
http.matchers.header
http.matchers.header_regexp
http.matchers.host
http.matchers.method
http.matchers.not
http.matchers.path
http.matchers.path_regexp
http.matchers.protocol
http.matchers.query
http.matchers.remote_ip
http.matchers.vars
http.matchers.vars_regexp
http.reverse_proxy.selection_policies.first
http.reverse_proxy.selection_policies.header
http.reverse_proxy.selection_policies.ip_hash
http.reverse_proxy.selection_policies.least_conn
http.reverse_proxy.selection_policies.random
http.reverse_proxy.selection_policies.random_choose
http.reverse_proxy.selection_policies.round_robin
http.reverse_proxy.selection_policies.uri_hash
http.reverse_proxy.transport.fastcgi
http.reverse_proxy.transport.http
pki
tls
tls.certificates.automate
tls.certificates.load_files
tls.certificates.load_folders
tls.certificates.load_pem
tls.handshake_match.sni
tls.issuance.acme
tls.issuance.internal
tls.issuance.zerossl
tls.stek.distributed
tls.stek.standard

2. How I run Caddy:

Docker, with custom executable file aquired from the download page with tls.dns.cloudflare plugin

a. System environment:

Docker

b. Command:

paste command here

d. My complete Caddyfile or JSON config:

http://project.tld, http://www.project.tld {
		root * /home/project/www
		templates
		file_server {
			index index.html
		}
}

ssl.project.tld {
		root * /home/project/www
		templates
		file_server {
			index index.html
		}
        tls {
            dns cloudflare APIKEY(Edit zone DNS	Zone.DNS)
        }
}

3. The problem I’m having:

No matter what i do, when i use Cloudflare Proxy, i get stuck in a redirect loop because of cloudflare requesting the http content to proxy, but caddy gives it a 301 redirect to https so all the user sees when they visit, is that 301 redirect, which keeps looping

Basically i need to disable the auto upgrade function, but i use caddyfiles, not json configs, which DONT have that feature as far as i can find in the damned docs

4. Error messages and/or full log output:

ERR_TOO_MANY_REDIRECTS in chrome

5. What I already tried:

I have no clue what to try, litterally no bloody clue.
The only thing that fixes it is one of two

  1. use full End2End Encryption with cloudflare (not an option due to the nature of the project i am wokring on)
  2. Dont use cloudflare proxy (again, not an option as i self host my crap and need cloudflare’s cache to not kill my home network when someone accesses my webserver

6. Links to relevant resources:

You can disable the auto HTTPS redirects using the global option:

{
	auto_https disable_redirects
}

But I’m not sure I understand why the redirect loop is happening. Are you talking about requests to http://project.tld or to https://ssl.project.tld? Could you use curl -v and show what you see in the response? What’s in your Caddy logs (add log to your sites to enable access logging to stdout)?

Also, FYI, “SSL” is an outdated term, it’s now called TLS. SSL is the old version of the protocol, which is no longer supported by browsers.

I cant do a global config as I need it to function normally on other domains. I am hosting TWO websites off of the same CaddyServer, one of which NEEDS the autoTLS for security on a few subdomains, however the project is intended on bringing back the old design of the internet. And be compatible with the oldest of internet browsers still available to people

And i am unable to provide logs because docker is being completely unhelpful, like… this is all the logs the container shows for caddy,

{"level":"info","ts":1603904889.2980137,"msg":"using provided configuration","config_file":"/opt/caddy/caddy.conf","config_adapter":"caddyfile"}
{"level":"info","ts":1603904889.3038065,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["localhost:2019","[::1]:2019","127.0.0.1:2019"]}
{"level":"info","ts":1603904889.3044424,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0002ea1c0"}
{"level":"warn","ts":1603904889.3054857,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [cloudflare origin certificate *.domain.tld domain.tld]: no URL to issuing certificate"}
{"level":"info","ts":1603904889.3060846,"logger":"http","msg":"skipping automatic certificate management because one or more matching certificates are already loaded","domain":"code.domain.tld","server_name":"srv1"}
{"level":"info","ts":1603904889.3062313,"logger":"http","msg":"skipping automatic certificate management because one or more matching certificates are already loaded","domain":"cdn.domain.tld","server_name":"srv1"}
{"level":"info","ts":1603904889.3063,"logger":"http","msg":"skipping automatic certificate management because one or more matching certificates are already loaded","domain":"git.domain.tld","server_name":"srv1"}
{"level":"info","ts":1603904889.3063579,"logger":"http","msg":"skipping automatic certificate management because one or more matching certificates are already loaded","domain":"www.domain.tld","server_name":"srv1"}
{"level":"info","ts":1603904889.3064024,"logger":"http","msg":"skipping automatic certificate management because one or more matching certificates are already loaded","domain":"domain.tld","server_name":"srv1"}
{"level":"info","ts":1603904889.3064764,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv1"}
{"level":"info","ts":1603904889.3065662,"logger":"http","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv0","http_port":80}
{"level":"warn","ts":1603904889.3066568,"logger":"http","msg":"user server is listening on same interface as automatic HTTP->HTTPS redirects; user-configured routes might override these redirects","server_name":"srv0","interface":"tcp/:80"}
{"level":"info","ts":1603904889.3124702,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["ssl.project.tld"]}
{"level":"info","ts":1603904889.3131485,"logger":"tls","msg":"cleaned up storage units"}
{"level":"info","ts":1603904889.3226454,"msg":"autosaved config","file":"/opt/caddy/xdg-config/caddy/autosave.json"}
{"level":"info","ts":1603904889.3226726,"msg":"serving initial configuration"}

I wish, i wish i was joking. Ever since caddy2 rolled out, i have had nothing but headaches with caddy logging.

Full caddy config for ssl.project.tld

ssl.project.tld {
		root * /home/project/www
		templates
		file_server * {
			hide   .git .gitignore .gitmodules
			index  index.htm index.html
			browse /home/project/www/assets/file-template.htm
		}

	tls {
		dns cloudflare redacted-key-with(Zone.Zone.READ, Zone.DNS.Edit)
	}
}

And here is that curl -v with the site link redacted at the moment as it is no where near ready

*   Trying 104.28.16.71:443...
* Connected to ssl.project.tld (104.28.16.71) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=CA; L=San Francisco; O=Cloudflare, Inc.; CN=sni.cloudflaressl.com
*  start date: Oct 27 00:00:00 2020 GMT
*  expire date: Oct 26 23:59:59 2021 GMT
*  subjectAltName: host "ssl.project.tld" matched cert's "*.project.tld"
*  issuer: C=US; O=Cloudflare, Inc.; CN=Cloudflare Inc ECC CA-3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x56331e13f9b0)
> GET / HTTP/2
> Host: ssl.project.tld
> user-agent: curl/7.73.0
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
< HTTP/2 301 
< date: Wed, 28 Oct 2020 17:28:21 GMT
< cache-control: max-age=3600
< expires: Wed, 28 Oct 2020 18:28:21 GMT
< location: http://ssl.project.tld/
< cf-request-id: 0611d9f9f80000dbc859a2e000000001
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report?s=Ir10LsDaPpZvBgW5oymYyQMw11Qi31i7kHidLd%2Fck2eDFvYE4mOAky09xgeHd8Zmz45hQdDcFriBxgYwusD4Q8MU01RfS8rEhxJitFzO"}],"group":"cf-nel","max_age":604800}
< nel: {"report_to":"cf-nel","max_age":604800}
< server: cloudflare
< cf-ray: 5e965f6ffc04dbc8-SEA
< 
* Connection #0 to host ssl.project.tld left intact```

The redirect is triggered by Cloudflare, not by Caddy. So it’s a Cloudflare issue. I don’t use Cloudflare, so I don’t have much to offer on that topic.

What headaches are you having exactly? If a container isn’t retaining logs, then I don’t see how that’s Caddy’s fault?

Docker is retaining logs, that is not an issue.

The issue is that i cannot SAVE those logs to a file, meaning i cannot back them up to a external storage or sift through them with a text editor to see where the error actually is. I sent the only logs docker gives for caddy,

In caddy1 i had per-site logs like so log /opt/docker/caddy/logs/sub.domain.tld.log, however caddy2 does not appear to have that functionality at all, and it SOMETIMES sends docker errors, but most of the time, it does not at all send any form of error logs. And i do beleive i have posted about it here on these forums at one point and i was basically told that stdout logs were a thing, but the post was specifically about log files as an stdout log doesnt even give parsable json because it adds a timestamp as PLAINTEXT before the json which screws up JSON parsers and linters

EDIT: I noticed that I went off on unrelated tangent

But the issue still remains that the client is unable to connect due to a 301 redirect loop,

yes/no, Cloudflare is acting as a Proxy to the site, meaning a curl request will appear like it is cloudflare that is initiating the upgrade, when infact it is actually caddy initiating the auto TLS,
Cloudflare is requesting http://ssl.project.tld and caddy is returning a 301 redirect > https://ssl.project.tld
Which cloudflare then passes to the client, when the client basically requests the content again, and it loops endlessly with 301-redirects to the same page

You can configure access logs to write to a file in Caddy v2:

But that’s true, you can’t configure the rest of the logs to be written to a file from Caddyfile config yet, but that will come eventually, as global config. I’ll try to tackle that in a bit, but it’s tricky to do.

yes/no, Cloudflare is acting as a Proxy to the site, meaning a curl request will appear like it is cloudflare that is initiating the upgrade, when infact it is actually caddy initiating the auto TLS,
Cloudflare is requesting http://ssl.project.tld and caddy is returning a 301 redirect > https://ssl.project.tld

If Caddy triggered the redirect, you would be seeing Server: Caddy in the headers, but that’s not there. So I’m pretty certain Cloudflare is doing the redirect.

1 Like

Cloudflare’s Proxy acts as a middle man for security and DDOS protection, by masking the IP address and port blocking anything not a HTTP/S port, and preventing webserver exploits, to my knowledge atleast, which is why i use it because i self host. Because it acts as a layer of security between my server, and the public

Yes, but as far as I know, it doesn’t hide those headers. It still proxies through the headers from the upstream.

No, because when i went to the normal non-ssl domain i still see the same hearders


This issue only occurs when caddy attempts to use any form of SSL that is not cloudflare’s End2End Encrypted SSL (normal or strict) which are set per domain as a *.domain.tld and not a per sub.domain.tld basis.

I believe @Whitestrake’s write-up here is relevant to the issue you’re seeing and can guide you to a solution:

Will take a look at it

Just looked over it, that only covers if i am using cloudflare for SSL at all, when in this case, i am not

All SSL is going to be done server-side since one cannot manage a single subdomain with cloudflare’s ssl

Cloudflare’s ssl settings affect domain.tld, *.domain.tld

I only want SSL on ssl.domain.tld as the target for this website, is browsers that are long out of date, its a retro project, not a modern one

yes, normally disabling autossl would fix this, but i have another website being hosted off the same caddy instance

The “orange cloud” option in the DNS panel is what dictates whether you connect to Cloudflare or not.

Having SSL/TLS mode Off simply means that Cloudflare will always connect to your origin server over HTTP, and will downgrade HTTPS requests from clients.

Off disables secure HTTPS connections between both visitors and Cloudflare and between Cloudflare and your origin web server. Visitors can only view your website over HTTP. Any connections attempted via HTTPS result in a HTTP 301 redirect to unencrypted HTTP.
https://support.cloudflare.com/hc/en-us/articles/200170416-End-to-end-HTTPS-with-Cloudflare-Part-3-SSL-options#h_0d6603ef-2297-4806-af0b-febafce4283f

It’s no surprise here that you’re getting a loop.

Connect over HTTPS → Cloudflare downgrades you. Connect over HTTP → Caddy’s Automatic HTTPS upgrades you. Rinse, repeat.

2 Likes

Well… thats dumb. What if people want to use cloudflare, but also want to provide their own certificate?! wtf?

UPDATE: APPEARNTLY in Page Rules on cloudflare you can change the SSL setting for individual subdomains

So i set that to strict and now it works

You really can’t have it both ways.

Either you want people to connect to you, in which case you provide a certificate, or you want people to connect to Cloudflare, in which case Cloudflare provides a certificate.

3 Likes

This topic was automatically closed after 30 days. New replies are no longer allowed.