Cloudflare connection timed out

Help
28

It needs to be the full command:

curl -H "Host:naff.casa" localhost

This is the same as curl localhost (which just makes a command line HTTP request to localhost, shorthand for “this computer”); the goal is to confirm that the linux box itself is accepting port 80 requests and sending it to Caddy correctly.

The -H "Host:naff.casa flag just includes a header to tell Caddy - should we actually reach it - which website we’re asking for, since Caddy’s not configured to serve a website called localhost.

If this works, we know port 80 is fine locally and we can start expanding out from there to find out where packets are getting blocked. If it times out, we start maybe wondering if Docker has a problem.

29

rnaff@linuxbox:/docker/cloudflare$ curl -H “Host:naff.casa” localhost
word
rnaff@linuxbox:/docker/cloudflare$

pretty neat that it replied with word. Didn’t realize curl was this hip?

30

That’s a good result. It connected and a server replied. (Word up)

Try again with a slightly modified command for me?

curl -ILH “Host:naff.casa” localhost
31

curl is so gangster.

rnaff@linuxbox:/docker/cloudflare$ curl -ILH “Host:naff.casa” localhost
HTTP/1.1 400 Bad Request: invalid header name
Content-Type: text/plain; charset=utf-8
Connection: close

32

Certified 90s blud.

Well, in another twist in this thrilling murder mystery, this does not appear to be Caddy responding.

There is no Server: Caddy header, and Caddy doesn’t throw a hissy fit at a header like Host:naff.casa.

Here’s what this should like on a box with Caddy in Docker responding on port 80:

whitestrake in /opt/docker at athena is 🐳 v20.10.17
➜ curl -IH "Host:auth.whitestrake.net" localhost
HTTP/1.1 308 Permanent Redirect
Connection: close
Location: https://auth.whitestrake.net/
Server: Caddy
Date: Thu, 25 Aug 2022 05:05:54 GMT

So we’ve got at least two problems right now:

  1. Caddy isn’t responding on port 80 - something else is?
  2. We still haven’t figured out where along the request chain the timeout happens

Do you have another linux box on the LAN you could run another curl command from? Try:

curl -ILH "Host:naff.casa" [IP address of your server running Caddy here]
33

ya, I ran it from proxmox’s shell:

oot@naffhouse:~# curl -ILH “Host:naff.casa” 192.168.1.37
HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Length: 5
Content-Type: text/html; charset=utf-8
Etag: “rh35ze5”
Last-Modified: Tue, 23 Aug 2022 20:52:26 GMT
Server: Caddy
Date: Thu, 25 Aug 2022 05:13:46 GMT

root@naffhouse:~#

34

And just to clarify, you have Caddy in Docker on Ubuntu as a VM on Proxmox?

35

yessir, with opnsense running baremetal on diff mini serrver

36

Well, we can update our list of problems a little bit, at least.

  1. Caddy is responding on port 80 to requests over LAN (although something else appears to be responding to localhost requests??) - might not be a problem
  2. Somewhere between the internet and the LAN, packets for port 80 are still being dropped

OPNsense has some pretty good diagnostic tools, though.

Before we proceed in that direction, first, lets get one more curl, from Proxmox:

curl -ILH "Host:naff.casa" 98.167.142.137
37

ok here’s the output for your latest command request via prox:

root@naffhouse:~# curl -ILH “Host:naff.casa” 98.167.142.137
HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Length: 5
Content-Type: text/html; charset=utf-8
Etag: “rh35ze5”
Last-Modified: Tue, 23 Aug 2022 20:52:26 GMT
Server: Caddy
Date: Thu, 25 Aug 2022 05:30:02 GMT

38

That’s… ostensibly a good result! From this we can infer that OPNsense is working as expected (and that it is hairpinning NAT, too). :+1:

The reason I say ostensibly is that now we have to blame something beyond OPNsense for the lack of external port 80 access - which means the issue may be beyond your ability to easily rectify yourself.

The first culprit I usually find in these situations is the ISP. Are you on a residential service? Could your ISP be blocking port 80?

1 Like
39

Morning Matt,

Yes, I’m residential.

I’ve been self hosting many services for over a year though.

I can get to *.naff.casa for many different purposes outside of my local network – on the world wide web.

so is it necessary to use a subdomain to beat my isp block on port 80? that doesn’t even make sense to me as I write this.

40

Indeed, there’s absolutely no reason having port 80 blocked would prevent you from hosting services for the last year!

Caddy can also solve TLS-ALPN challenges that require only HTTPS on port 443 (i.e. 80 can be ignored/disabled).

The problem is, this requires a direct connection to Caddy itself as the challenge is “solved” during the TLS handshake. Putting something like Cloudflare in the way means that LetsEncrypt isn’t talking to Caddy, it’s talking to Cloudflare (which is then talking to Caddy), breaking TLS-ALPN challenges and leaving you with only HTTP challenges as a viable option to requisition certificates.

Your other subdomains (except for jellyfin?) are not proxied by Cloudflare (see the * record is DNS only) and so the HTTP challenge works for these.

So with Cloudflare in the way blocking TLS-ALPN, and your ISP in the way blocking HTTP, you’ve got no validation methods left by default.

I can too - but not on HTTP only:

image
image1235×846 19 KB

I’ll note that some browsers like Firefox automatically try HTTPS if they can, so having a service unavailable on port 80 might have been a completely transparent/non-issue for you.

Your options are:

  1. Enable DNS validation by installing the Cloudflare DNS provider module, or;
  2. Fix TLS-ALPN validation by removing the Cloudflare proxy (grey-cloud all services), or;
  3. Fix HTTP validation by calling your ISP and getting port 80 unblocked, or;
  4. Use Cloudflare tunnels.
1 Like
41

I turned off cloudflare proxy.

Should I remove the staging code from my Caddyfile?

BTW dude – since you’re a pro at all of this, I can imagine you do well as a DevOp or sys admin.

Any advice for a homelabber looking to go PRO?

42

Yes, you can set it back to the way it was before you incorporated my extra config.

Believe it or not, all of this is stuff you can pick up at the enthusiast level. We’ve just done some network troubleshooting, essentially, with the advantage of some in-depth knowledge of ACME and Caddy. I appreciate the compliment though!

If you want to sysadmin for a business, practice with things that businesses use.

Whether that’s bread and butter Windows Server VMs on ESXI or Hyper-V serving AD/Storage/RDS (maybe with hybrid cloud?), or whether that’s picking one of the big clouds (AWS/Azure/GCP) and mastering deployment of their technological stacks and securing and scaling them, or maybe going hard into container orchestration with K8s or something.

Whatever you do, practice business-class service levels, with business-class technologies.

43

thanks again for all of your help.

Cheers

1 Like
45

I didn’t want to create an entire new thread on this sub so I figured I would just reply to this thread since you’re the person helping me anyways.

I cannot get www.naff.casa to redirect to naff.casa

It gives me the error, too many redirects.

Here’s my caddy file:

www.naff.casa {
    redir https://www.naff.casa
}

I’ve tried it with the (uri) option too and it didn’t make a difference

any thoughts?

I also cleared my cookies and browser history on all devices, and tried it on a new device, no luck

46

Your vhost reads like the following:

“Serve www.naff.casa and redirect everything to https://www.naff.casa”, which is an infinite loop.

If you want to redirect www.naff.casa to naff.casa, you will have to use

www.naff.casa {
    redir https://naff.casa
}

The uri option you mentioned, which would look like

www.naff.casa {
    redir https://naff.casa{uri}
}

keeps the uri when redirecting:
For example, https://www.naff.casa/example would then redirect to https://naff.casa/example - keeping the /example part.

redir https://naff.casa on the other hand, would drop it.
So, https://www.naff.casa/example would always redirect to https://naff.casa/.

docs/caddyfile/patterns#redirect-www-subdomain shows an example including its counterpart (non-www to www redirect).
Maybe redir (Caddyfile directive) — Caddy Documentation should cross-link that somewhere :thinking:

2 Likes
47

A post was split to a new topic: Certificate issuing problems with Caddy, Proxmox VE while ISP is blocking port :80

48

This topic was automatically closed after 30 days. New replies are no longer allowed.