I am stuck on something very basic. Coming from NginX and Traefik

Help
1

1. The problem I’m having:

All works good until I try reverse_proxy.

2. Error messages and/or full log output:

Blockquote


May 17 22:24:29 caddy caddy[1039]: {"level":"error","ts":1684362269.5702627,"logger":"http.log.error","msg":"EOF","request":{

3. Caddy version:

2.6.4

4. How I installed and ran Caddy:

running directly as a system service

a. System environment:

on Ubuntu 20.4 server

b. Command:

I don't know what you mean.  I run it and the web page works, but not the reverse proxy

d. My complete Caddy config:

firewall.domain.cc {
        reverse_proxy 10.20.80.1:82

}


# The Caddyfile is an easy way to configure your Caddy web server.
#
# Unless the file starts with a global options block, the first
# uncommented line is always the address of your site.
#
# To use your own domain name (with automatic HTTPS), first make
# sure your domain's A/AAAA DNS records are properly pointed to
# this machine's public IP, then replace ":80" below with your
# domain name.

:80 {
        # Set this path to your site's directory.
        root * /var/www/html

        # Enable the static file server.
        file_server

        # Another common task is to set up a reverse proxy:
        # reverse_proxy localhost:8080

        # Or serve a PHP site through php-fpm:
        # php_fastcgi localhost:9000
}

# Refer to the Caddy docs for more information:
# https://caddyserver.com/docs/caddyfile

I had Traefik working for the longest time. I was having a lot of trouble getting Caddy working, but I finally have it up and running. I can get to my basic web page- you can see @ hacura.cc

The problem is, any reverse proxy I try doesn’t work. I think it’s not getting the certs it needs. I have no idea how to enable logs. The documentation on the official website is clear, but since I don’t know what I am doing I can’t figure it out without examples. I am using caddyfile, I hope I don’t have to use JSON. It’s something really basic. I know it’s not my firewall. I can’t imagine it’s any other service in my network either… but who knows. Any help is greatly appreciated.

2

How are you 100% sure of that? (It might not be, but it often is.)

The error message, EOF, means that Caddy is receiving that “end of file” message when contacting your backend at 10.20.30.1:82.

Try doing curl -v "http://10.20.30.1:82" on the same machine as Caddy and see what happens.

3

I only say that because everything was working fine with Traefik, and I didn’t change anything in my firewall except the IP that port 80 and 443 points to. In fact, I can routinely switch back to Traefik, change the IP in the firewall, and everything works again. I guess there has to be something else in my network that I’m missing…

Here is the output of the code you requested


user@caddy:/var$ curl -v "http://10.20.80.1:82"
*   Trying 10.20.80.1:82...
* Connected to 10.20.80.1 (10.20.80.1) port 82 (#0)
> GET / HTTP/1.1
> Host: 10.20.80.1:82
> User-Agent: curl/7.81.0
> Accept: */*
>
* Empty reply from server
* Closing connection 0
curl: (52) Empty reply from server

Also, this is the same behavior I was getting on two different attempts with Portainer. I was so frustrated I made a whole new VM, fresh from an Ubuntu ISO, and all it runs is Caddy. You’re right, it may be my firewall… I don’t understand this stuff well enough to rule that out… but I don’t see what would be different than Traefik.

4

Yeah, so that machine is just getting an empty reply from the backend. It looks like it’s connecting fine but the backend is sending an empty response. I would focus on troubleshooting your backend config.

From the Caddy side everything looks fine.

5

When you say backend config… you mean, for example, my firewall’s GUI? Or Truenas? Or Portainer? …these are the services I am trying to reach via subdomains, and they all work fine when I fire up Traefik and switch my router’s port forwarding to point to Traefik.

6

I don’t know – I’m not familiar with any of them. Probably whatever is listening on 10.20.30.1:82. It’s sending an empty reply to Caddy.

7

I just don’t see how. If I type in 10.20.30.1:82, it pops up my firewall GUI no problem.

8

From the same machine as the curl command and Caddy?

9

You should remove this stuff from your config though, this is preventing HTTP->HTTPS redirects from working in Caddy. That stuff is just meant as a default/example, not something you should keep.

10

I commented it out. Still no luck. I even purged my cloudfare cache, and turned on developer mode.

Here is my caddyfile now. Everything does the same thing… like you said, it seems to have no response from everything. Incredibly frustrating. Something in my network is blocking this. It seems like it must be my firewall. I looked through all the settings and don’t see anything weird. I rebooted a couple times too. Even just updated it for the heck of it. :confused:


firewall.domain.cc {
        reverse_proxy https://10.20.80.1:82
11

Don’t use https:// on every proxy, you should only do that if you specifically know the upstream is serving HTTPS. Almost always, internal services will be serving HTTP.

1 Like
12

Did you go to public school? You’re a genius. Of course I was pointing to secure ports.

Changed to unsecure ports and it works!

I guess that makes sense, because if I’m pointing to secure ports, the cert won’t match what the backend is expecting to use.

I believe there is a way to point to a secure port, and ignore the automated attempt at grabbing a cert? I think that’s what Traefik was doing the whole freaking time…

13

So , yes, almost everything is working now.

Except for backends that require HTTPS. Proxmox, for example, is impossible to connect HTTP.

I need to figure out how to make caddy stop trying to grab a cert for this, as it is already generated by the backend. I see there is a way to do this in the docs, but since there is no examples it’s basically egyptian language to me. I have tried this, and it doesn’t load the caddyfile (errors out)

prox.domain.cc {
      reverse_proxy 10.20.80.254:8006
      transport http {
           tls  
       }
}
14

I got it!!!


prox.domain.cc {
reverse_proxy 10.20.80.254:8006 {
transport http {
tls_insecure_skip_verify
}
}
}

Now everything works! I am still having issues with documents in nextcloud… something caddy isn’t doing right there, but otherwise this is great!

15

Just be aware that disables security, making TLS essentially pointless. You might as well serve over HTTP.

1 Like
16

Maybe I am using it improperly… These domains are web servers that are https only, and handle their own certs. I don’t think I quite understand how this is supposed to work with caddy, but if I don’t skip verify, it won’t work. I believe caddy is trying to pull a cert when the webpage already has its own… Do I need to copy the caddy cert that it generates and add it to the web server I’m connecting to?

Also, I don’t see any errors when using this method… it seems to have a secure connection?

image

17

Your connection to Caddy is secure and trusted, but caddy’s connection to the backend is not.

18

OK. I will try disabling the tls skip, and place caddy’s CA in the backend. From what I can tell, that should resolve the issue.

19

No it’s the other way around, Caddy doesn’t trust the cert your backend is serving. You would need to copy the cert from your backend to Caddy then configure tls_trusted_ca_certs reverse_proxy (Caddyfile directive) — Caddy Documentation.

Or, configure your backend with a publicly trusted cert instead of a self-signed one (which is probably a lot of effort)

Or just proxy to your backend over HTTP instead of HTTPS because you’re turning off security anyway; if the traffic is in your private network, the only risk is that other things in your network may be able to see the traffic, but as long as you trust the devices & software in your network that’s okay.

1 Like
20

Thank you so much. That makes sense now.

I did try initially, to change the backend to HTTP, but with services such as Proxmox, they don’t allow it. Their argument is security, yet I find that funny because in my case I accidentally circumvented it.

My initial attempt here didn’t work. I can’t ‘download’ the pem file from proxmox pbs, but I can view it. So i copied the contents, and created a pem file where caddy is installed. Caddy reloaded ok without errors, but I get a 502 error

pbs.example.cc {
     basicauth * {
          loginuser $2a$1-example-hash-oWBxKAzHhgH0cm38*#&$**&@iVZL5czYwTS.u92pfN1.O3IasC
                }
        reverse_proxy https://10.20.80.234:8007 {
             transport http {
                tls
                tls_trusted_ca_certs /caddy/proxy.pem
                            }
                                                }
}