Reverse Proxy to https with local ACME certificate

Help
1

1. Caddy version (caddy version):

v2.3.0 h1:fnrqJLa3G5vfxcxmOH/+kJOcunPLhSBnjgIvjXV/QTA=

2. How I run Caddy:

Caddy native

a. System environment:

Debian 10, systemd disabled, no Docker.

b. Command:

caddy run

c. Service/unit/compose file:

N/A

d. My complete Caddyfile or JSON config:

Frontend

# Global Option Block
{
        # General Option
        debug
}

# ACME Server
acme.roadrunner {
        acme_server
        tls internal
}

#
# Reverse proxy
#

bpass.intrafit.nl {
        reverse_proxy https://caddytest.roadrunner
}

Backend

# Global Option Block
{
        # General Option
        debug
}

caddytest.roadrunner {
        respond "Hello, this is your internal website @ 192.168.2.50"

        tls {
                ca https://acme.roadrunner/acme/local/directory
                ca_root /root/root.crt
        }
}

3. The problem I’m having:

When trying to access the backend through the frontend ie https://bpass.intrafit.nl, I get a blank page.

4. Error messages and/or full log output:

Frontend

root@RJ-CaddyTK ~# caddy run
2021/02/09 16:39:57.631 INFO using adjacent Caddyfile
2021/02/09 16:39:57.637 INFO admin admin endpoint started {“address”: “tcp/localhost:2019”, “enforce_origin”: false, “origins”: [“localhost:2019”, “[::1]:2019”, “127.0.0.1:2019”]}
2021/02/09 16:39:57.656 INFO tls.cache.maintenance started background certificate maintenance {“cache”: “0xc000229960”}
2021/02/09 16:39:57.683 INFO http server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {“server_name”: “srv0”, “https_port”: 443}
2021/02/09 16:39:57.686 INFO http enabling automatic HTTP->HTTPS redirects {“server_name”: “srv0”}
2021/02/09 16:39:57.777 INFO pki.ca.local root certificate is already trusted by system {“path”: “storage:pki/authorities/local/root.crt”}
2021/02/09 16:39:57.780 DEBUG http starting server loop {“address”: “[::]:443”, “http3”: false, “tls”: true}
2021/02/09 16:39:57.781 DEBUG http starting server loop {“address”: “[::]:80”, “http3”: false, “tls”: false}
2021/02/09 16:39:57.781 INFO http enabling automatic TLS certificate management {“domains”: [“bpass.intrafit.nl”, “acme.roadrunner”]}
2021/02/09 16:39:57.782 DEBUG tls loading managed certificate {“domain”: “bpass.intrafit.nl”, “expiration”: “2021/05/10 15:38:41.000”, “issuer_key”: “acme-v02.api.letsencrypt.org-directory”, “storage”: “FileStorage:/root/.local/share/caddy”}
2021/02/09 16:39:57.792 WARN tls stapling OCSP {“error”: “no OCSP stapling for [acme.roadrunner]: no OCSP server specified in certificate”}
2021/02/09 16:39:57.796 INFO autosaved config {“file”: “/root/.config/caddy/autosave.json”}
2021/02/09 16:39:57.797 INFO serving initial configuration
2021/02/09 16:39:57.791 INFO tls cleaned up storage units

Backend

root@RJ-Caddytest ~# caddy run
2021/02/09 16:39:33.577 INFO using adjacent Caddyfile
2021/02/09 16:39:33.583 INFO admin admin endpoint started {“address”: “tcp/localhost:2019”, “enforce_origin”: false, “origins”: [“localhost:2019”, “[::1]:2019”, “127.0.0.1:2019”]}
2021/02/09 16:39:33.587 INFO http server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {“server_name”: “srv0”, “https_port”: 443}
2021/02/09 16:39:33.590 INFO http enabling automatic HTTP->HTTPS redirects {“server_name”: “srv0”}
2021/02/09 16:39:33.591 DEBUG http starting server loop {“address”: “[::]:443”, “http3”: false, “tls”: true}
2021/02/09 16:39:33.593 DEBUG http starting server loop {“address”: “[::]:80”, “http3”: false, “tls”: false}
2021/02/09 16:39:33.595 INFO http enabling automatic TLS certificate management {“domains”: [“caddytest.roadrunner”]}
2021/02/09 16:39:33.593 INFO tls cleaned up storage units
2021/02/09 16:39:33.589 INFO tls.cache.maintenance started background certificate maintenance {“cache”: “0xc00022b810”}
2021/02/09 16:39:33.631 WARN tls stapling OCSP {“error”: “no OCSP stapling for [caddytest.roadrunner]: no OCSP server specified in certificate”}
2021/02/09 16:39:33.634 INFO autosaved config {“file”: “/root/.config/caddy/autosave.json”}
2021/02/09 16:39:33.635 INFO serving initial configuration

5. What I already tried:

I successfully tried to access the backend directly ie https://caddytest.roadrunner I do get proper a response, including the message that the CA is untrusted.
When I shut down the backend, I get a Page not working.
I tried several different Caddyfile setups, including the option

transport http {
   tls_insecure_skip_verify
}

6. Links to relevant resources:

  • List item
2

Ah - it’s because the Host header is passed through on reverse_proxy, so the backend thinks you’re making a request for bpass.intrafit.nl and not caddytest.roadrunner, so the host matcher doesn’t match.

To fix this, you need to override the Host header with the hostname in your proxy upstream. The reverse_proxy docs have an example for this at the bottom of the page:

Set the upstream Host header to the address of the upstream (by default, it will retain its original, incoming value):

reverse_proxy https://caddytest.roadrunner {
	header_up Host {http.reverse_proxy.upstream.hostport}
}
3

Thanks that works! I couldn’t figure it out and thought I was getting crazy since I had a working reverse proxy last week but that was when I was still using IP addresses in the Caddyfile :- :man_facepalming:

1 Like
4

@francislavoie can it be that header_up Host {http.reverse_proxy.upstream.hostport} will cause the external address https://bpass.intrafit.nl to change to the backend (internal) address https://dockertest.roadrunner?

Although it’s working with the test backend like in the initial post, when I implement this for Nextcloud, the webaddress changes. From my internal network this still resolves to the backend (local DNS) but with a certificate warning. Externally this abviously doesn’t work.

5

That placeholder specifically means “the proxy upstream’s host+port as configured”, so if you don’t configure it as dockertest.roadrunner, I don’t see how it would ever get that value. But maybe I misunderstand your question. Could you clarify what you mean?

6

Sorry for the confusion. Below the 2 Caddyfiles.

Frontend

nextcloud.intrafit.nl {
   reverse_proxy https://dockertest.roadrunner {
      header_up Host {http.reverse_proxy.upstream.hostport}
   }
}

Backend

dockertest.roadrunner {

           tls {
              ca https://acme.roadrunner/acme/local/directory
              ca_root /root/nextcloud/root.crt
              }


        root    * /var/www/html
        file_server

        php_fastcgi 127.0.0.1:9000
        header {
                # enable HSTS
                # Strict-Transport-Security max-age=31536000;
        }

        redir /.well-known/carddav /remote.php/dav 301
        redir /.well-known/caldav /remote.php/dav 301

        # .htaccess / data / config / ... shouldn't be accessible from outside
        @forbidden {
                path    /.htaccess
                path    /data/*
                path    /config/*
                path    /db_structure
                path    /.xml
                path    /README
                path    /3rdparty/*
                path    /lib/*
                path    /templates/*
                path    /occ
                path    /console.php
        }

        respond @forbidden 404
}

When I type in a browser http://nextcloud.intrafit.nl it changes the address into https://dockertest.roadrunner. As long as I’m in the LAN this works and the local ACME certificate for dockertest.roadrunner is used. The browser will just complain about the untrusted certificate. Externally this doesn’t work of course. I want that the trusted certificate for nextcloud.intrafit.nl is used and then proxied to the backend.

The initial test below is working as expected.

bpass.intrafit.nl {
   reverse_proxy https://caddytest.roadrunner {
      header_up Host {http.reverse_proxy.upstream.hostport}
   }
}

caddytest.roadrunner {
        respond "Hello, this is your internal website @ 192.168.2.50"

        tls {
                ca https://acme.roadrunner/acme/local/directory
                ca_root /root/root.crt
        }
}
7

In that case, you’ll need to change your Host header again when proxying to NextCloud so that it knows the real external hostname. NextCloud is probably building its URLs and such based on the hostname passed to it.

In your backend you’ll probably do something like this:

reverse_proxy nextcloud:80 {
	header_up Host "nextcloud.intrafit.nl"
}

But you didn’t include how you’re actually proxying/serving NextCloud so :man_shrugging:

To avoid the backend needing to hardcode the incoming host, you could set a header like X-Forwarded-Host to the original hostname, which you could then read from when re-setting the host to proxy down to your actual app – or maybe NextCloud even already supports X-Forwarded-Host, I dunno.

1 Like
8

Sorrry sorry, I should have included the full Caddydile that configures the Nextcloud node. I’ve updated the post above with the full file.

9

You can do this:

php_fastcgi 127.0.0.1:9000 {
	header_up Host "nextcloud.intrafit.nl"
}

Because php_fastcgi is actually a fancy shortcut/wrapper around reverse_proxy, so you can pass it most of the same options

2 Likes
10

@francislavoie thank you so much! I would never have figure this out without your help.

11

Like I said though - you may want to try with the X-Forwarded-Host approach in your frontend instance, it might let you avoid that extra line of config in the backend.

12

@francislavoie I also think its more clean to leave the backend setup alone and take care of in in the frontend so I tried to get some more info about X-Forwarded-Host but in the Caddy documentation there is not a lot. I never heard of it before and I understand it’s a (experimental) protocol thing? Nothing specific for Caddy.

Looking at the Nextcloud documentation I found that they support it. But I be fair, I don’t understand much of it.

A reverse proxy can define HTTP headers with the original client IP address, and Nextcloud can use those headers to retrieve that IP address. Nextcloud uses the de-facto standard header ‘X-Forwarded-For’ by default, but this can be configured with the forwarded_for_headers parameter. This parameter is an array of PHP lookup strings, for example ‘X-Forwarded-For’ becomes ‘HTTP_X_FORWARDED_FOR’. Incorrectly setting this parameter may allow clients to spoof their IP address as visible to Nextcloud, even when going through the trusted proxy! The correct value for this parameter is dependent on your proxy software.

https://docs.nextcloud.com/server/20/admin_manual/configuration_server/reverse_proxy_configuration.html

Before drowning in another thing, are you able to give a pointer to use X-Forwarded-Host, based on the above info?

Thanks,
Robbert

13

The X-Forwarded-For header isn’t the same thing. That one is for announcing to the upstream app the original IP address of the request. The X-Forwarded-Host would be to announce the original Host of the request.

Their docs don’t seem to mention support for X-Forwarded-Host, but it might support it anyways. If it doesn’t, then it’s still easy enough to work around it.

So basically in your frontend, you’ll just need to add this to your reverse_proxy block:

   reverse_proxy https://dockertest.roadrunner {
      header_up Host {http.reverse_proxy.upstream.hostport}
      header_up X-Forwarded-Host {host}
   }

And then remove the header_up Host "nextcloud.intrafit.nl" line from your backend instance; the X-Forwarded-For header should pass that through transparently to NextCloud.

If that doesn’t work, then the additional step is to add this instead to the backend:

php_fastcgi 127.0.0.1:9000 {
	header_up Host {header.X-Forwarded-Host}
}

This will have Caddy pull the host value from the header that the frontend set, so you can avoid hardcoding it in the backend. This should work even if NextCloud doesn’t support the header.

2 Likes
14

This works!

Thanks very much!

1 Like
15

This topic was automatically closed after 30 days. New replies are no longer allowed.