Reverse proxy for ssh tunnel

Help
1

1. The problem I’m having:

Hi, I want to use Caddy as a reverse proxy on example.com for my server (localserver) that I run at home behind a always changing ipv6 adress.

As a first try I used

autossh -M 0 -f -o ConnectTimeout=10 -o ServerAliveInterval=60 -o ServerAliveCountMax=2 -i /home/user/.ssh/id_ed25519 -N -R 8888:localserver:8000 example.com

where the localserver was reachable in the internet on example.com:8888. In the next step I want to utilize Caddy to have it reachable on example.com and ssl encrypted. My current Caddyfile is not working as intented. Can someone help?

2. Error messages and/or full log output:

no entries

3. Caddy version:

2.6.2

4. How I installed and ran Caddy:

ubuntu 22.04 verison package sources

a. System environment:

Linux ubuntu 6.8.0-49-generic #49-Ubuntu SMP PREEMPT_DYNAMIC x86_64 x86_64 x86_64 GNU/Linux

b. Command:

 sudo caddy run --config /etc/caddy/Caddyfile

c. Error Messages

2024/11/25 09:50:07.353 ERROR   http.log.error  dial tcp [ipv6-IP]:8888: connect: connection refused      {"request": {"remote_ip": "ipv4", "remote_port": "53918", "proto": "HTTP/2.0", "method": "GET", "host": "example.com", "uri": "/", "headers": {"Sec-Fetch-Mode": ["navigate"], "Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"], "Accept-Language": ["de,en-US;q=0.7,en;q=0.3"], "Accept-Encoding": ["gzip, deflate, br, zstd"], "Dnt": ["1"], "Cookie": [], "Upgrade-Insecure-Requests": ["1"], "Sec-Fetch-Dest": ["document"], "Sec-Fetch-Site": ["none"], "User-Agent": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:132.0) Gecko/20100101 Firefox/132.0"], "Priority": ["u=0, i"], "Te": ["trailers"], "Sec-Fetch-User": ["?1"]}, "tls": {"resumed": false, "version": 772, "cipher_suite": 4865, "proto": "h2", "server_name": "example.com"}}, "duration": 0.015068564, "status": 502, "err_id": "1uqd76vsu", "err_trace": "reverseproxy.statusError (reverseproxy.go:1272)"}
2024/11/25 09:50:07.442 ERROR   http.log.error  dial tcp [ipv6-IP]:8888: connect: connection refused      {"request": {"remote_ip": "ipv4", "remote_port": "53918", "proto": "HTTP/2.0", "method": "GET", "host": "example.com", "uri": "/favicon.ico", "headers": {"Sec-Fetch-Dest": ["image"], "User-Agent": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:132.0) Gecko/20100101 Firefox/132.0"], "Accept": ["image/avif,image/webp,image/png,image/svg+xml,image/*;q=0.8,*/*;q=0.5"], "Referer": ["https://example.com/"], "Cookie": [], "Sec-Fetch-Mode": ["no-cors"], "Sec-Fetch-Site": ["same-origin"], "Priority": ["u=6"], "Te": ["trailers"], "Accept-Language": ["de,en-US;q=0.7,en;q=0.3"], "Accept-Encoding": ["gzip, deflate, br, zstd"], "Dnt": ["1"]}, "tls": {"resumed": false, "version": 772, "cipher_suite": 4865, "proto": "h2", "server_name": "example.com"}}, "duration": 0.000841018, "status": 502, "err_id": "du61zvrt2", "err_trace": "reverseproxy.statusError (reverseproxy.go:1272)"}

d. My complete Caddy config:

example.com {
        reverse_proxy example.com:8888
}

5. Links to relevant resources:

2

That’s a really old version. Please use our official installation instructions: Install — Caddy Documentation

You shouldn’t run Caddy directly like this. Run it as a systemd service: Keep Caddy Running — Caddy Documentation

If you have a tunnel, do you mean to do reverse_proxy localhost:8888 instead, which would then pass through the tunnel listening on port 8888?

3

Hi Francis,
thanks for your quick answer. I updated Caddy to: v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk= and it runs as a systemd service now.
I tried changing example.com to localhost, still not the expected behaviour.

{"level":"error","ts":1732538610.5072064,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"1.2.3.4","remote_port":"9455","client_ip":"1.2.3.4","proto":"HTTP/2.0","method":"GET","host":"example.ceom","uri":"/","headers":{"Dnt":["1"],"Cookie":["REDACTED"],"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-Dest":["document"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-Site":["none"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:132.0) Gecko/20100101 Firefox/132.0"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"],"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"Accept-Encoding":["gzip, deflate, br, zstd"],"Sec-Fetch-User":["?1"],"Priority":["u=0, i"],"Te":["trailers"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"example.ceom"}},"bytes_read":0,"user_id":"","duration":0.000329726,"size":0,"status":502,"resp_headers":{"Server":["Caddy"],"Alt-Svc":["h3=\":443\"; ma=2592000"]}}
{"level":"info","ts":1732538611.9577575,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"1.2.3.4","remote_port":"9455","client_ip":"1.2.3.4","proto":"HTTP/2.0","method":"GET","host":"example.ceom","uri":"/favicon.ico","headers":{"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"Accept-Encoding":["gzip, deflate, br, zstd"],"Dnt":["1"],"Sec-Fetch-Dest":["image"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:132.0) Gecko/20100101 Firefox/132.0"],"Referer":["https://example.ceom/"],"Cookie":["REDACTED"],"Sec-Fetch-Mode":["no-cors"],"Sec-Fetch-Site":["same-origin"],"Priority":["u=6"],"Te":["trailers"],"Accept":["image/avif,image/webp,image/png,image/svg+xml,image/*;q=0.8,*/*;q=0.5"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"example.ceom"}},"bytes_read":0,"user_id":"","duration":0.000062918,"size":0,"status":0,"resp_headers":{"Server":["Caddy"],"Alt-Svc":["h3=\":443\"; ma=2592000"]}}```
4

I think you changed the wrong one. I mean the reverse_proxy one, not your site address.

5

Hi Francis,
sadyl I changed the right one to localhost, so no easy fix…
I made some changes to my config file in the meanwhile. For a start, serving a static html file in /var/www/ works with the standard config. I added some lines for logging.

        root * /var/www
        # Enable the static file server.
        file_server
        log {
                output file /var/log/caddy/log.log {
                        roll_size 10mb
                        roll_keep 20
                        roll_keep_for 720h
                }
        }
}

So the basics work. Can I add a reverse proxy to example.com/p with this lines?

        reverse_proxy /p/* localhost:8888

If I add them, I still get a 404 error. I’m starting to think it is an issue with firewall or system. I had the same configuration running with the same localserver, ssh tunnel and a other public linux vps running apache…

6

Could you please:

  • Add debug to your global options
  • Run curl -vL against your website, with the intent of being routed to the reverse proxy (example.com/p/test or as appropriate)
  • Post the output of curl as well as the logs produced by Caddy for that request
7

Done:

$ usr@srv curl -vL example.com/p
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Host example.com:80 was resolved.
* IPv6: 2a01:111:111:300::1
* IPv4: 12.122.12.12
*   Trying [2a01:111:111:300::1]:80...
* Connected to example.com (2a01:111:111:300::1) port 80
> GET /p HTTP/1.1
> Host: example.com
> User-Agent: curl/8.5.0
> Accept: */*
> 
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://example.com/p
< Server: Caddy
< Date: Tue, 26 Nov 2024 08:56:32 GMT
< Content-Length: 0
< 

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Closing connection
* Clear auth, redirects to port from 80 to 443
* Issue another request to this URL: 'https://example.com/p'
* Host example.com:443 was resolved.
* IPv6: 2a01:111:111:300::1
* IPv4: 12.122.12.12
*   Trying [2a01:111:111:300::1]:443...
* Connected to example.com (2a01:111:111:300::1) port 443
* ALPN: curl offers h2,http/1.1
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [15 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [2021 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [78 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=example.com
*  start date: Nov 24 21:38:28 2024 GMT
*  expire date: Feb 22 21:38:27 2025 GMT
*  subjectAltName: host "example.com" matched cert's "example.com"
*  issuer: C=US; O=Let's Encrypt; CN=E5
*  SSL certificate verify ok.
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
*   Certificate level 1: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [122 bytes data]
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://example.com/p
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: example.com]
* [HTTP/2] [1] [:path: /p]
* [HTTP/2] [1] [user-agent: curl/8.5.0]
* [HTTP/2] [1] [accept: */*]
} [5 bytes data]
> GET /p HTTP/2
> Host: example.com
> User-Agent: curl/8.5.0
> Accept: */*
> 
{ [5 bytes data]
< HTTP/2 404 
< alt-svc: h3=":443"; ma=2592000
< server: Caddy
< content-length: 0
< date: Tue, 26 Nov 2024 08:56:32 GMT
< 

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Connection #1 to host example.com left intact
8

You made a request to example.com/p, but you configured your reverse proxy for /p/*:

/p doesn’t match /p/*, so that request isn’t being sent to the reverse_proxy and is instead being handled by the file_server, which explains the 404.

Try making a request to /p/ or /p/foo instead if you want the request to be handled by the reverse_proxy.

9

Thanks for the hints, Matthew. Now it is working!

{
        debug
}

example.com {
        root * /var/www
        file_server
        reverse_proxy /p/ localhost:8888
        log {
                output file /var/log/caddy/log.log {
                        roll_size 10mb
                        roll_keep 20
                        roll_keep_for 720h
                }
        }
}

curl says:

usr@srv:~$ curl -vL example.com/p/
* Host example.com:80 was resolved.
* IPv6: 2a01:111:111:111::1
* IPv4: 12.122.12.12
*   Trying [2a01:111:111:111::1]:80...
* Connected to example.com (2a01:111:111:111::1) port 80
> GET /p/ HTTP/1.1
> Host: example.com
> User-Agent: curl/8.5.0
> Accept: */*
>
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://example.com/p/
< Server: Caddy
< Date: Tue, 26 Nov 2024 13:15:14 GMT
< Content-Length: 0
<
* Closing connection
* Clear auth, redirects to port from 80 to 443
* Issue another request to this URL: 'https://example.com/p/'
* Host example.com:443 was resolved.
* IPv6: 2a01:111:111:111::1
* IPv4: 12.122.12.12
*   Trying [2a01:111:111:111::1]:443...
* Connected to example.com (2a01:111:111:111::1) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=example.com
*  start date: Nov 24 21:38:28 2024 GMT
*  expire date: Feb 22 21:38:27 2025 GMT
*  subjectAltName: host "example.com" matched cert's "example.com"
*  issuer: C=US; O=Let's Encrypt; CN=E5
*  SSL certificate verify ok.
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
*   Certificate level 1: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://example.com/p/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: example.com]
* [HTTP/2] [1] [:path: /p/]
* [HTTP/2] [1] [user-agent: curl/8.5.0]
* [HTTP/2] [1] [accept: */*]
> GET /p/ HTTP/2
> Host: example.com
> User-Agent: curl/8.5.0
> Accept: */*
>
< HTTP/2 200
< accept-ranges: bytes
< alt-svc: h3=":443"; ma=2592000
< content-type: text/html; charset=utf-8
< date: Tue, 26 Nov 2024 13:15:14 GMT
< last-modified: Sun, 23 Jun 2024 19:30:38 GMT
< server: Caddy
< vary: Accept-Encoding
<
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <link rel="icon" href="/photoview-logo.svg" type="image/svg+xml" />
    <link rel="apple-touch-icon" href="/logo192.png" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <meta name="theme-color" content="#000000" />

    <link rel="manifest" href="/manifest.json" />

    <meta name="apple-mobile-web-app-title" content="Photoview" />
    <meta name="apple-mobile-web-app-capable" content="yes" />
    <meta name="apple-mobile-web-app-status-bar-style" content="white" />

    <script type="module" crossorigin src="/assets/index.8d4c57fa.js"></script>
    <link rel="stylesheet" href="/assets/index.0ac106ed.css">
  </head>
  <body>
    <noscript>You need to enable JavaScript to run Photoview.</noscript>
    <div id="root"></div>

  </body>
</html>
* Connection #1 to host example.com left intact

but only with example.com/p/ and not with example.com/p
I thought in context of urls the behavioru should be the same?

One new issue is the relaive paths of files are not found. The browser says:

Loading of modules from  "https:/example.com/assets/index.8d4c57fa.js" is blocked because of MIME types.

How can I point in to the url including the /p/?
"https:/example.com/p/assets/index.8d4c57fa.js"

10

This will now match only EXACTLY requests for /p/ and will not match requests for /p/foo or /p/bar, nor will it match /p, as you note.

Not at all. Path matchers are exact, literal matchers. Refer to the documentation:

Path matching is an exact match by default, not a prefix match. You must append a * for a fast prefix match. Note that /foo* will match /foo and /foo/ as well as /foobar; you might actually want /foo/* instead.

—https://caddyserver.com/docs/caddyfile/matchers#path-matchers

/p/* would match the path /p/assets/index.8d4c57fa.js.