Caddy ipv6 reverse proxy throws "connect: invalid argument"

1. The problem I’m having:

I’m trying to create a simple reverse proxy to an HTTPS endpoint via ipv6 and it fails with a 502 error from the browser, with the caddy logs showing a connect: invalid argument error.

If I SSH into the caddy host server and do a curl to the same ipv6 endpoint using this command curl --interface eth0 https://[fe80::ea9f:80ff:fe46:cbfd] --insecure, it works as expected so I know that the endpoint is reachable.

2. Error messages and/or full log output:

DBG ts=1720236138.4832432 logger=events msg=event name=tls_get_certificate id=2c6ef356-eea2-4be1-b855-a9c7d8fffbf0 origin=tls data={"client_hello":{"CipherSuites":[4865,4866,4867],"ServerName":"main-router.domain.org","SupportedCurves":[25497,29,23,24],"SupportedPoints":null,"SignatureSchemes":[1027,2052,1025,1283,2053,1281,2054,1537,513],"SupportedProtos":["h3"],"SupportedVersions":[772],"RemoteAddr":{"IP":"192.168.1.220","Port":63740,"Zone":""},"LocalAddr":{"IP":"192.168.1.22","Port":443,"Zone":""}}}
DBG ts=1720236138.4834726 logger=tls.handshake msg=no matching certificates and no custom selection logic identifier=main-router.domain.org
DBG ts=1720236138.4834821 logger=tls.handshake msg=choosing certificate identifier=*.domain.org num_choices=1
DBG ts=1720236138.4834976 logger=tls.handshake msg=default certificate selection results identifier=*.domain.org subjects=["*.domain.org"] managed=true issuer_key=acme-v02.api.letsencrypt.org-directory hash=d011eff14afd649eccab10379efeed425d66ddcda96f11b93b40a407ad70871a
DBG ts=1720236138.4835072 logger=tls.handshake msg=matched certificate in cache remote_ip=192.168.1.220 remote_port=63740 subjects=["*.domain.org"] managed=true expiration=1724204063 hash=d011eff14afd649eccab10379efeed425d66ddcda96f11b93b40a407ad70871a
DBG ts=1720236149.9222312 logger=http.handlers.reverse_proxy msg=selected upstream dial=[fe80::ea9f:80ff:fe46:cbfd]:443 total_upstreams=1
DBG ts=1720236149.9225698 logger=http.handlers.reverse_proxy msg=upstream roundtrip upstream=[fe80::ea9f:80ff:fe46:cbfd]:443 duration=0.00025357 request={"remote_ip":"192.168.1.220","remote_port":"63740","client_ip":"192.168.1.220","proto":"HTTP/3.0","method":"GET","host":"router-main.domain.org","uri":"/","headers":{"Sec-Fetch-Site":["none"],"Sec-Fetch-Dest":["document"],"Upgrade-Insecure-Requests":["1"],"X-Forwarded-For":["192.168.1.220"],"X-Forwarded-Host":["router-main.domain.org"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36 Edg/126.0.0.0"],"Cookie":[],"Sec-Ch-Ua":["\"Not/A)Brand\";v=\"8\", \"Chromium\";v=\"126\", \"Microsoft Edge\";v=\"126\""],"Sec-Ch-Ua-Platform":["\"Windows\""],"Sec-Fetch-Mode":["navigate"],"Accept-Language":["en-US,en;q=0.9"],"Priority":["u=0, i"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"X-Forwarded-Proto":["https"],"Accept-Encoding":["gzip, deflate, br, zstd"],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Fetch-User":["?1"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h3","server_name":"main-router.domain.org"}} error=dial tcp [fe80::ea9f:80ff:fe46:cbfd]:443: connect: invalid argument
ERR ts=1720236149.9226553 logger=http.log.error msg=dial tcp [fe80::ea9f:80ff:fe46:cbfd]:443: connect: invalid argument request={"remote_ip":"192.168.1.220","remote_port":"63740","client_ip":"192.168.1.220","proto":"HTTP/3.0","method":"GET","host":"router-main.domain.org","uri":"/","headers":{"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-Dest":["document"],"Accept-Language":["en-US,en;q=0.9"],"Cookie":[],"Upgrade-Insecure-Requests":["1"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"],"Priority":["u=0, i"],"Sec-Ch-Ua":["\"Not/A)Brand\";v=\"8\", \"Chromium\";v=\"126\", \"Microsoft Edge\";v=\"126\""],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Ch-Ua-Platform":["\"Windows\""],"Accept-Encoding":["gzip, deflate, br, zstd"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36 Edg/126.0.0.0"],"Sec-Fetch-Site":["none"],"Sec-Fetch-User":["?1"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h3","server_name":"main-router.domain.org"}} duration=0.000499604 status=502 err_id=0j3a8qnin err_trace=reverseproxy.statusError (reverseproxy.go:1267)

3. Caddy version:

Version 2.7.6

4. How I installed and ran Caddy:

Using Docker

a. System environment:

Official docker container

b. Command:

None, just what the normal docker build command runs.

c. Service/unit/compose file:

N/A

d. My complete Caddy config:

{
    debug
}

*.domain.org {

    # Main Router
    @router-main host router-main.domain.org
    handle @router-main {
        reverse_proxy https://[fe80::ea9f:80ff:fe46:cbfd] {
            header_down -X-Frame-Options
            transport http {
                tls_insecure_skip_verify
            }
        }
    }

    respond 404
}

:80 {
    respond 404
}

5. Links to relevant resources:

See c++ - connect() returns "invalid argument" with ipv6 address - Stack Overflow

I think since it’s an fe80* address, it has extra requirements that aren’t fulfilled.

Caddy doesn’t currently have a way to specify which interface to use in reverse_proxy, so I guess it’s impossible to use link-local addresses for now.

I don’t use IPv6 at all, so I don’t know what to suggest, but you’ll probably need to find a different way. Can you use a different IPv6 address that isn’t link-local?

1 Like

According to the documentation it might be possible to add % to the address;

iereverse_proxy https://[fe80::ea9f:80ff:fe46:cbfd%eth0]

@francislavoie, I don’t think I have any other way, I need to be able to access my router via ipv6 and I need to use the link-local domain ipv6 address for that.

@hmoffatt, I tried using the interface in the ip in the way you suggested https://[fe80::ea9f:80ff:fe46:cbfd%eth0] and it fails unfortunately with the following error:

Error: adapting config using caddyfile: parsing caddyfile tokens for 'handle': parsing caddyfile tokens for 'reverse_proxy': parsing upstream 'https://[fe80::ea9f:80ff:fe46:cbfd%eth0]': parsing upstream URL: parse "https://[fe80::ea9f:80ff:fe46:cbfd%eth0]": invalid URL escape "%et", at /etc/caddy/Caddyfile:93, at /etc/caddy/Caddyfile:99

This method does work in curl as well using curl https://[fe80::ea9f:80ff:fe46:cbfd%eth0] --insecure but not in Caddy it seems.

Yeah the trouble with that syntax is that we use url.Parse to allow URL schemes like https:// but in URLs % means escapes. So it’s not compatible.

You could try this though:

reverse_proxy [fe80::ea9f:80ff:fe46:cbfd%eth0] {
	transport http {
		tls
	}
}

This drops https:// (if the address doesn’t contain :// it doesn’t try to url.Parse), and set the tls option of the transport to force on TLS. Using https:// is a shortcut for the transport + tls, but in this case I don’t think you can use that shortcut.

2 Likes

Ah yes you’re right, after your comment I looked further into the GO documentation and saw that you could specify the interface via TCP instead of HTTP.

I got it working using tcp6/[fe80::ea9f:80ff:fe46:cbfd%eth0]:443.

Thank you so much for your help!

1 Like

Awesome. I didn’t know that syntax was possible, good to confirm it does. I’ll update the reverse_proxy docs to mention that syntax.

Done:

1 Like