Using Caddy X-Forwarded-For to disable *Arr stack authentication on local network

1. The problem I’m having:

I have a stack of docker containers, and most of them are in 2 networks Bridge and MacVlan Config. I am using Caddy to reverse proxy some of this dockers.
For example the Sonarr and other *Arr stack , have the option to provide auth for requests outside of the network.

I am also using Authelia , to manage a single place to maintain authentication.
The expected outcome would be that requests proxied by Caddy , to be recognised as coming from the same network, thus not receiving the Arr auth page, and avoiding the double log in. In the same time , I want to leave the basic Arr auth in place, for the usecases that I make some mistakes, and access to those services does not pass through Caddy.

I guess that X-Fowarded-For should be populated with the Caddy ip also ? In fact , i Have no X-Forwarded-For header populated. If my understanding is incorrect I am open to suggestions.

2. Error messages and/or full log output:

{"level":"info","ts":1731581837.1196558,"logger":"http.log.access.log4","msg":"handled request","request":{"remote_ip":"212.54.141.21","remote_port":"57743","client_ip":"212.54.141.21","proto":"HTTP/3.0","method":"GET","host":"sonarr.thnds.ro","uri":"/","headers":{"Sec-Ch-Ua":["\"Chromium\";v=\"130\", \"Brave\";v=\"130\", \"Not?A_Brand\";v=\"99\""],"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-Mode":["navigate"],"Sec-Ch-Ua-Mobile":["?0"],"Accept-Language":["en-GB,en;q=0.6"],"Accept-Encoding":["gzip, deflate, br, zstd"],"Cookie":["REDACTED"],"Priority":["u=0, i"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8"],"Sec-Gpc":["1"],"Sec-Fetch-Site":["none"],"Sec-Fetch-User":["?1"],"Sec-Fetch-Dest":["document"],"Sec-Ch-Ua-Platform":["\"macOS\""]},"tls":{"resumed":true,"version":772,"cipher_suite":4865,"proto":"h3","server_name":"sonarr.thnds.ro"}},"bytes_read":0,"user_id":"","duration":0.000614253,"size":111,"status":302,"resp_headers":{"Content-Type":["text/html; charset=utf-8"],"Referrer-Policy":["strict-origin-when-cross-origin"],"Date":["Thu, 14 Nov 2024 10:57:16 GMT"],"Location":["https://permis.thnds.ro/?rd=https%3A%2F%2Fsonarr.thnds.ro%2F&rm=GET"],"Set-Cookie":["REDACTED"],"X-Frame-Options":["DENY"],"Content-Length":["111"],"X-Dns-Prefetch-Control":["off"],"Server":["Caddy"],"X-Content-Type-Options":["nosniff"],"Permissions-Policy":["accelerometer=(), autoplay=(), camera=(), display-capture=(), geolocation=(), gyroscope=(), keyboard-map=(), magnetometer=(), microphone=(), midi=(), payment=(), picture-in-picture=(), screen-wake-lock=(), sync-xhr=(), xr-spatial-tracking=(), interest-cohort=()"]}}

Curl example from within the sonarr docker instance

 # curl -vL sonarr.thnds.ro
* Host sonarr.thnds.ro:80 was resolved.
* IPv6: (none)
* IPv4: 212.54.141.21
*   Trying 212.54.141.21:80...
* Connected to sonarr.thnds.ro (212.54.141.21) port 80
> GET / HTTP/1.1
> Host: sonarr.thnds.ro
> User-Agent: curl/8.9.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://sonarr.thnds.ro/
< Server: Caddy
< Date: Thu, 14 Nov 2024 11:14:12 GMT
< Content-Length: 0
<
* shutting down connection #0
* Clear auth, redirects to port from 80 to 443
* Issue another request to this URL: 'https://sonarr.thnds.ro/'
* Host sonarr.thnds.ro:443 was resolved.
* IPv6: (none)
* IPv4: 212.54.141.21
*   Trying 212.54.141.21:443...
* Connected to sonarr.thnds.ro (212.54.141.21) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=sonarr.thnds.ro
*  start date: Nov 12 13:24:31 2024 GMT
*  expire date: Feb 10 13:24:30 2025 GMT
*  subjectAltName: host "sonarr.thnds.ro" matched cert's "sonarr.thnds.ro"
*  issuer: C=US; O=Let's Encrypt; CN=E5
*  SSL certificate verify ok.
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
*   Certificate level 1: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://sonarr.thnds.ro/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: sonarr.thnds.ro]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.9.1]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: sonarr.thnds.ro
> User-Agent: curl/8.9.1
> Accept: */*
>
* Request completely sent off
< HTTP/2 302
< alt-svc: h3=":443"; ma=2592000
< content-type: text/html; charset=utf-8
< date: Thu, 14 Nov 2024 11:14:12 GMT
< location: https://authelia.thnds.ro/?rd=https%3A%2F%2Fsonarr.thnds.ro%2F&rm=GET
< permissions-policy: accelerometer=(), autoplay=(), camera=(), display-capture=(), geolocation=(), gyroscope=(), keyboard-map=(), magnetometer=(), microphone=(), midi=(), payment=(), picture-in-picture=(), screen-wake-lock=(), sync-xhr=(), xr-spatial-tracking=(), interest-cohort=()
< referrer-policy: strict-origin-when-cross-origin
< server: Caddy
< set-cookie: authelia_session=729zMCq9_dituruFqW*52UzWK1tEphM2; expires=Thu, 14 Nov 2024 12:14:13 GMT; domain=thnds.ro; path=/; HttpOnly; secure; SameSite=Lax
< x-content-type-options: nosniff
< x-dns-prefetch-control: off
< x-frame-options: DENY
< content-length: 111
* Ignoring the response-body
<
* Connection #1 to host sonarr.thnds.ro left intact
* Issue another request to this URL: 'https://authelia.thnds.ro/?rd=https%3A%2F%2Fsonarr.thnds.ro%2F&rm=GET'
* Host authelia.thnds.ro:443 was resolved.
* IPv6: (none)
* IPv4: 212.54.141.21
*   Trying 212.54.141.21:443...
* Connected to authelia.thnds.ro (212.54.141.21) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=authelia.thnds.ro
*  start date: Nov 11 17:52:43 2024 GMT
*  expire date: Feb  9 17:52:42 2025 GMT
*  subjectAltName: host "authelia.thnds.ro" matched cert's "authelia.thnds.ro"
*  issuer: C=US; O=Let's Encrypt; CN=E5
*  SSL certificate verify ok.
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
*   Certificate level 1: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://authelia.thnds.ro/?rd=https%3A%2F%2Fsonarr.thnds.ro%2F&rm=GET
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: authelia.thnds.ro]
* [HTTP/2] [1] [:path: /?rd=https%3A%2F%2Fsonarr.thnds.ro%2F&rm=GET]
* [HTTP/2] [1] [user-agent: curl/8.9.1]
* [HTTP/2] [1] [accept: */*]
> GET /?rd=https%3A%2F%2Fsonarr.thnds.ro%2F&rm=GET HTTP/2
> Host: authelia.thnds.ro
> User-Agent: curl/8.9.1
> Accept: */*
>
* Request completely sent off
< HTTP/2 200
< alt-svc: h3=":443"; ma=2592000
< content-security-policy: default-src 'self'; frame-src 'none'; object-src 'none'; style-src 'self' 'nonce-4OSzM2V775GThzAbD3nRMPXUOJdeGvMa'; frame-ancestors 'none'; base-uri 'self'
< content-type: text/html; charset=utf-8
< date: Thu, 14 Nov 2024 11:14:12 GMT
< permissions-policy: accelerometer=(), autoplay=(), camera=(), display-capture=(), geolocation=(), gyroscope=(), keyboard-map=(), magnetometer=(), microphone=(), midi=(), payment=(), picture-in-picture=(), screen-wake-lock=(), sync-xhr=(), xr-spatial-tracking=(), interest-cohort=()
< referrer-policy: strict-origin-when-cross-origin
< server: Caddy
< x-content-type-options: nosniff
< x-dns-prefetch-control: off
< x-frame-options: DENY
< content-length: 1064
<
<!DOCTYPE html>
<html lang="en">
<head>
  <base href="https://authelia.thnds.ro/" />
  <meta property="csp-nonce" content="4OSzM2V775GThzAbD3nRMPXUOJdeGvMa" />
  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1" />
  <meta name="theme-color" content="#000000" />
  <meta name="description" content="Authelia login portal for your apps" />
  <link rel="manifest" href="./manifest.json" />
  <link rel="icon" href="./favicon.ico" />
  <title>Login - Authelia</title>
  <script type="module" crossorigin src="./static/js/index.BvAVkNRl.js"></script>
  <link rel="stylesheet" crossorigin href="./static/css/index.Bv3M1sZD.css">
</head>

<body
    data-basepath=""
    data-duoselfenrollment="false"
    data-logooverride="false"
    data-privacypolicyaccept=""
    data-privacypolicyurl=""
    data-rememberme="true"
    data-resetpassword="true"
    data-resetpasswordcustomurl=""
    data-theme="light"
>
  <noscript>You need to enable JavaScript to run this app.</noscript>
  <div id="root"></div>
</body>
</html>
* Connection #2 to host authelia.thnds.ro left intact

3. Caddy version:

v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=

4. How I installed and ran Caddy:

a. System environment:

  • Docker version 20.10.12, build e91ed57

  • Linux tiny 6.8.0-40-generic #40~22.04.3-Ubuntu SMP PREEMPT_DYNAMIC Tue Jul 30 17:30:19 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

b. Command:

docker compose up -d

c. Service/unit/compose file:

version: '3'

services:
  caddy:
    image: caddy:latest
    container_name: caddy
    hostname: caddy
    domainname: caddy
    restart: unless-stopped
    networks:
      Bbackend:
      Afrontend:
        ipv4_address: 192.168.1.12
    volumes:
      - /share/dockers/caddy/config:/config
      - /share/dockers/caddy/logs:/logs
      - /share/dockers/caddy/config/Caddyfile:/etc/caddy/Caddyfile
      - caddy_data:/data

networks:
  Bbackend:
    external: true
  Afrontend:
    external: true

volumes:
  caddy_data:
    external: true

d. My complete Caddy config:

(logging) {
	log {
		output file /logs/{args[0]}.log {
			format json
			roll_size 2mb
			roll_keep 30
			roll_keep_for 720h
		}
	}
}

(secure) {
	forward_auth {args.0} authelia:9141 {
		uri /api/verify?rd=https://authelia.thnds.ro
		copy_headers Remote-User Remote-Groups Remote-Name Remote-Email
	}
}

{
	servers {
		trusted_proxies static 192.168.1.0/24 172.18.0.0/16 172.17.0.0/16
		client_ip_headers X-Forwarded-For X-Real-IP
	}
}

authelia.thnds.ro {
	import logging permis
	reverse_proxy authelia:9141
}

sonarr.thnds.ro {
	import logging sonarr
	import secure *
	reverse_proxy sonarr:8989
}

:80, :443 {
	import logging catchall
	respond "Bye!" 403 {
		close
	}
}

5. Links to relevant resources:

Howdy @icemanftg, welcome to the Caddy community.

I could help you figure out how to configure Caddy to disable its own auth when a client is on the LAN, but this would leave double-logins in place for external requests, which is probably undesirable.

But what you want is to disable the *Arr auth for local clients, not Caddy auth. That has to be configured on the *Arr side itself and you’d have to refer to their documentation/support.

What I can tell you is that, generally speaking, you shouldn’t bother manually setting X-Forwarded-* in Caddy unless your app has specific implementation requirements that you’re trying to meet. Caddy handles setting X-Forwarded-For, X-Forwarded-Proto, and X-Forwarded-Host for you by default.

https://caddyserver.com/docs/caddyfile/directives/reverse_proxy#defaults

Additionally, you’ve set Caddy’s trusted proxies to just about every LAN slice. This isn’t strictly necessary unless you put another HTTP proxy in front of Caddy. If you’re having clients connect directly to Caddy (looks like you are from the diagram - the router is a NAT/packet forwarder, not a HTTP proxy), you can remove that as it has no effect. Alternatively if you do need it, you could use trusted_proxies static private_ranges as a shortcut for all those CIDRs.

https://caddyserver.com/docs/caddyfile/options#static

1 Like

Hi @Whitestrake ,

You are correct, disabling Caddy Auth is not what I desire to do .

I have already configured *Arr to disable auth for local clients.

The router is actually the physical WIFI router , It’s in the diagram just to show that the incoming requests on ports 80 and 443 are forwarded to Caddy .

Regarding the X-Forwarded-* being set by default, I understand that it should be the case, I just don’t see them being set. (Or maybe I’m looking in the wrong place) .

I guess my question was a bit unclear, because even tho *Arr should not use internal auth for requests from the same network, somehow it still does, and it leads me to think that the X-Forwarded-For may not be correctly populated.

You can check by enabling debug and inspecting the upstream roundtrip messages. They will list all the headers Caddy provides when it makes the request to the backend servers.

The way you’ve described it makes it seem more like the *Arr apps are broken and ignoring your configuration? If you’ve disabled auth for LAN and they’re still requiring it for LAN requests, how could Caddy possibly fix that? Am I missing something here?

1 Like

I will look at the debug logs thanks.

My guess was that my config was not correct and caddy does not add the local ip to the request, thus *Arr having no info of being called from the same LAN. I’ll check the messages and come back to you if I find something relevant there.

Is Caddy not itself within the LAN?

If Caddy didn’t add X-Forwarded-For information, or the *Arr apps ignored it, would not the request from Caddy come from a private IP address?

1 Like

Caddy is within the same LAN.
I formatted the log so I could read it.
The X-Forwaded-For was set to [“5.12.212.145”] , while I was hopping to be set to [“5.12.212.145”, “192.168.1.11”].

I’m starting to thing that i’m missing some knowledge and probably understanding the X-Forwarded-For scope incorrectly .

[
  {
    "level": "debug",
    "ts": 1732375991.6759307,
    "logger": "http.handlers.reverse_proxy",
    "msg": "upstream roundtrip",
    "upstream": "sonarr:8989",
    "duration": 0.000991788,
    "request": {
      "remote_ip": "5.12.212.145",
      "remote_port": "61130",
      "client_ip": "5.12.212.145",
      "proto": "HTTP/3.0",
      "method": "GET",
      "host": "sonarr.thnds.ro",
      "uri": "/api/v3/command",
      "headers": {
        "Accept-Encoding": [
          "gzip, deflate, br, zstd"
        ],
        "Sec-Gpc": [
          "1"
        ],
        "Remote-User": [
          "admin"
        ],
        "Remote-Name": [
          "opsie.doupsie"
        ],
        "X-Forwarded-Proto": [
          "https"
        ],
        "Sec-Fetch-Site": [
          "same-origin"
        ],
        "X-Api-Key": [
          "b560389926f04785a6s171b1347ec3ab"
        ],
        "Cookie": [
          "REDACTED"
        ],
        "Remote-Groups": [
          "admins,dev"
        ],
        "Remote-Email": [
          "oupsie.doupsie@gmail.com"
        ],
        "X-Forwarded-For": [
          "5.12.212.145"
        ],
        "Accept": [
          "*/*"
        ],
        "Referer": [
          "https://sonarr.thnds.ro/"
        ],
        "User-Agent": [
          "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
        ],
        "Priority": [
          "u=1, i"
        ],
        "Sec-Ch-Ua": [
          "\"Brave\";v=\"131\", \"Chromium\";v=\"131\", \"Not_A Brand\";v=\"24\""
        ],
        "Sec-Ch-Ua-Platform": [
          "\"macOS\""
        ],
        "X-Requested-With": [
          "XMLHttpRequest"
        ],
        "Sec-Ch-Ua-Mobile": [
          "?0"
        ],
        "Sec-Fetch-Dest": [
          "empty"
        ],
        "X-Forwarded-Host": [
          "sonarr.thnds.ro"
        ],
        "Accept-Language": [
          "en-GB,en;q=0.6"
        ],
        "Sec-Fetch-Mode": [
          "cors"
        ]
      },
      "tls": {
        "resumed": true,
        "version": 772,
        "cipher_suite": 4865,
        "proto": "h3",
        "server_name": "sonarr.thnds.ro"
      }
    },
    "headers": {
      "Pragma": [
        "no-cache"
      ],
      "X-Application-Version": [
        "4.0.10.2544"
      ],
      "Content-Type": [
        "application/json; charset=utf-8"
      ],
      "Date": [
        "Sat, 23 Nov 2024 15:33:10 GMT"
      ],
      "Server": [
        "Kestrel"
      ],
      "Cache-Control": [
        "no-cache, no-store"
      ],
      "Expires": [
        "-1"
      ]
    },
    "status": 200
  }
]

I see:

and:

Which makes me think that this log, at least, is for a successful request. Do you have any logs of unsuccessful requests?

Should it also contain caddys 192.168.1.11 IP ?

{
  "level": "debug",
  "ts": 1732376640.0494282,
  "logger": "http.handlers.reverse_proxy",
  "msg": "upstream roundtrip",
  "upstream": "authelia:9141",
  "duration": 0.000810471,
  "request": {
    "remote_ip": "5.12.212.145",
    "remote_port": "57944",
    "client_ip": "5.12.212.145",
    "proto": "HTTP/1.1",
    "method": "GET",
    "host": "sonarr.thnds.ro",
    "uri": "/api/verify?rd=https://permis.thnds.ro",
    "headers": {
      "Accept": [
        "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
      ],
      "User-Agent": [
        "Uptime-Kuma/1.23.15"
      ],
      "X-Forwarded-Method": [
        "GET"
      ],
      "X-Forwarded-Uri": [
        "/"
      ],
      "X-Forwarded-For": [
        "5.12.212.145"
      ],
      "X-Forwarded-Proto": [
        "https"
      ],
      "X-Forwarded-Host": [
        "sonarr.thnds.ro"
      ]
    },
    "tls": {
      "resumed": false,
      "version": 772,
      "cipher_suite": 4865,
      "proto": "",
      "server_name": "sonarr.thnds.ro"
    }
  },
  "headers": {
    "Content-Length": [
      "111"
    ],
    "X-Frame-Options": [
      "DENY"
    ],
    "Set-Cookie": [
      "REDACTED"
    ],
    "Permissions-Policy": [
      "accelerometer=(), autoplay=(), camera=(), display-capture=(), geolocation=(), gyroscope=(), keyboard-map=(), magnetometer=(), microphone=(), midi=(), payment=(), picture-in-picture=(), screen-wake-lock=(), sync-xhr=(), xr-spatial-tracking=(), interest-cohort=()"
    ],
    "X-Dns-Prefetch-Control": [
      "off"
    ],
    "Location": [
      "https://permis.thnds.ro/?rd=https%3A%2F%2Fsonarr.thnds.ro%2F&rm=GET"
    ],
    "Date": [
      "Sat, 23 Nov 2024 15:43:59 GMT"
    ],
    "Content-Type": [
      "text/html; charset=utf-8"
    ],
    "X-Content-Type-Options": [
      "nosniff"
    ],
    "Referrer-Policy": [
      "strict-origin-when-cross-origin"
    ]
  },
  "status": 302
}

No, because Caddy isn’t forwarding for itself. It’s forwarding on behalf of its own client.

The server on the other end can see Caddy’s IP because the request is coming from Caddy’s IP. It’s the direct peer.

If you check the logs for your upstream server they should show the request originating from Caddy.

In the case of your latest 302 debug, that came from Authelia rather than from any of the *Arrs.

Did you have a read of the Authelia docs?

In particular this is important for Access Control Rules as the network criteria relies on the X-Forwarded-For header. This header is expected to have a true representation of the client’s actual IP address.

Forwarded Headers | Integration | Authelia

This criteria is a list of values which can be an IP Address, network address range in CIDR notation, or an alias from the global section. It matches against the first address in the X-Forwarded-For header, or if there are none it will fall back to the IP address of the packet TCP source IP address. For this reason it’s important for you to configure the proxy server correctly in order to accurately match requests with this criteria. Note: you may combine CIDR networks with the alias rules as you please.

Access Control | Configuration | Authelia

Authelia utilises the X-Forwarded-For from Caddy to determine that the client is an external IP. If you have Authelia requiring a login for external IPs, you’ll have it knocking back Caddy’s clients if Caddy’s clients are external IPs.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.