Problem with caddy reverse proxying to bitwarden on diet pi

Krooms · September 10, 2022, 3:49pm

1. Output of `caddy version`:

caddy.Version=v2.6.0-beta.3

2. How I run Caddy:

As a service.

a. System environment:

Installed caddy as a service on a raspberry pi running dietpi OS
using this documentation(https://caddyserver.com/docs/install#debian-ubuntu-raspbian)

b. Command:

no commands, because it's running as a service

c. Service/unit/compose file:

not such file

d. My complete Caddy config:

passwordvault.grimsgrams.de {
        reverse_proxy 192.168.8.8:8001
                log {
                format console output file /var/log/caddy/bitwarden.log
        }

OR

passwordvault.grimsgrams.de {
        reverse_proxy 192.168.8.8:8083
                log {
                format console output file /var/log/caddy/bitwarden.log
        }

3. The problem I’m having:

Setting up caddy works well with the webserver and also works well, if I redirect to 192.168.8.8:8083 (AdGuard on my Raspberry).
With the webserver and linking to adguard, I get a secure HTTPS connection with a working certificate.
As soon as I change the redirecting to bitwarden (same IP as adguard, but Port 8001) connecting via passwordvault.grimsgrams.de fails (http error 502).
Connecting to bitwarden via the internal IP works fine (except for not having a secure connection, but that’s OK).

4. Error messages and/or full log output:

curl -v passwordvault.grimsgrams.de
*   Trying 91.47.234.79:80...
* Connected to passwordvault.grimsgrams.de (91.47.234.79) port 80 (#0)
> GET / HTTP/1.1
> Host: passwordvault.grimsgrams.de
> User-Agent: curl/7.74.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://passwordvault.grimsgrams.de/
< Server: Caddy
< Date: Sat, 10 Sep 2022 15:27:42 GMT
< Content-Length: 0
<
* Closing connection 0

journalctl --no-pager -u caddy
-- Journal begins at Sat 2022-09-10 17:35:40 CEST, ends at Sat 2022-09-10 17:36:42 CEST. --
Sep 10 17:36:00 KroomsPi systemd[1]: Starting Caddy...
Sep 10 17:36:02 KroomsPi caddy[398]: caddy.HomeDir=/var/lib/caddy
Sep 10 17:36:02 KroomsPi caddy[398]: caddy.AppDataDir=/var/lib/caddy/.local/share/caddy
Sep 10 17:36:02 KroomsPi caddy[398]: caddy.AppConfigDir=/var/lib/caddy/.config/caddy
Sep 10 17:36:02 KroomsPi caddy[398]: caddy.ConfigAutosavePath=/var/lib/caddy/.config/caddy/autosave.json
Sep 10 17:36:02 KroomsPi caddy[398]: caddy.Version=v2.6.0-beta.3 h1:2VFvttWPkJPEhXep+4vtDLcevGs2gS7z41ruJGymaYI=
Sep 10 17:36:02 KroomsPi caddy[398]: runtime.GOOS=linux
Sep 10 17:36:02 KroomsPi caddy[398]: runtime.GOARCH=arm64
Sep 10 17:36:02 KroomsPi caddy[398]: runtime.Compiler=gc
Sep 10 17:36:02 KroomsPi caddy[398]: runtime.NumCPU=4
Sep 10 17:36:02 KroomsPi caddy[398]: runtime.GOMAXPROCS=4
Sep 10 17:36:02 KroomsPi caddy[398]: runtime.Version=go1.19
Sep 10 17:36:02 KroomsPi caddy[398]: os.Getwd=/
Sep 10 17:36:02 KroomsPi caddy[398]: LANG=de_DE.UTF-8
Sep 10 17:36:02 KroomsPi caddy[398]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Sep 10 17:36:02 KroomsPi caddy[398]: NOTIFY_SOCKET=/run/systemd/notify
Sep 10 17:36:02 KroomsPi caddy[398]: HOME=/var/lib/caddy
Sep 10 17:36:02 KroomsPi caddy[398]: LOGNAME=caddy
Sep 10 17:36:02 KroomsPi caddy[398]: USER=caddy
Sep 10 17:36:02 KroomsPi caddy[398]: INVOCATION_ID=f1f42f34c32a478aa4034df7c1706d48
Sep 10 17:36:02 KroomsPi caddy[398]: JOURNAL_STREAM=8:10159
Sep 10 17:36:02 KroomsPi caddy[398]: {"level":"info","ts":1662824162.254439,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
Sep 10 17:36:02 KroomsPi caddy[398]: {"level":"info","ts":1662824162.2655158,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
Sep 10 17:36:02 KroomsPi caddy[398]: {"level":"info","ts":1662824162.2666066,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
Sep 10 17:36:02 KroomsPi caddy[398]: {"level":"info","ts":1662824162.2666938,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
Sep 10 17:36:02 KroomsPi caddy[398]: {"level":"info","ts":1662824162.269999,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/var/lib/caddy/.local/share/caddy"}
Sep 10 17:36:02 KroomsPi caddy[398]: {"level":"info","ts":1662824162.2701986,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
Sep 10 17:36:02 KroomsPi caddy[398]: {"level":"info","ts":1662824162.2707179,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x400050e460"}
Sep 10 17:36:02 KroomsPi caddy[398]: {"level":"info","ts":1662824162.2725594,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
Sep 10 17:36:02 KroomsPi caddy[398]: {"level":"info","ts":1662824162.2727833,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
Sep 10 17:36:02 KroomsPi caddy[398]: {"level":"info","ts":1662824162.2728179,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["passwordvault.grimsgrams.de"]}
Sep 10 17:36:02 KroomsPi caddy[398]: {"level":"info","ts":1662824162.292351,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/.config/caddy/autosave.json"}
Sep 10 17:36:02 KroomsPi caddy[398]: {"level":"info","ts":1662824162.2927136,"msg":"serving initial configuration"}
Sep 10 17:36:02 KroomsPi caddy[398]: {"level":"info","ts":1662824162.2928298,"logger":"tls","msg":"finished cleaning storage units"}
Sep 10 17:36:02 KroomsPi systemd[1]: Started Caddy.
Sep 10 17:36:22 KroomsPi caddy[398]: {"level":"error","ts":1662824182.93754,"logger":"http.log.error.log0","msg":"EOF","request":{"remote_ip":"91.47.234.79","remote_port":"60598","proto":"HTTP/2.0","method":"GET","host":"passwordvault.grimsgrams.de","uri":"/","headers":{"Accept-Encoding":["gzip, deflate, br"],"Sec-Ch-Ua-Platform":["\"Windows\""],"Sec-Fetch-Site":["none"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-User":["?1"],"Accept-Language":["de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"],"Dnt":["1"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.102 Safari/537.36 OPR/90.0.4480.84"],"Cache-Control":["max-age=0"],"Sec-Ch-Ua":["\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"104\", \"Opera\";v=\"90\""],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Fetch-Dest":["document"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"h2","server_name":"passwordvault.grimsgrams.de"}},"duration":0.02858424,"status":502,"err_id":"kqyp76gyg","err_trace":"reverseproxy.statusError (reverseproxy.go:1271)"}
Sep 10 17:36:34 KroomsPi caddy[398]: {"level":"error","ts":1662824194.363522,"logger":"http.log.error.log0","msg":"EOF","request":{"remote_ip":"91.47.234.79","remote_port":"60620","proto":"HTTP/2.0","method":"GET","host":"passwordvault.grimsgrams.de","uri":"/","headers":{"Upgrade-Insecure-Requests":["1"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Accept-Language":["de,de-DE;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6"],"Dnt":["1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 Edg/105.0.1343.33"],"Sec-Fetch-Site":["none"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-User":["?1"],"Sec-Ch-Ua":["\"Microsoft Edge\";v=\"105\", \" Not;A Brand\";v=\"99\", \"Chromium\";v=\"105\""],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Ch-Ua-Platform":["\"Windows\""],"Sec-Fetch-Dest":["document"],"Accept-Encoding":["gzip, deflate, br"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"h2","server_name":"passwordvault.grimsgrams.de"}},"duration":0.009199852,"status":502,"err_id":"durbqb1hj","err_trace":"reverseproxy.statusError (reverseproxy.go:1271)"}
Sep 10 17:36:42 KroomsPi caddy[398]: {"level":"error","ts":1662824202.1527026,"logger":"http.log.error.log0","msg":"EOF","request":{"remote_ip":"91.47.234.79","remote_port":"60620","proto":"HTTP/2.0","method":"GET","host":"passwordvault.grimsgrams.de","uri":"/","headers":{"Dnt":["1"],"Sec-Fetch-Site":["cross-site"],"Sec-Fetch-Dest":["document"],"Accept-Encoding":["gzip, deflate, br"],"Sec-Ch-Ua":["\"Microsoft Edge\";v=\"105\", \" Not;A Brand\";v=\"99\", \"Chromium\";v=\"105\""],"Sec-Ch-Ua-Mobile":["?0"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Accept-Language":["de,de-DE;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6"],"Cache-Control":["max-age=0"],"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-User":["?1"],"Sec-Ch-Ua-Platform":["\"Windows\""],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 Edg/105.0.1343.33"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"h2","server_name":"passwordvault.grimsgrams.de"}},"duration":0.004617,"status":502,"err_id":"pp0gq2q2r","err_trace":"reverseproxy.statusError (reverseproxy.go:1271)"}

5. What I already tried:

Read many discussions here and all over the internet.
I don’t get, why it works when I setup caddy as a webserver or link it to adguard (listening on port 8083), but fails when I try to redirect to bitwarden (listening on port 8001).

I increased the buffer size (sysctl -w net.core.rmem_max=2500000) to get rid of error “failed to sufficiently increase receive buffer size”.

6. Links to relevant resources:

Mohammed90 · September 10, 2022, 3:55pm

Why are you including log inline with reverse_proxy here (plus adding the log customization stuff in reverse_proxy block)?

This is what’s breaking your setup. log is its own directive. Adding log in this manner makes Caddy believe log is an upstream address and it’ll try to load-balance to it. Review the docs for reverse proxy and log for more information.

Krooms · September 10, 2022, 4:20pm

Thx for your reply.

I messed up the formatting of the Caddyfile pasting the code here.
I edited my start post.

To be save, I deleted the “log”-part in my Caddyfile so it looks like this now:

passwordvault.grimsgrams.de {
        reverse_proxy 192.168.8.8:8001
        }

Error is the same. HTTP error 502 with linking to bitwarden on port 8001, no problem with linking to adguard on port 8083.

I also set my raspberry as exposed host in my router to be sure there is no problem with port forwarding.

Mohammed90 · September 10, 2022, 11:15pm

Oh, ok, thanks for clearing that up! Are Caddy and Bitwarden on different hosts? If yes (they’re not different hosts), is the firewall on the host of Bitwarden blocking access to port 8001? While we’re at it, enable debug-level logging by adding this to the top of your Caddyfile:

{
    debug
}

Krooms · September 24, 2022, 9:13pm

Sorry for the late response, I was on vacation.

Caddy and Bitwarden are running on the same host. Adguard is also running on the same host.

Connecting to Bitwarden and Adguard via the local network is working, connecting over the internet via caddy works for Adguard and doesn’t work for Bitwarden (Error 502).

I’ve added the “debug”-code to my Caddyfile.

francislavoie · September 24, 2022, 9:46pm

And what do you see in your logs? We asked you to do that so you could show us the logs.

Krooms · September 28, 2022, 7:48pm

Here is my LOG:

-- Journal begins at Sun 2022-09-25 00:18:19 CEST, ends at Sun 2022-09-25 00:22:03 CEST. --
Sep 25 00:18:38 KroomsPi systemd[1]: Starting Caddy...
Sep 25 00:18:40 KroomsPi caddy[396]: caddy.HomeDir=/var/lib/caddy
Sep 25 00:18:40 KroomsPi caddy[396]: caddy.AppDataDir=/var/lib/caddy/.local/share/caddy
Sep 25 00:18:40 KroomsPi caddy[396]: caddy.AppConfigDir=/var/lib/caddy/.config/caddy
Sep 25 00:18:40 KroomsPi caddy[396]: caddy.ConfigAutosavePath=/var/lib/caddy/.config/caddy/autosave.json
Sep 25 00:18:40 KroomsPi caddy[396]: caddy.Version=v2.6.1 h1:EDqo59TyYWhXQnfde93Mmv4FJfYe00dO60zMiEt+pzo=
Sep 25 00:18:40 KroomsPi caddy[396]: runtime.GOOS=linux
Sep 25 00:18:40 KroomsPi caddy[396]: runtime.GOARCH=arm64
Sep 25 00:18:40 KroomsPi caddy[396]: runtime.Compiler=gc
Sep 25 00:18:40 KroomsPi caddy[396]: runtime.NumCPU=4
Sep 25 00:18:40 KroomsPi caddy[396]: runtime.GOMAXPROCS=4
Sep 25 00:18:40 KroomsPi caddy[396]: runtime.Version=go1.19.1
Sep 25 00:18:40 KroomsPi caddy[396]: os.Getwd=/
Sep 25 00:18:40 KroomsPi caddy[396]: LANG=de_DE.UTF-8
Sep 25 00:18:40 KroomsPi caddy[396]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Sep 25 00:18:40 KroomsPi caddy[396]: NOTIFY_SOCKET=/run/systemd/notify
Sep 25 00:18:40 KroomsPi caddy[396]: HOME=/var/lib/caddy
Sep 25 00:18:40 KroomsPi caddy[396]: LOGNAME=caddy
Sep 25 00:18:40 KroomsPi caddy[396]: USER=caddy
Sep 25 00:18:40 KroomsPi caddy[396]: INVOCATION_ID=c10e0de275bc45879be5db5c983d20e3
Sep 25 00:18:40 KroomsPi caddy[396]: JOURNAL_STREAM=8:11621
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"info","ts":1664057920.5705435,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"warn","ts":1664057920.5734434,"msg":"Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":3}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"info","ts":1664057920.578228,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"info","ts":1664057920.5793183,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"info","ts":1664057920.57939,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"info","ts":1664057920.5801876,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x400057caf0"}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"info","ts":1664057920.5823567,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/var/lib/caddy/.local/share/caddy"}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"info","ts":1664057920.582541,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"debug","ts":1664057920.5901093,"logger":"http","msg":"starting server loop","address":"[::]:443","tls":true,"http3":true}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"info","ts":1664057920.5901873,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"debug","ts":1664057920.590458,"logger":"http","msg":"starting server loop","address":"[::]:80","tls":false,"http3":false}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"info","ts":1664057920.5904896,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"info","ts":1664057920.590507,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["passwordvault.grimsgrams.de","pihole.grimsgrams.de"]}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"debug","ts":1664057920.6034493,"logger":"tls","msg":"loading managed certificate","domain":"passwordvault.grimsgrams.de","expiration":1670537570,"issuer_key":"acme-v02.api.letsencrypt.org-directory","storage":"FileStorage:/var/lib/caddy/.local/share/caddy"}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"debug","ts":1664057920.6097052,"logger":"tls.cache","msg":"added certificate to cache","subjects":["passwordvault.grimsgrams.de"],"expiration":1670537570,"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"e6970d889ad7d24595c58cf9ceca0f8a4b124452352b61334e2417cc303b4784","cache_size":1,"cache_capacity":10000}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"debug","ts":1664057920.6098135,"logger":"events","msg":"event","name":"cached_managed_cert","id":"6154e301-760d-4e5a-af75-d1b6cad8141a","origin":"tls","data":{"sans":["passwordvault.grimsgrams.de"]}}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"debug","ts":1664057920.6116226,"logger":"tls","msg":"loading managed certificate","domain":"pihole.grimsgrams.de","expiration":1671829186,"issuer_key":"acme-v02.api.letsencrypt.org-directory","storage":"FileStorage:/var/lib/caddy/.local/share/caddy"}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"info","ts":1664057920.613832,"logger":"tls","msg":"finished cleaning storage units"}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"debug","ts":1664057920.616058,"logger":"tls.cache","msg":"added certificate to cache","subjects":["pihole.grimsgrams.de"],"expiration":1671829186,"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"c21013a063d4c1a367ed800b548137170ae21cefe6edff5fabb42910b10d21bd","cache_size":2,"cache_capacity":10000}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"debug","ts":1664057920.6161718,"logger":"events","msg":"event","name":"cached_managed_cert","id":"cbfaabc8-8b26-4ee7-a58d-7835b37755a1","origin":"tls","data":{"sans":["pihole.grimsgrams.de"]}}
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"info","ts":1664057920.6176996,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/.config/caddy/autosave.json"}
Sep 25 00:18:40 KroomsPi systemd[1]: Started Caddy.
Sep 25 00:18:40 KroomsPi caddy[396]: {"level":"info","ts":1664057920.6195388,"msg":"serving initial configuration"}
Sep 25 00:21:56 KroomsPi caddy[396]: {"level":"debug","ts":1664058116.465516,"logger":"events","msg":"event","name":"tls_get_certificate","id":"07c8bb66-998c-460c-a4d0-c47659790a93","origin":"tls","data":{"client_hello":{"CipherSuites":[4865,4867,4866],"ServerName":"pihole.grimsgrams.de","SupportedCurves":[29,23,24,25,256,257,258,259,260],"SupportedPoints":null,"SignatureSchemes":[1027,1283,1539,515,2052,2053,2054,1025,1281,1537,513],"SupportedProtos":["h3"],"SupportedVersions":[772],"Conn":{}}}}
Sep 25 00:21:56 KroomsPi caddy[396]: {"level":"debug","ts":1664058116.4665167,"logger":"tls.handshake","msg":"choosing certificate","identifier":"pihole.grimsgrams.de","num_choices":1}
Sep 25 00:21:56 KroomsPi caddy[396]: {"level":"debug","ts":1664058116.4673607,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"pihole.grimsgrams.de","subjects":["pihole.grimsgrams.de"],"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"c21013a063d4c1a367ed800b548137170ae21cefe6edff5fabb42910b10d21bd"}
Sep 25 00:21:56 KroomsPi caddy[396]: {"level":"debug","ts":1664058116.4674842,"logger":"tls.handshake","msg":"matched certificate in cache","remote_ip":"99.51.213.14","remote_port":"49228","subjects":["pihole.grimsgrams.de"],"managed":true,"expiration":1671829186,"hash":"c21013a063d4c1a367ed800b548137170ae21cefe6edff5fabb42910b10d21bd"}
Sep 25 00:21:56 KroomsPi caddy[396]: {"level":"debug","ts":1664058116.4839773,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"192.168.8.8:8083","total_upstreams":1}
Sep 25 00:21:56 KroomsPi caddy[396]: {"level":"debug","ts":1664058116.4938273,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"192.168.8.8:8083","duration":0.008639963,"request":{"remote_ip":"99.51.213.14","remote_port":"49228","proto":"HTTP/3.0","method":"GET","host":"pihole.grimsgrams.de","uri":"/","headers":{"Sec-Fetch-Site":["none"],"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"Sec-Fetch-Mode":["navigate"],"X-Forwarded-Proto":["https"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"],"Alt-Used":["pihole.grimsgrams.de"],"Sec-Fetch-Dest":["document"],"User-Agent":["Mozilla/5.0 (iPhone; CPU iPhone OS 13_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.5 Mobile/15E148 Snapchat/10.77.5.59 (like Safari/604.1)"],"Sec-Fetch-User":["?1"],"X-Forwarded-For":["99.51.213.14"],"Dnt":["1"],"Upgrade-Insecure-Requests":["1"],"Priority":["u=1"],"Accept-Encoding":["gzip, deflate, br"],"X-Forwarded-Host":["pihole.grimsgrams.de"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","server_name":""}},"headers":{"Content-Length":["0"],"Location":["/login.html"],"Date":["Sat, 24 Sep 2022 22:21:56 GMT"]},"status":302}
Sep 25 00:21:56 KroomsPi caddy[396]: {"level":"debug","ts":1664058116.5061252,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"192.168.8.8:8083","total_upstreams":1}
Sep 25 00:21:56 KroomsPi caddy[396]: {"level":"debug","ts":1664058116.569924,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"192.168.8.8:8083","duration":0.062879445,"request":{"remote_ip":"99.51.213.14","remote_port":"49228","proto":"HTTP/3.0","method":"GET","host":"pihole.grimsgrams.de","uri":"/login.html","headers":{"Sec-Fetch-Site":["none"],"X-Forwarded-For":["99.51.213.14"],"Accept-Encoding":["gzip, deflate, br"],"Sec-Fetch-Dest":["document"],"Sec-Fetch-Mode":["navigate"],"User-Agent":["Mozilla/5.0 (iPhone; CPU iPhone OS 13_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.5 Mobile/15E148 Snapchat/10.77.5.59 (like Safari/604.1)"],"Upgrade-Insecure-Requests":["1"],"Priority":["u=1"],"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"Alt-Used":["pihole.grimsgrams.de"],"Dnt":["1"],"X-Forwarded-Proto":["https"],"X-Forwarded-Host":["pihole.grimsgrams.de"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"],"Sec-Fetch-User":["?1"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","server_name":""}},"headers":{"Accept-Ranges":["bytes"],"Content-Length":["848"],"Content-Type":["text/html; charset=utf-8"],"Vary":["Accept-Encoding"],"Date":["Sat, 24 Sep 2022 22:21:56 GMT"]},"status":200}
Sep 25 00:21:56 KroomsPi caddy[396]: {"level":"debug","ts":1664058116.6007109,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"192.168.8.8:8083","total_upstreams":1}
Sep 25 00:21:56 KroomsPi caddy[396]: {"level":"debug","ts":1664058116.6104825,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"192.168.8.8:8083","total_upstreams":1}
Sep 25 00:21:56 KroomsPi caddy[396]: {"level":"debug","ts":1664058116.6454313,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"192.168.8.8:8083","duration":0.044306371,"request":{"remote_ip":"99.51.213.14","remote_port":"49228","proto":"HTTP/3.0","method":"GET","host":"pihole.grimsgrams.de","uri":"/login.b36f973dfa3e44e7d110.js","headers":{"Accept-Encoding":["gzip, deflate, br"],"Sec-Fetch-Dest":["script"],"X-Forwarded-Proto":["https"],"User-Agent":["Mozilla/5.0 (iPhone; CPU iPhone OS 13_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.5 Mobile/15E148 Snapchat/10.77.5.59 (like Safari/604.1)"],"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"Dnt":["1"],"Sec-Fetch-Mode":["no-cors"],"Accept":["*/*"],"Sec-Fetch-Site":["same-origin"],"Alt-Used":["pihole.grimsgrams.de"],"Referer":["https://pihole.grimsgrams.de/login.html"],"X-Forwarded-For":["99.51.213.14"],"X-Forwarded-Host":["pihole.grimsgrams.de"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","server_name":""}},"headers":{"Content-Encoding":["gzip"],"Content-Type":["application/javascript"],"Date":["Sat, 24 Sep 2022 22:21:56 GMT"],"Accept-Ranges":["bytes"],"Vary":["Accept-Encoding"]},"status":200}
Sep 25 00:21:56 KroomsPi caddy[396]: {"level":"debug","ts":1664058116.682693,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"192.168.8.8:8083","duration":0.071742074,"request":{"remote_ip":"99.51.213.14","remote_port":"49228","proto":"HTTP/3.0","method":"GET","host":"pihole.grimsgrams.de","uri":"/login.b36f973dfa3e44e7d110.css","headers":{"Sec-Fetch-Dest":["style"],"X-Forwarded-For":["99.51.213.14"],"Alt-Used":["pihole.grimsgrams.de"],"X-Forwarded-Host":["pihole.grimsgrams.de"],"Sec-Fetch-Site":["same-origin"],"Priority":["u=2"],"Accept":["text/css,*/*;q=0.1"],"Referer":["https://pihole.grimsgrams.de/login.html"],"Accept-Encoding":["gzip, deflate, br"],"Dnt":["1"],"Sec-Fetch-Mode":["no-cors"],"User-Agent":["Mozilla/5.0 (iPhone; CPU iPhone OS 13_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.5 Mobile/15E148 Snapchat/10.77.5.59 (like Safari/604.1)"],"X-Forwarded-Proto":["https"],"Accept-Language":["de,en-US;q=0.7,en;q=0.3"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","server_name":""}},"headers":{"Content-Encoding":["gzip"],"Content-Type":["text/css; charset=utf-8"],"Date":["Sat, 24 Sep 2022 22:21:56 GMT"],"Accept-Ranges":["bytes"],"Vary":["Accept-Encoding"]},"status":200}
Sep 25 00:21:57 KroomsPi caddy[396]: {"level":"debug","ts":1664058117.1642697,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"192.168.8.8:8083","total_upstreams":1}
Sep 25 00:21:57 KroomsPi caddy[396]: {"level":"debug","ts":1664058117.165947,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"192.168.8.8:8083","total_upstreams":1}
Sep 25 00:21:57 KroomsPi caddy[396]: {"level":"debug","ts":1664058117.172446,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"192.168.8.8:8083","duration":0.00589126,"request":{"remote_ip":"99.51.213.14","remote_port":"49228","proto":"HTTP/3.0","method":"GET","host":"pihole.grimsgrams.de","uri":"/assets/apple-touch-icon-180x180.png","headers":{"User-Agent":["Mozilla/5.0 (iPhone; CPU iPhone OS 13_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.5 Mobile/15E148 Snapchat/10.77.5.59 (like Safari/604.1)"],"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"X-Forwarded-Proto":["https"],"Priority":["u=6"],"Referer":["https://pihole.grimsgrams.de/login.html"],"Sec-Fetch-Site":["same-origin"],"X-Forwarded-For":["99.51.213.14"],"X-Forwarded-Host":["pihole.grimsgrams.de"],"Sec-Fetch-Mode":["no-cors"],"Alt-Used":["pihole.grimsgrams.de"],"Accept-Encoding":["gzip, deflate, br"],"Dnt":["1"],"Accept":["image/avif,image/webp,*/*"],"Sec-Fetch-Dest":["image"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","server_name":""}},"headers":{"Content-Encoding":["gzip"],"Accept-Ranges":["bytes"],"Vary":["Accept-Encoding"],"Content-Type":["image/png"],"Date":["Sat, 24 Sep 2022 22:21:57 GMT"]},"status":200}
Sep 25 00:21:57 KroomsPi caddy[396]: {"level":"debug","ts":1664058117.1724465,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"192.168.8.8:8083","duration":0.006500315,"request":{"remote_ip":"99.51.213.14","remote_port":"49228","proto":"HTTP/3.0","method":"GET","host":"pihole.grimsgrams.de","uri":"/assets/favicon.png","headers":{"Priority":["u=6"],"Referer":["https://pihole.grimsgrams.de/login.html"],"X-Forwarded-For":["99.51.213.14"],"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"Accept-Encoding":["gzip, deflate, br"],"Sec-Fetch-Mode":["no-cors"],"Sec-Fetch-Site":["same-origin"],"Alt-Used":["pihole.grimsgrams.de"],"X-Forwarded-Proto":["https"],"X-Forwarded-Host":["pihole.grimsgrams.de"],"User-Agent":["Mozilla/5.0 (iPhone; CPU iPhone OS 13_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.5 Mobile/15E148 Snapchat/10.77.5.59 (like Safari/604.1)"],"Dnt":["1"],"Sec-Fetch-Dest":["image"],"Accept":["image/avif,image/webp,*/*"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","server_name":""}},"headers":{"Accept-Ranges":["bytes"],"Date":["Sat, 24 Sep 2022 22:21:57 GMT"],"Vary":["Accept-Encoding"],"Content-Length":["1296"],"Content-Type":["image/png"]},"status":200}
Sep 25 00:22:03 KroomsPi caddy[396]: {"level":"debug","ts":1664058123.3797922,"logger":"events","msg":"event","name":"tls_get_certificate","id":"36f3d150-cb84-47ec-bb59-f53ccdf08eb9","origin":"tls","data":{"client_hello":{"CipherSuites":[4865,4867,4866,49195,49199,52393,52392,49196,49200,49162,49161,49171,49172,156,157,47,53],"ServerName":"passwordvault.grimsgrams.de","SupportedCurves":[29,23,24,25,256,257],"SupportedPoints":"AA==","SignatureSchemes":[1027,1283,1539,2052,2053,2054,1025,1281,1537,515,513],"SupportedProtos":["h2","http/1.1"],"SupportedVersions":[772,771],"Conn":{}}}}
Sep 25 00:22:03 KroomsPi caddy[396]: {"level":"debug","ts":1664058123.3803349,"logger":"tls.handshake","msg":"choosing certificate","identifier":"passwordvault.grimsgrams.de","num_choices":1}
Sep 25 00:22:03 KroomsPi caddy[396]: {"level":"debug","ts":1664058123.3804743,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"passwordvault.grimsgrams.de","subjects":["passwordvault.grimsgrams.de"],"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"e6970d889ad7d24595c58cf9ceca0f8a4b124452352b61334e2417cc303b4784"}
Sep 25 00:22:03 KroomsPi caddy[396]: {"level":"debug","ts":1664058123.3805752,"logger":"tls.handshake","msg":"matched certificate in cache","remote_ip":"99.51.213.14","remote_port":"65277","subjects":["passwordvault.grimsgrams.de"],"managed":true,"expiration":1670537570,"hash":"e6970d889ad7d24595c58cf9ceca0f8a4b124452352b61334e2417cc303b4784"}
Sep 25 00:22:03 KroomsPi caddy[396]: {"level":"debug","ts":1664058123.4016612,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"192.168.8.8:8001","total_upstreams":1}
Sep 25 00:22:03 KroomsPi caddy[396]: {"level":"debug","ts":1664058123.4203072,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"192.168.8.8:8001","duration":0.017316648,"request":{"remote_ip":"99.51.213.14","remote_port":"65277","proto":"HTTP/2.0","method":"GET","host":"passwordvault.grimsgrams.de","uri":"/","headers":{"Sec-Fetch-Dest":["document"],"Sec-Fetch-Site":["none"],"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"X-Forwarded-Host":["passwordvault.grimsgrams.de"],"X-Forwarded-Proto":["https"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"],"Accept-Encoding":["gzip, deflate, br"],"Sec-Fetch-Mode":["navigate"],"User-Agent":["Mozilla/5.0 (iPhone; CPU iPhone OS 13_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.5 Mobile/15E148 Snapchat/10.77.5.59 (like Safari/604.1)"],"X-Forwarded-For":["99.51.213.14"],"Upgrade-Insecure-Requests":["1"],"Dnt":["1"],"Sec-Fetch-User":["?1"],"Te":["trailers"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"h2","server_name":"passwordvault.grimsgrams.de"}},"error":"EOF"}
Sep 25 00:22:03 KroomsPi caddy[396]: {"level":"error","ts":1664058123.4277337,"logger":"http.log.error","msg":"EOF","request":{"remote_ip":"99.51.213.14","remote_port":"65277","proto":"HTTP/2.0","method":"GET","host":"passwordvault.grimsgrams.de","uri":"/","headers":{"Accept-Encoding":["gzip, deflate, br"],"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-Mode":["navigate"],"Sec-Fetch-Site":["none"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"],"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"Dnt":["1"],"Sec-Fetch-Dest":["document"],"Sec-Fetch-User":["?1"],"Te":["trailers"],"User-Agent":["Mozilla/5.0 (iPhone; CPU iPhone OS 13_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.5 Mobile/15E148 Snapchat/10.77.5.59 (like Safari/604.1)"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"h2","server_name":"passwordvault.grimsgrams.de"}},"duration":0.026393259,"status":502,"err_id":"7ibw40azi","err_trace":"reverseproxy.statusError (reverseproxy.go:1271)"}
Sep 25 00:22:03 KroomsPi caddy[396]: {"level":"debug","ts":1664058123.4624364,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"192.168.8.8:8001","total_upstreams":1}
Sep 25 00:22:03 KroomsPi caddy[396]: {"level":"debug","ts":1664058123.4667258,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"192.168.8.8:8001","duration":0.003963278,"request":{"remote_ip":"99.51.213.14","remote_port":"65277","proto":"HTTP/2.0","method":"GET","host":"passwordvault.grimsgrams.de","uri":"/favicon.ico","headers":{"Accept-Encoding":["gzip, deflate, br"],"Sec-Fetch-Mode":["no-cors"],"Sec-Fetch-Site":["same-origin"],"X-Forwarded-For":["99.51.213.14"],"Dnt":["1"],"X-Forwarded-Proto":["https"],"Referer":["https://passwordvault.grimsgrams.de/"],"Sec-Fetch-Dest":["image"],"Accept":["image/avif,image/webp,*/*"],"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"Te":["trailers"],"User-Agent":["Mozilla/5.0 (iPhone; CPU iPhone OS 13_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.5 Mobile/15E148 Snapchat/10.77.5.59 (like Safari/604.1)"],"X-Forwarded-Host":["passwordvault.grimsgrams.de"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"h2","server_name":"passwordvault.grimsgrams.de"}},"error":"EOF"}
Sep 25 00:22:03 KroomsPi caddy[396]: {"level":"error","ts":1664058123.46695,"logger":"http.log.error","msg":"EOF","request":{"remote_ip":"99.51.213.14","remote_port":"65277","proto":"HTTP/2.0","method":"GET","host":"passwordvault.grimsgrams.de","uri":"/favicon.ico","headers":{"Accept":["image/avif,image/webp,*/*"],"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"Accept-Encoding":["gzip, deflate, br"],"Referer":["https://passwordvault.grimsgrams.de/"],"Sec-Fetch-Dest":["image"],"Sec-Fetch-Mode":["no-cors"],"User-Agent":["Mozilla/5.0 (iPhone; CPU iPhone OS 13_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.5 Mobile/15E148 Snapchat/10.77.5.59 (like Safari/604.1)"],"Dnt":["1"],"Sec-Fetch-Site":["same-origin"],"Te":["trailers"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"h2","server_name":"passwordvault.grimsgrams.de"}},"duration":0.00499113,"status":502,"err_id":"k7k2ybb4k","err_trace":"reverseproxy.statusError (reverseproxy.go:1271)"}

Krooms · October 7, 2022, 11:18pm

Anything you can see in these logs?

I have no idea, what the problem is.

emilylange · October 8, 2022, 1:11am

The EOF sounds a bit like your upstream (bitwarden) is closing the http connection early.
Are you using the official bitwarden server or vaultwarden?

Are we talking about http:// or https:// here?

Also, could you perhaps provide some curl examples to the bitwarden upstream?
E.g.

curl --head 192.168.8.8:8001 (or curl --head --insecure https://192.168.8.8:8001 for https://)
curl --head --header "Host: passwordvault.grimsgrams.de" 192.168.8.8:8001

system · October 10, 2022, 3:49pm

This topic was automatically closed after 30 days. New replies are no longer allowed.