Unsolicited response received on idle HTTP channel

hevisko · May 6, 2021, 9:44am

1. Caddy version (`caddy version`):

v2.4.0-beta.1 h1:Ed/tIaN3p6z8M3pEiXWJL/T8JmCqV62FrSJCHKquW/I=

2. How I run Caddy:

a. System environment:

Inside a LXC container on ProxMox, behind FortiGate VM
Linux caddy01 5.4.103-1-pve #1 SMP PVE 5.4.103-1 (Sun, 07 Mar 2021 15:55:09 +0100) x86_64 GNU/Linux

b. Command:

systemctl restart caddy

c. Service/unit/compose file:


# cat /etc/systemd/system/caddy.service

[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target

[Service]
Environment="CLOUDFLARE_API_TOKEN=...."
User=caddy
Group=caddy
ExecStart=/usr/local/bin/caddy-downloaded run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/local/bin/caddy-downloaded reload --config /etc/caddy/Caddyfile
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
#ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

[Install]

d. My complete Caddyfile or JSON config:

(dnstls) {
        #define this token in systemd service file
        tls {
                dns cloudflare {env.CLOUDFLARE_API_TOKEN}
        }
}
(statics) {
        redir /favicon* https://statics.tracks4africa.co.za{uri}
        respond /robots.txt 200 {
                body "User-agent: *
Allow: /
Allow: /media/users/images/
Disallow: /maps/africa/
Disallow: /maps/trips/
Disallow: /maps/corrections/
Disallow: /listings/advanced_search/
Disallow: /listings/getlisted/
Disallow: /listings/category_search/
Disallow: /listings/link/
Disallow: /listings/correction/
Disallow: /listings/photos/
Disallow: /listings/comments/
Disallow: /listings/media/
Disallow: /listings/map/
Disallow: /accounts/
"
                close
        }
}

import /etc/caddy/sites-enabled/*

/etc/caddy/sites-enabled# for i in *; do echo '-------'; echo $i ;echo "===========";/usr/local/bin/caddy-downloaded fmt $i;done
-------
django_t4a_retailers.conf
===========
#Caddyfile format for each of the vhosts

#InternalDomain
#ExternalDomain
retailers.tracks4africa.co.za {
        import statics
        log {
                output file /var/log/caddy/retailers.tracks4africa.co.za-access.log
        }
        reverse_proxy {
                # django-2021-other01
                #-
                #==
                to 10.131.32.30:50010

                # stage-django-2021-other01
                # localhost
                #-
                #==
                header_up +USWGI_SCHEME https
        }
        # charset                 utf-8
        # client_max_body_size    25M
}
-------
django_t4a_t4a-2021-django.conf
===========
#Caddyfile format
#This is specific to the stage.tracks4africa.co.za / tracks4africa.co.za only


#InternalDomain
tracks4africa.net {
        import statics
        import dnstls
        log {
                output file /var/log/caddy/tracks4africa.net-access.log
        }
        #return      301 https://$host$request_uri
        reverse_proxy {
                # t4a-2021-django01
                to 10.131.32.20:40001
                # t4a-2021-django02
                # stage-t4a-2021-django01
                header_up +USWGI_SCHEME https
        }
        # charset                 utf-8
        # client_max_body_size    25M
}
#ExternalDomain
tracks4africa.co.za {
        import statics
        log {
                output file /var/log/caddy/tracks4africa.co.za-access.log
        }
        #return      301 https://$host$request_uri
        reverse_proxy {
                # t4a-2021-django01
                to 10.131.32.20:40001
                # t4a-2021-django02
                # stage-t4a-2021-django01
                header_up +USWGI_SCHEME https
        }
        # charset                 utf-8
        # client_max_body_size    25M
}
-------
django_t4a_t4a-guide-app.conf
===========
#Caddyfile format for each of the vhosts

#InternalDomain
#ExternalDomain
t4a-guide-app.tracks4africa.co.za {
        import statics
        log {
                output file /var/log/caddy/t4a-guide-app.tracks4africa.co.za-access.log
        }
        reverse_proxy {
                # django-2021-other01
                #-
                #==
                to 10.131.32.30:50007

                # stage-django-2021-other01
                # localhost
                #-
                #==
                header_up +USWGI_SCHEME https
        }
        # charset                 utf-8
        # client_max_body_size    25M
}
-------
django_t4a_t4a_submission_api.conf
===========
#Caddyfile format for each of the vhosts

#InternalDomain
submission-api.tracks4africa.net {
        import statics
        import dnstls
        log {
                output file /var/log/caddy/submission-api.tracks4africa.net-access.log
        }
        reverse_proxy {
                # django-2021-other01
                #-
                #==
                to 10.131.32.30:60001

                # stage-django-2021-other01
                # localhost
                #-
                #==
                header_up +USWGI_SCHEME https
        }
        # charset                 utf-8
        # client_max_body_size    25M
}
#ExternalDomain
submission-api.tracks4africa.co.za {
        import statics
        log {
                output file /var/log/caddy/submission-api.tracks4africa.co.za-access.log
        }
        reverse_proxy {
                # django-2021-other01
                #-
                #==
                to 10.131.32.30:60001

                # stage-django-2021-other01
                # localhost
                #-
                #==
                header_up +USWGI_SCHEME https
        }
        # charset                 utf-8
        # client_max_body_size    25M
}
-------
www_t4a.conf
===========
www.tracks4africa.co.za {
        redir https://tracks4africa.co.za permanent
}

3. The problem I’m having:

seeing these in the log files lately:
"Unsolicited response received on idle HTTP channel " followed by page content.

I suspect it’s the monitoring services I’m using that checks for the HTTP 200 response and then closes the connection before the data was being transfered from the downstream Django

4. Error messages and/or full log output:

May 6 09:37:41 caddy01 caddy-downloaded[32321]: {“level”:“error”,“ts”:1620293861.1722639,“logger”:“http.log.error.log2”,“msg”:“EOF”,“request”:{“remote_addr”:“66.249.73.143:53685”,“proto”:“HTTP/1.1”,“method”:“GET”,“host”:“tracks4africa.co.za”,“uri”:"/listings/item/w382336/samfya-marines-beach-lodge/",“headers”:{“Accept”:[“text/html,application/xhtml+xml,application/signed-exchange;v=b3,application/xml;q=0.9,/;q=0.8”],“From”:[“googlebot(at)googlebot.com”],“User-Agent”:[“Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.97 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”],“Accept-Encoding”:[“gzip, deflate, br”],“If-Modified-Since”:[“Sun, 02 May 2021 20:45:39 GMT”],“Amp-Cache-Transform”:[“google;v=“1…7"”],“Connection”:[“keep-alive”]},“tls”:{“resumed”:false,“version”:772,“cipher_suite”:4865,“proto”:”",“proto_mutual”:true,“server_name”:“tracks4africa.co.za”}},“duration”:90.77586902,“status”:502,“err_id”:“496phe1ba”,“err_trace”:“reverseproxy.statusError (reverseproxy.go:817)”}
May 6 09:37:43 caddy01 caddy-downloaded[32321]: {“level”:“error”,“ts”:1620293863.9625528,“logger”:“http.log.error.log2”,“msg”:“EOF”,“request”:{“remote_addr”:“62.119.69.92:10400”,“proto”:“HTTP/2.0”,“method”:“GET”,“host”:“tracks4africa.co.za”,“uri”:"/maps/w139076/",“headers”:{“Accept”:[“text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.9”],“Sec-Fetch-Site”:[“same-origin”],“Sec-Fetch-Mode”:[“navigate”],“Referer”:[“https://tracks4africa.co.za/listings/item/w139076/konkiep-lapa-rest-camp-campsite/"],“Accept-Encoding”:["gzip, deflate, br”],“Cookie”:["__cfduid=d1905de7b4975001d437b1505f22376781620293706; _ga=GA1.3.2015295586.1620293707; _gid=GA1.3.223497462.1620293707"],“User-Agent”:[“Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36”],“Sec-Ch-Ua-Mobile”:["?0"],“Upgrade-Insecure-Requests”:[“1”],“Sec-Fetch-User”:["?1"],“Sec-Fetch-Dest”:[“document”],“Accept-Language”:[“sv-SE,sv;q=0.9,en-US;q=0.8,en;q=0.7”],“Sec-Ch-Ua”:["" Not A;Brand";v=“99”, “Chromium”;v=“90”, “Google Chrome”;v=“90"”]},“tls”:{“resumed”:false,“version”:772,“cipher_suite”:4865,“proto”:“h2”,“proto_mutual”:true,“server_name”:“tracks4africa.co.za”}},“duration”:91.380005458,“status”:502,“err_id”:“a3urcgyrj”,“err_trace”:“reverseproxy.statusError (reverseproxy.go:817)”}
May 6 09:39:18 caddy01 caddy-downloaded[32321]: 2021/05/06 09:39:18 Unsolicited response received on idle HTTP channel starting with “\n\n\n\n\n\n<html lang=“en”>\n \n \n <meta charset=“utf-8”>\n <meta http-equiv=“X-UA-Compatible” content=“IE=edge”>\n <meta name=“viewport” content=“width=device-width, initial-scale=1”>\n \n \n \n<link rel=“apple-touch-icon” sizes=“57x57” href=“https://static.tracks4africa.co.za/apple-touch-icon-57x57.png”>\n<link rel=“apple-touch-icon” sizes=“60x60” href=“https://static.tracks4africa.co.za/apple-touch-icon-60x60.png”>\n<link rel=“apple-touch-icon” sizes=“72x72” href=“https://static.tracks4africa.co.za/apple-touch-icon-72x72.png”>\n<link rel=“apple-touch-icon” sizes=“76x76” href=“https://static.tracks4africa.co.za/apple-touch-icon-76x76.png”>\n<link rel=“apple-touch-icon” sizes=“114x114” href=“https://static.tracks4africa.co.za/apple-touch-icon-114x114.png”>\n<link rel=“apple-touch-icon” sizes=“120x120” href=“https://static.tracks4africa.co.za/apple-touch-icon-120x120.png”>\n<link rel=“apple-touch-icon” sizes=“144x144” href=“https://static.tracks4africa.co.za/apple-touch-icon-144x144.png”>\n<link rel=“apple-touch-icon” sizes=“152x152” href=“https://static.tracks4africa.co.za/apple-touch-icon-152x152.png”>\n<link rel=“apple-touch-icon” sizes=“180x180” href=“https://static.tracks4africa.co.za/apple-touch-icon-180x180.png”>\n<link rel=“icon” type=“image/png” href=“https://static.tracks4africa.co.za/favicon-32x32.png” sizes=“32x32”>\n<link rel=“icon” type=“image/png” href=“https://static.tracks4africa.co.za/favicon-194x194.png” sizes=“194x194”>\n<link rel=“icon” type=“image/png” href=“https://static.tracks4africa.co.za/favicon-96x96.png” sizes=“96x96”>\n<link rel=“icon” type=“image/png” href=“https://static.tracks4africa.co.za/android-chrome-192x192.png” sizes=“192x192”>\n<link rel=“icon” type=“image/png” href=“https://static.tracks4africa.co.za/favicon-16x16.png” sizes=“16x16”>\n<link rel=“manifest” href=“https://static.tracks4africa.co.za/manifest.json”>\n<link rel=“shortcut icon” href=“https://static.tracks4africa.co.za/favicon.ico”>\n<meta name=“msapplication-TileColor” content=”#da532c">\n<meta name=“msapplication-TileImage” content=“https://static.tracks4africa.co.za/mstile-144x144.png”>\n<meta name=“msapplication-config” content=“https://static.tracks4africa.co.za/browserconfig.xml”>\n<meta name=“theme-color” content="#ffffff">\n\n \n \n <meta property=“og:title” content=“Tracks4Africa” />\n <meta property=“og:type” content=“company” />\n <meta property=“og:url” content=“https://tracks4africa.co.za/” />\n <meta property=“og:site_name” content=“Tracks4Africa” />\n <meta property=“fb:admins” content=“100000128267084” />\n <meta property=“fb:app_id” content=“219545334764267” />\n <meta property=“og:image” content=“https://static.tracks4africa.co.za/images/t4a-logo.png” />\n <link rel=“img_src” href=“https://static.tracks4africa.co.za/mages/t4a-logo.png” />\n \n \n <meta name=“format-detection” content=“telephone=no” />\n\n Welcome to Tracks4Africa\n\n <link rel=“stylesheet” type=“text/css” \n href=“https://static.tracks4africa.co.za/css/normalize.css” media=“screen, print”>\n <link href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" rel=“stylesheet”>\n <link href=“https://stackpath.bootstrapcdn.com/bootstrap/3.0.3/css/bootstrap-theme.min.css” rel=“stylesheet” media=“screen”>\n \n\n \n \n\n <link rel=“stylesheet” type=“text/css” \n href=“https://static.tracks4africa.co.za/css/style3.css” media=“screen, print”>\n <link rel=“stylesheet” type=“text/css” \n href=“https://static.tracks4africa.co.za/css/page-header-middle.css” media=“screen, print”>\n <link rel=“stylesheet” type=“text/css” \n href=“https://static.tracks4africa.co.za/css/page-content.css” media=“screen, print”>\n <link rel=“stylesheet” type=“text/css” \n "; err=

5. What I already tried:

Googled.

6. Links to relevant resources:

github.com/valeriansaliou/vigil

Vigil causes error messages to show up in Caddy's logs

opened 10:21PM - 19 Dec 18 UTC

closed 08:39PM - 21 Dec 18 UTC

whalehub

I'm currently hosting my sites behind the reverse proxy Caddy and I'm using Vigi…l to poll the status of these sites as HTTP targets. Now, the problem is that whenever Vigil polls a site, the following error message shows up in Caddy's logs: ``` 2018/12/19 23:14:51 Unsolicited response received on idle HTTP channel starting with "0\r\n\r\n"; err=<nil> ``` The error message appears to be harmless in the sense that both Vigil and Caddy work fine in spite of it, but I still think that it's something that should be looked into because something seems to be wrong with the HTTP request that Vigil sends out when it polls a site. I found some information about this error message on [this](https://github.com/golang/go/issues/19895) Golang issue.

francislavoie · May 6, 2021, 5:07pm

That message is coming from the Go stdlib, and it’s an indication that your app backend is responding weirdly to a request. This seems to be an issue with Django+uwsgi, because I’m seeing a similar issue on the Traefik GitHub issues.

system · June 5, 2021, 9:45am

This topic was automatically closed after 30 days. New replies are no longer allowed.