Unwanted let's encrypt challenges in logs

Help
1

1. The problem I’m having:

Hello,

I am using Caddy with NextCloud aio. NextCloud span different docker containers, some are using Caddy.
One of them is demanding automatically ssl certificates and doing a challenge for it. Looking at the logs of the container, I discovered some weird logs :

MobaXterm_l34uJ9mbP2
MobaXterm_l34uJ9mbP22251×449 66.2 KB

I first thought that my server was compromised, but the maintainer of nextcloud AIO
seems to think that it can happens by being triggered from outside, without being compromised - without giving more explanations.

From my understanding, the parameters of which tls challenges are made, and how they are made is using a CaddyFile, so I do not know how this could happens without being compromised. I need to understand what happened to see if the server is - or not compromised, and am looking for scenarios/explanations which could lead to such logs.

I didn’t find any config file in the docker container that would be used by Caddy to produce such results.

2. Error messages and/or full log output:

{"level":"error","ts":1719169925.689722,"logger":"http.acme_client","msg":"challenge failed","identifier":"cf.xijingping.gay","challenge_type":"http-01","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"2607:f8b0:400f:803::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code","instance":"","subproblems":[]}}
{"level":"error","ts":1719169925.6898139,"logger":"http.acme_client","msg":"validating authorization","identifier":"cf.xijingping.gay","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"2607:f8b0:400f:803::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code","instance":"","subproblems":[]},"order":"https://acme-v02.api.letsencrypt.org/acme/order/1682207537/281049282167","attempt":1,"max_attempts":3}
{"level":"error","ts":1719169925.6898699,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"cf.xijingping.gay","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:connection - 2607:f8b0:400f:803::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code"}
{"level":"error","ts":1719169925.689932,"logger":"tls.obtain","msg":"will retry","error":"[cf.xijingping.gay] Obtain: [cf.xijingping.gay] solving challenge: cf.xijingping.gay: [cf.xijingping.gay] authorization failed: HTTP 400 urn:ietf:params:acme:error:connection - 2607:f8b0:400f:803::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code (ca=https://acme-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":1.866532842,"max_duration":2592000}
{"level":"error","ts":1719169987.534045,"logger":"http.acme_client","msg":"challenge failed","identifier":"cf.xijingping.gay","challenge_type":"http-01","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"2607:f8b0:400a:80a::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code","instance":"","subproblems":[]}}
{"level":"error","ts":1719169987.5341198,"logger":"http.acme_client","msg":"validating authorization","identifier":"cf.xijingping.gay","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"2607:f8b0:400a:80a::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code","instance":"","subproblems":[]},"order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/145270514/17382224113","attempt":1,"max_attempts":3}
{"level":"error","ts":1719169987.5342908,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"cf.xijingping.gay","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:connection - 2607:f8b0:400a:80a::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code"}
{"level":"error","ts":1719169987.5343559,"logger":"tls.obtain","msg":"will retry","error":"[cf.xijingping.gay] Obtain: [cf.xijingping.gay] solving challenge: cf.xijingping.gay: [cf.xijingping.gay] authorization failed: HTTP 400 urn:ietf:params:acme:error:connection - 2607:f8b0:400a:80a::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code (ca=https://acme-staging-v02.api.letsencrypt.org/directory)","attempt":2,"retrying_in":120,"elapsed":63.710956695,"max_duration":2592000}
{"level":"error","ts":1719174433.8767147,"logger":"http.acme_client","msg":"challenge failed","identifier":"cf.xijingping.gay","challenge_type":"http-01","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"2607:f8b0:400a:807::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code","instance":"","subproblems":[]}}
{"level":"error","ts":1719174433.876824,"logger":"http.acme_client","msg":"validating authorization","identifier":"cf.xijingping.gay","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"2607:f8b0:400a:807::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code","instance":"","subproblems":[]},"order":"https://acme-v02.api.letsencrypt.org/acme/order/1682207537/281063204897","attempt":1,"max_attempts":3}
{"level":"error","ts":1719174433.8769455,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"cf.xijingping.gay","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:connection - 2607:f8b0:400a:807::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code"}
{"level":"error","ts":1719174433.8770165,"logger":"tls.obtain","msg":"will retry","error":"[cf.xijingping.gay] Obtain: [cf.xijingping.gay] solving challenge: cf.xijingping.gay: [cf.xijingping.gay] authorization failed: HTTP 400 urn:ietf:params:acme:error:connection - 2607:f8b0:400a:807::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code (ca=https://acme-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":1.384294412,"max_duration":2592000}
{"level":"error","ts":1719174495.2034523,"logger":"http.acme_client","msg":"challenge failed","identifier":"cf.xijingping.gay","challenge_type":"http-01","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"2607:f8b0:400a:807::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code","instance":"","subproblems":[]}}
{"level":"error","ts":1719174495.2035277,"logger":"http.acme_client","msg":"validating authorization","identifier":"cf.xijingping.gay","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"2607:f8b0:400a:807::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code","instance":"","subproblems":[]},"order":"https://acme-staging-v02.api.letsencrypt.org/acme/order/145270514/17383111413","attempt":1,"max_attempts":3}
{"level":"error","ts":1719174495.2035854,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"cf.xijingping.gay","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:connection - 2607:f8b0:400a:807::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code"}
{"level":"error","ts":1719174495.2036767,"logger":"tls.obtain","msg":"will retry","error":"[cf.xijingping.gay] Obtain: [cf.xijingping.gay] solving challenge: cf.xijingping.gay: [cf.xijingping.gay] authorization failed: HTTP 400 urn:ietf:params:acme:error:connection - 2607:f8b0:400a:807::200e: Fetching https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be: received disallowed redirect status code (ca=https://acme-staging-v02.api.letsencrypt.org/directory)","attempt":2,"retrying_in":120,"elapsed":62.71093021,"max_duration":2592000}
.

3. Caddy version:

v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A=

4. How I installed and ran Caddy:

a. System environment:

nextcloud/all-in-one:20240404_082330-latest docker image

b. Command:

PASTE OVER THIS, BETWEEN THE ``` LINES.
Please use the preview pane to ensure it looks nice.

c. Service/unit/compose file:

PASTE OVER THIS, BETWEEN THE ``` LINES.
Please use the preview pane to ensure it looks nice.

d. My complete Caddy config:

{
        # auto_https will create redirects for https://{host}:8443 instead of https://{host}
        # https redirects are added manually in the http://:80 block
        auto_https disable_redirects

        storage file_system {
                root /mnt/docker-aio-config/caddy/
        }

        log {
                level ERROR
        }

        servers {
                protocols h1 h2 h2c
        }

        on_demand_tls {
                ask http://localhost:9876/
        }
}

http://:80 {
        redir https://{host}{uri} permanent
}

https://:8443 {
        reverse_proxy localhost:8000

        tls {
                on_demand
                issuer acme {
                        disable_tlsalpn_challenge
                }
        }
}

5. Links to relevant resources:

Link to Nextcloud AIO : GitHub - nextcloud/all-in-one: 📦 The official Nextcloud installation method. Provides easy deployment and maintenance with most features included in this one Nextcloud instance.

2

Is localhost:9876 configured to reject requests for domains you don’t want certificates for?

  • ask will cause Caddy to make an HTTP request to the given URL, asking whether a domain is allowed to have a certificate issued.
    The request has a query string of ?domain= containing the value of the domain name.
    If the endpoint returns a 2xx status code, Caddy will be authorized to obtain a certificate for that name. Any other status code will result in cancelling issuance of the certificate and erroring the TLS handshake.

—https://caddyserver.com/docs/caddyfile/options#on-demand-tls

If the ask endpoint does not properly reject bad domains, anyone across the internet can craft an SNI request for arbitrary hostnames on your server and prompt an ACME challenge from your Caddy instance. This leaves you open to abuse and should be rectified. Upstream ACME providers will have rate limits to mitigate your server abusing theirs, but you may find yourself with cluttered logs and have your renewal attempts rejected later due to said rate limit abuse.

To protect yourself, the listener at localhost:9876 must return non-200 responses for any domain other than the ones you actually want to serve.

3 Likes
3

Hello,

Here is the code returned by http://localhost:9876 :

<?php

$domain = $_GET['domain'] ?? '';

if (strpos($domain, '.') === false) {
    http_response_code(400);
} elseif (strpos($domain, '/') !== false) {
    http_response_code(400);
} elseif (strpos($domain, ':') !== false) {
    http_response_code(400);
} elseif (filter_var($domain, FILTER_VALIDATE_DOMAIN, FILTER_FLAG_HOSTNAME) === false) {
    http_response_code(400);
} elseif (filter_var($domain, FILTER_VALIDATE_IP)) {
    http_response_code(400);
} else {
    // Commented because logging is disabled as otherwise all attempts will be logged which spams the logs
    // error_log($domain . ' was accepted as valid domain.');
    http_response_code(200);
}

It doesn’t seem to check if the domain is actually the one provided by the user and seems vulnerable to external requests for arbitrary hostnames, as you suggested. I will open an issue. Thanks for helping !

1 Like
4

It seems to me like it would be amazingly ideal for Nextcloud to use this endpoint to check its own config.php for the trusted_domains key and allow any requests that match one of the keys.

2 Likes
5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.