HAProxy woes - auto certs & https redirection issues

Hi!
I’m running my webserver on a server at my house which means no static IP and ISP making it a little bit of a headache. To get around this, I’ve got two cloud servers with static IPs that connect to my main server via a VPN. My DNS settings point to the cloud servers that then proxy everything through the VPN and to my home server.

My biggest issues are with automatic certificates (but so far the renewals seem to have no problems) not liking this setup and I can’t seem to get HTTP to HTTPS redirection to function, either. I can access sites over port 80 and over port 443 but on sites that expect to be accessed over HTTPS, I get an error instead of a redirection.

Caddyfile:

bitwarden.example.com {
        proxyprotocol 10.10.0.0/16
        proxy /notifications/hub/negotiate 172.16.1.2:80 {
                transparent
        }
        proxy /notifications/hub 172.16.1.2:3012 {
                websocket
        }
        proxy / 172.16.1.2:80 {
                transparent
        }
}

calendar.example.com {
        proxyprotocol 10.10.0.0/16
        proxy / http://172.16.0.5:5232/ {
                transparent
                header_upstream X-Script-Name /
        }
}

cloud.example.com {
        proxyprotocol 10.10.0.0/16

        proxy / http://172.16.0.15:2015 {
                transparent
        }
}
http://, https:// {
        tls cert key
        redir https://example.com
}

HAProxy:

global
        log /dev/log    local0
        log /dev/log    local1 notice
        chroot /var/lib/haproxy
        stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
        stats timeout 30s
        user haproxy
        group haproxy
        daemon
        ca-base /etc/ssl/certs
        crt-base /etc/ssl/private
        ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+$
        ssl-default-bind-options no-sslv3

defaults
        log     global
        mode    tcp
        option  tcplog
        option  dontlognull
        timeout connect 5000
        timeout client  50000
        timeout server  50000
        errorfile 400 /etc/haproxy/errors/400.http
        errorfile 403 /etc/haproxy/errors/403.http
        errorfile 408 /etc/haproxy/errors/408.http
        errorfile 500 /etc/haproxy/errors/500.http
        errorfile 502 /etc/haproxy/errors/502.http
        errorfile 503 /etc/haproxy/errors/503.http
        errorfile 504 /etc/haproxy/errors/504.http

frontend http
        bind *:80
        bind :::80
        mode tcp
        default_backend http_backend
frontend https
        bind *:443
        bind :::443
        mode tcp
        default_backend https_backend

backend http_backend
        mode tcp
        stick store-request src
        stick-table type ip size 200k expire 30m
        server http_server 10.10.1.2:80 send-proxy-v2
backend https_backend
        mode tcp
        stick store-request src
        stick-table type ip size 200k expire 30m
        server https_server 10.10.1.2:443 send-proxy-v2

When I browse to a site over plain HTTP I get:

400 Bad Request

And with this setup if I try and add a new subdomain and wait for it to autofetch certificates:

Oct 01 10:07:20 atitlan caddy[40124]: 2019/10/01 10:07:20 [ERROR][surprise.example.com]failed to obtain certificate: acme: Error -> One or more domains had a problem:
Oct 01 10:07:20 atitlan caddy[40124]: [surprise.example.com] acme: error: 400 :: urn:ietf:params:acme:error:dns :: No valid IP addresses found for surprise.example.com, url:
Oct 01 10:07:20 atitlan caddy[40124]:  (attempt 1/3; challenge=http-01)
Oct 01 10:07:21 atitlan caddy[40124]: 2019/10/01 10:07:21 [INFO] [surprise.example.com] acme: Obtaining bundled SAN certificate
Oct 01 10:07:21 atitlan caddy[40124]: 2019/10/01 10:07:21 [INFO] [surprise.example.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/587925984
Oct 01 10:07:21 atitlan caddy[40124]: 2019/10/01 10:07:21 [INFO] [surprise.example.com] acme: Could not find solver for: tls-alpn-01
Oct 01 10:07:21 atitlan caddy[40124]: 2019/10/01 10:07:21 [INFO] [surprise.example.com] acme: use http-01 solver
Oct 01 10:07:21 atitlan caddy[40124]: 2019/10/01 10:07:21 [INFO] [surprise.example.com] acme: Trying to solve HTTP-01
Oct 01 10:07:22 atitlan caddy[40124]: 2019/10/01 10:07:22 [INFO] Unable to deactivated authorizations: https://acme-v02.api.letsencrypt.org/acme/authz-v3/587925984
Oct 01 10:07:22 atitlan caddy[40124]: 2019/10/01 10:07:22 [ERROR][surprise.example.com] failed to obtain certificate: acme: Error -> One or more domains had a problem:
Oct 01 10:07:22 atitlan caddy[40124]: [surprise.example.com] acme: error: 400 :: urn:ietf:params:acme:error:dns :: No valid IP addresses found for surprise.example.com, url:
Oct 01 10:07:22 atitlan caddy[40124]:  (attempt 2/3; challenge=http-01)
Oct 01 10:07:23 atitlan caddy[40124]: 2019/10/01 10:07:23 [INFO] [surprise.example.com] acme: Obtaining bundled SAN certificate
Oct 01 10:07:23 atitlan caddy[40124]: 2019/10/01 10:07:23 [INFO] [surprise.example.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/587926452
Oct 01 10:07:23 atitlan caddy[40124]: 2019/10/01 10:07:23 [INFO] [surprise.example.com] acme: Could not find solver for: tls-alpn-01
Oct 01 10:07:23 atitlan caddy[40124]: 2019/10/01 10:07:23 [INFO] [surprise.example.com] acme: use http-01 solver
Oct 01 10:07:23 atitlan caddy[40124]: 2019/10/01 10:07:23 [INFO] [surprise.example.com] acme: Trying to solve HTTP-01
Oct 01 10:07:23 atitlan caddy[40124]: 2019/10/01 10:07:23 [INFO] Unable to deactivated authorizations: https://acme-v02.api.letsencrypt.org/acme/authz-v3/587926452
Oct 01 10:07:23 atitlan caddy[40124]: 2019/10/01 10:07:23 [ERROR][surprise.example.com] failed to obtain certificate: acme: Error -> One or more domains had a problem:
Oct 01 10:07:23 atitlan caddy[40124]: [surprise.example.com] acme: error: 400 :: urn:ietf:params:acme:error:dns :: No valid IP addresses found for surprise.example.com, url:
Oct 01 10:07:23 atitlan caddy[40124]:  (attempt 3/3; challenge=http-01)
Oct 01 10:07:24 atitlan caddy[40124]: 2019/10/01 10:07:24 [INFO] [surprise.example.com] acme: Obtaining bundled SAN certificate
Oct 01 10:07:25 atitlan caddy[40124]: 2019/10/01 10:07:25 [INFO] [surprise.example.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/587926880
Oct 01 10:07:25 atitlan caddy[40124]: 2019/10/01 10:07:25 [INFO] [surprise.example.com] acme: use tls-alpn-01 solver
Oct 01 10:07:25 atitlan caddy[40124]: 2019/10/01 10:07:25 [INFO] [surprise.example.com] acme: Trying to solve TLS-ALPN-01
Oct 01 10:07:26 atitlan caddy[40124]: 2019/10/01 10:07:26 [INFO] Unable to deactivated authorizations: https://acme-v02.api.letsencrypt.org/acme/authz-v3/587926880
Oct 01 10:07:26 atitlan caddy[40124]: 2019/10/01 10:07:26 [ERROR][surprise.example.com] failed to obtain certificate: acme: Error -> One or more domains had a problem:
Oct 01 10:07:26 atitlan caddy[40124]: [surprise.example.com] acme: error: 400 :: urn:ietf:params:acme:error:dns :: No valid IP addresses found for surprise.example.com, url:
Oct 01 10:07:26 atitlan caddy[40124]:  (attempt 1/3; challenge=tls-alpn-01)
Oct 01 10:07:27 atitlan caddy[40124]: 2019/10/01 10:07:27 [INFO] [surprise.example.com] acme: Obtaining bundled SAN certificate
Oct 01 10:07:27 atitlan caddy[40124]: 2019/10/01 10:07:27 [INFO] [surprise.example.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/587927319
Oct 01 10:07:27 atitlan caddy[40124]: 2019/10/01 10:07:27 [INFO] [surprise.example.com] acme: use tls-alpn-01 solver
Oct 01 10:07:27 atitlan caddy[40124]: 2019/10/01 10:07:27 [INFO] [surprise.example.com] acme: Trying to solve TLS-ALPN-01
Oct 01 10:07:28 atitlan caddy[40124]: 2019/10/01 10:07:28 [INFO] Unable to deactivated authorizations: https://acme-v02.api.letsencrypt.org/acme/authz-v3/587927319
Oct 01 10:07:28 atitlan caddy[40124]: 2019/10/01 10:07:28 [ERROR][surprise.example.com] failed to obtain certificate: acme: Error -> One or more domains had a problem:
Oct 01 10:07:28 atitlan caddy[40124]: [surprise.example.com] acme: error: 400 :: urn:ietf:params:acme:error:dns :: No valid IP addresses found for surprise.example.com, url:
Oct 01 10:07:28 atitlan caddy[40124]:  (attempt 2/3; challenge=tls-alpn-01)
Oct 01 10:07:28 atitlan caddy[40124]: 2019/10/01 10:07:28 [INFO] SIGQUIT: Quitting process immediately

Let me know if I can provide anything else to help out or if there is a friendlier solution for proxying from my proxy/frontend to my server at home. If it seems like a HAProxy headache, I don’t mind using something else.

Hi @sirrkitt, welcome to the Caddy community.

Who’s issuing this status 400? Caddy or HAProxy?

As for the error LetsEncrypt is giving you:

Indicates that you don’t have an A record for this subdomain. Make sure that you’ve got DNS records for this domain publicly available. While your setup might cause problems with solving the challenge until we get to the bottom of that, this particular error is separate and needs to be addressed as well.