Let's encrypt fails when starting Caddy

1. Caddy version

v2.1.1 h1:X9k1+ehZPYYrSqBvf/ocUgdLSRIuiNiMo7CvyGUQKeA=

2. How I run Caddy:

Installed on ubuntu using apt-get

a. System environment:

Ubuntu 18.04.5 LTS

b. Command:

caddy run

c. Service/unit/compose file:

paste full file contents here

d. My complete Caddyfile

https://gowevote.com {
  encode zstd gzip
  root * /home/deploy/apps/rails-app/current/public
  file_server

  @notStatic {
    not file
  }

  reverse_proxy @notStatic unix//home/deploy/apps/rails-app/shared/tmp/sockets/rails-app-puma.sock

  header / {
    Strict-Transport-Security "max-age=31536000"
  }
}

3. The problem I’m having:

When running sudo caddy run let encrypt fails ACME verification

4. Error messages and/or full log output:

deploy@wevoteServer:~/apps/we-vote/current/config$ caddy run
2020/09/11 21:41:22.129 INFO    using adjacent Caddyfile
2020/09/11 21:41:22.131 INFO    admin   admin endpoint started  {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127.0.0.1:2019"]}
2020/09/11 21:41:22.131 INFO    http    server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {"server_name": "srv0", "https_port": 443}
2020/09/11 21:41:22.131 INFO    http    enabling automatic HTTP->HTTPS redirects        {"server_name": "srv0"}
2020/09/11 21:41:22.131 INFO    tls     cleaned up storage units
run: loading initial config: loading new config: http app module: start: tcp: listening on :443: listen tcp :443: bind: permission denied
deploy@wevoteServer:~/apps/we-vote/current/config$ sudo caddy run
2020/09/11 21:41:39.200 INFO    using adjacent Caddyfile
2020/09/11 21:41:39.201 INFO    admin   admin endpoint started  {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127.0.0.1:2019"]}
2020/09/11 21:41:39.201 INFO    http    server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {"server_name": "srv0", "https_port": 443}
2020/09/11 21:41:39.201 INFO    http    enabling automatic HTTP->HTTPS redirects        {"server_name": "srv0"}
2020/09/11 21:41:39.202 INFO    tls     cleaned up storage units
2020/09/11 21:41:39.202 INFO    http    enabling automatic TLS certificate management   {"domains": ["gowevote.com"]}
2020/09/11 21:41:39.202 INFO    autosaved config        {"file": "/home/deploy/.config/caddy/autosave.json"}
2020/09/11 21:41:39.202 INFO    serving initial configuration
2020/09/11 21:41:39 [INFO][gowevote.com] Obtain certificate; acquiring lock...
2020/09/11 21:41:39 [INFO][gowevote.com] Obtain: Lock acquired; proceeding...
2020/09/11 21:41:39 [INFO][cache:0xc00056a960] Started certificate maintenance routine
2020/09/11 21:41:39 [INFO][gowevote.com] Waiting on rate limiter...
2020/09/11 21:41:39 [INFO][gowevote.com] Done waiting
2020/09/11 21:41:39 [INFO] [gowevote.com] acme: Obtaining bundled SAN certificate given a CSR
2020/09/11 21:41:40 [INFO] [gowevote.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/7152439146
2020/09/11 21:41:40 [INFO] [gowevote.com] acme: Could not find solver for: tls-alpn-01
2020/09/11 21:41:40 [INFO] [gowevote.com] acme: use http-01 solver
2020/09/11 21:41:40 [INFO] [gowevote.com] acme: Trying to solve HTTP-01
2020/09/11 21:41:51 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/7152439146
2020/09/11 21:41:51 [INFO] Unable to deactivate the authorization: https://acme-v02.api.letsencrypt.org/acme/authz-v3/7152439146
2020/09/11 21:41:51 [ERROR] error: one or more domains had a problem:
[gowevote.com] acme: error: 400 :: urn:ietf:params:acme:error:connection :: Fetching http://gowevote.com/.well-known/acme-challenge/vUqfiKAUyUc3CQrsAYcs8OQ6dmtySaXI_Uib1oIHVqo: Timeout during connect (likely firewall problem), url: 
 (challenge=http-01 remaining=[tls-alpn-01])
2020/09/11 21:41:53 [INFO] [gowevote.com] acme: Obtaining bundled SAN certificate given a CSR
2020/09/11 21:41:54 [INFO] [gowevote.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/7152441931
2020/09/11 21:41:54 [INFO] [gowevote.com] acme: use tls-alpn-01 solver
2020/09/11 21:41:54 [INFO] [gowevote.com] acme: Trying to solve TLS-ALPN-01
2020/09/11 21:41:55 http: TLS handshake error from 127.0.0.1:36994: EOF
2020/09/11 21:42:09 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/7152441931
2020/09/11 21:42:09 [INFO] Unable to deactivate the authorization: https://acme-v02.api.letsencrypt.org/acme/authz-v3/7152441931
2020/09/11 21:42:09 [ERROR] error: one or more domains had a problem:
[gowevote.com] acme: error: 400 :: urn:ietf:params:acme:error:connection :: Timeout during connect (likely firewall problem), url: 
 (challenge=tls-alpn-01 remaining=[])
2020/09/11 21:42:11 [ERROR] attempt 1: [gowevote.com] Obtain: [gowevote.com] error: one or more domains had a problem:
[gowevote.com] acme: error: 400 :: urn:ietf:params:acme:error:connection :: Timeout during connect (likely firewall problem), url: 
 - retrying in 1m0s (32.211194313s/720h0m0s elapsed)...
2020/09/11 21:43:11 [INFO] [gowevote.com] acme: Obtaining bundled SAN certificate given a CSR
2020/09/11 21:43:11 [INFO] [gowevote.com] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/110175679
2020/09/11 21:43:11 [INFO] [gowevote.com] acme: use tls-alpn-01 solver
2020/09/11 21:43:11 [INFO] [gowevote.com] acme: Trying to solve TLS-ALPN-01
2020/09/11 21:43:11 http: TLS handshake error from 127.0.0.1:37114: EOF
2020/09/11 21:43:29 [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/110175679
2020/09/11 21:43:29 [INFO] Unable to deactivate the authorization: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/110175679
2020/09/11 21:43:29 [ERROR] error: one or more domains had a problem:
[gowevote.com] acme: error: 400 :: urn:ietf:params:acme:error:connection :: Timeout during connect (likely firewall problem), url: 
 (challenge=tls-alpn-01 remaining=[http-01])
2020/09/11 21:43:31 [INFO] [gowevote.com] acme: Obtaining bundled SAN certificate given a CSR
2020/09/11 21:43:31 [INFO] [gowevote.com] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/110175970
2020/09/11 21:43:31 [INFO] [gowevote.com] acme: Could not find solver for: tls-alpn-01
2020/09/11 21:43:31 [INFO] [gowevote.com] acme: use http-01 solver
2020/09/11 21:43:31 [INFO] [gowevote.com] acme: Trying to solve HTTP-01
2020/09/11 21:43:47 [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/110175970
2020/09/11 21:43:47 [INFO] Unable to deactivate the authorization: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/110175970
2020/09/11 21:43:47 [ERROR] error: one or more domains had a problem:
[gowevote.com] acme: error: 400 :: urn:ietf:params:acme:error:connection :: Fetching http://gowevote.com/.well-known/acme-challenge/88pcnDUIig4CzTJc9DyIvNEmx5oTB19neHEkUqaMXKg: Timeout during connect (likely firewall problem), url: 
 (challenge=http-01 remaining=[])
2020/09/11 21:43:49 [ERROR] attempt 2: [gowevote.com] Obtain: [gowevote.com] error: one or more domains had a problem:
[gowevote.com] acme: error: 400 :: urn:ietf:params:acme:error:connection :: Fetching http://gowevote.com/.well-known/acme-challenge/88pcnDUIig4CzTJc9DyIvNEmx5oTB19neHEkUqaMXKg: Timeout during connect (likely firewall problem), url: 
 - retrying in 2m0s (2m9.868615766s/720h0m0s elapsed)...
2020/09/11 21:45:49 [INFO] [gowevote.com] acme: Obtaining bundled SAN certificate given a CSR
2020/09/11 21:45:49 [INFO] [gowevote.com] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/110177264
2020/09/11 21:45:49 [INFO] [gowevote.com] acme: Could not find solver for: tls-alpn-01
2020/09/11 21:45:49 [INFO] [gowevote.com] acme: use http-01 solver
2020/09/11 21:45:49 [INFO] [gowevote.com] acme: Trying to solve HTTP-01
2020/09/11 21:46:06 [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/110177264
2020/09/11 21:46:06 [INFO] Unable to deactivate the authorization: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/110177264
2020/09/11 21:46:06 [ERROR] error: one or more domains had a problem:
[gowevote.com] acme: error: 400 :: urn:ietf:params:acme:error:connection :: Fetching http://gowevote.com/.well-known/acme-challenge/7yW1AjBwZxacBU7uNK7WHMqBMMwFRMZFHNF7UpawBwg: Timeout during connect (likely firewall problem), url: 
 (challenge=http-01 remaining=[tls-alpn-01])
2020/09/11 21:46:08 [INFO] [gowevote.com] acme: Obtaining bundled SAN certificate given a CSR
2020/09/11 21:46:09 [INFO] [gowevote.com] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/110177381
2020/09/11 21:46:09 [INFO] [gowevote.com] acme: use tls-alpn-01 solver
2020/09/11 21:46:09 [INFO] [gowevote.com] acme: Trying to solve TLS-ALPN-01
2020/09/11 21:46:09 http: TLS handshake error from 127.0.0.1:37400: EOF
2020/09/11 21:46:26 [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/110177381
2020/09/11 21:46:26 [INFO] Unable to deactivate the authorization: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/110177381
2020/09/11 21:46:26 [ERROR] error: one or more domains had a problem:
[gowevote.com] acme: error: 400 :: urn:ietf:params:acme:error:connection :: Timeout during connect (likely firewall problem), url: 
 (challenge=tls-alpn-01 remaining=[])
2020/09/11 21:46:28 [ERROR] attempt 3: [gowevote.com] Obtain: [gowevote.com] error: one or more domains had a problem:
[gowevote.com] acme: error: 400 :: urn:ietf:params:acme:error:connection :: Timeout during connect (likely firewall problem), url: 
 - retrying in 2m0s (4m49.456988501s/720h0m0s elapsed)...

5. What I already tried:

When checking on let encrypt here it says likely firewall issue.

I’ve configured the firewall on Azure like this.

I am also facing issue with Caddy json file…


2020/09/12 21:36:10 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/7170101006
2020/09/12 21:36:10 [INFO] Unable to deactivate the authorization: https://acme-v02.api.letsencrypt.org/acme/authz-v3/7170101006
2020/09/12 21:36:10 [ERROR] error: one or more domains had a problem:
[suman-dev.herokuapp.com] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Invalid response from http://suman-dev.herokuapp.com/.well-known/acme-challenge/f_I1RCa0gluCZTaUvzqEnXmXGsdojG3LuAhig6sLdLg [52.201.119.187]: 404, url:
 (challenge=http-01 remaining=[tls-alpn-01])
2020/09/12 21:36:12 [INFO] [suman-dev.herokuapp.com] acme: Obtaining bundled SAN certificate given a CSR
2020/09/12 21:36:13 http: TLS handshake error from 127.0.0.1:56416: no certificate available for 'suman-dev.herokuapp.com'
2020/09/12 21:36:13 http: TLS handshake error from 127.0.0.1:56417: no certificate available for 'suman-dev.herokuapp.com'
2020/09/12 21:36:14 [INFO] [suman-dev.herokuapp.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/7170105937
2020/09/12 21:36:14 [INFO] [suman-dev.herokuapp.com] acme: use tls-alpn-01 solver
2020/09/12 21:36:14 [INFO] [suman-dev.herokuapp.com] acme: Trying to solve TLS-ALPN-01
2020/09/12 21:36:14 http: TLS handshake error from 127.0.0.1:56419: EOF
2020/09/12 21:36:15 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/7170105937
2020/09/12 21:36:16 [INFO] Unable to deactivate the authorization: https://acme-v02.api.letsencrypt.org/acme/authz-v3/7170105937
2020/09/12 21:36:16 [ERROR] error: one or more domains had a problem:
[suman-dev.herokuapp.com] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Cannot negotiate ALPN protocol "acme-tls/1" for tls-alpn-01 challenge, url:
 (challenge=tls-alpn-01 remaining=[])
2020/09/12 21:36:18 [ERROR] attempt 1: [suman-dev.herokuapp.com] Obtain: [suman-dev.herokuapp.com] error: one or more domains had a problem:
[suman-dev.herokuapp.com] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Cannot negotiate ALPN protocol "acme-tls/1" for tls-alpn-01 challenge, url:
 - retrying in 1m0s (20.857276271s/720h0m0s elapsed)...
2020/09/12 21:36:23 http: TLS handshake error from 127.0.0.1:56456: no certificate availabl````