The key authorization file from the server did not match this challenge

my setup: safe.arix.com points to a server running apache, which send all traffic for that name on ports 80 and 443 to my laptop. here are that apache virtual host declarations:

<VirtualHost *:80>
        ServerName safe.arix.com
        ProxyPass / http://192.168.1.67:80/
        ProxyPassReverse / http://192.168.1.67:80/
</VirtualHost>
<VirtualHost *:443>
        ServerName safe.arix.com
        ProxyPass / http://192.168.1.67:443/
        ProxyPassReverse / http://192.168.1.67:443/
</VirtualHost>

where my laptop is on .67 – when I run caddy I get the error below:

$ sudo ./caddy -conf $PWD/Caddyfile -agree -email test@arix.com -log stdout
Password:
Activating privacy features...2017/03/09 20:48:18 [INFO][safe.arix.com] acme: Obtaining bundled SAN certificate
2017/03/09 20:48:18 [INFO][safe.arix.com] acme: Trying to solve TLS-SNI-01
2017/03/09 20:48:24 [safe.arix.com] failed to get certificate: acme: Error 400 - urn:acme:error:connection - Failed to connect to 107.198.136.173:443 for TLS-SNI-01 challenge
Error Detail:
	Validation for safe.arix.com:443
	Resolved to:
		107.198.136.173
	Used: 107.198.136.173

the caddy file contains:

safe.arix.com {
	proxy / localhost:8002 {
		transparent
	}
}

and I have a small listener on port 8002:

$ telnet localhost 8002
Trying ::1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.1
HOST: safe.arix.com

HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: text/html; charset=utf-8
Content-Length: 8
ETag: W/"8-nkAJZlx2jOXdTDuvHL7ID4JMl6o"
Date: Fri, 10 Mar 2017 04:44:50 GMT
Connection: keep-alive

running

^C
Connection closed by foreign host.

so what am I missing?

On launch, Caddy is attempting to requisition a certificate for safe.arix.com from LetsEncrypt.

This line:

2017/03/09 20:48:24 [safe.arix.com] failed to get certificate: acme: Error 400 - urn:acme:error:connection - Failed to connect to 107.198.136.173:443 for TLS-SNI-01 challenge

Indicates that the ACME server wasn’t able to make a connection to your server on port :443. Double check that your server is reachable at 107.198.136.173 and listening on :443. You could alternatively use DNS validation to bypass this check.

Caddy will exit on launch unless you resolve this issue or turn off Automatic HTTPS, there’s likely nothing wrong with the service running at :8002.

2 Likes

thank you. you were correct. I had many hurdles to overcome before I managed to set it up properly. at this point the error is different:

Activating privacy features...2017/03/09 22:35:11 [INFO][safe.arix.com] acme: Obtaining bundled SAN certificate
2017/03/09 22:35:11 [INFO][safe.arix.com] acme: Could not find solver for: dns-01
2017/03/09 22:35:11 [INFO][safe.arix.com] acme: Trying to solve HTTP-01
2017/03/09 22:35:11 [INFO] Received request for domain 192.168.1.67 with method GET
2017/03/09 22:35:12 [safe.arix.com] failed to get certificate: acme: Error 403 - urn:acme:error:unauthorized - The key authorization file from the server did not match this challenge [3J0FXpf2cyKnbUPqzRF0m2ys0Dut0q1__79PilObmPQ.rbQdJJTMuirfTRwAFE0eaGsLdQZvMcxeHbHsD0AK0RQ] != [TEST]
Error Detail:
	Validation for safe.arix.com:80
	Resolved to:
		107.198.136.173
	Used: 107.198.136.173

what next?

urn:acme:error:unauthorized - The key authorization file from the server did not match this challenge [3J0FXpf2cyKnbUPqzRF0m2ys0Dut0q1__79PilObmPQ.rbQdJJTMuirfTRwAFE0eaGsLdQZvMcxeHbHsD0AK0RQ] != [TEST]

Indicates to me that the ACME server successfully connected to 107.198.136.173:80, but the challenge response was incorrect, so the certificate request was denied.

A quick curl -I 107.198.136.173 returns:

HTTP/1.1 200 OK
Date: Fri, 10 Mar 2017 07:56:37 GMT
Server: Apache
X-Powered-By: PHP/5.6.23
Content-Type: text/html; charset=UTF-8

Indicating that Caddy is not listening on that IP address and port - an Apache server is. Your DNS may be pointing at the wrong server, or your server is misconfigured such that the wrong webserver is listening on port 80.

To be unambiguous: for Caddy to request a certificate for any given domain such as example.com, a public DNS lookup must resolve example.com to an accessible IP address, and requests to that IP address on port :80 and :443 must be answered by the instance of Caddy that initiated the request.

3 Likes

getting back to this…

it may be, then, that I just can’t test with my existing environment. you’re right that if you query that IP address on port 80 you’ll get my apache server, but if you query it by the right name (both www.arix.com and safe.arix.com point to the same IP address):

curl -I safe.arix.com

you’ll get:

HTTP/1.1 503 Service Unavailable
Date: Fri, 10 Mar 2017 18:29:34 GMT
Server: Apache
Connection: close
Content-Type: text/html; charset=iso-8859-1

the request gets routed by Apache to my laptop where I would have Caddy running (if I didn’t have these issues) and it would proxy the request to port 8002, where I have a listener running.

my Caddyfile asks that traffic be proxied for the name safe.arix.com but before that happens Caddy has to come up and I’m guessing that means I have to have a public IP dedicated to the host running Caddy and thus I can’t test the way I’m testing.

am I correct in my deductions?

in re-reading your previous statement I see that “requests to that IP address” would mean I’m right. I can’t test by re-routing by name. I have to set up a separate IP address to do the testing.

thank you for all the help. it’s a trying path to get this to work

ok, I’ve switched environments and I will continue the discussion here. this time I’m able to run Caddy. as before I have a listener (on port 8000 this time) and a simple proxy like this:

graph.speakr.com {
  proxy / localhost:8000 {
    transparent
  }
}

however when I make a request, I get a curious failure:

$ printf "GET / HTTP/1.0\n\n" |nc graph.speakr.com 80
HTTP/1.0 404 Not Found
Content-Type: text/plain; charset=utf-8
Server: Caddy
X-Content-Type-Options: nosniff
Date: Tue, 14 Mar 2017 19:57:56 GMT
Content-Length: 19

No such site at :80

which brings me to my lack of understanding of Caddy. by default, it answers on port 2015, I read, unless it qualifies for encryption, in which case it listens on 443. so that means it doesn’t listen on port 80 at all, unless told to. I thought that might explain my failure but changing the server name at the top of the Caddyfile to http://graph.speakr.com didn’t fix the problem, nor does graph.speakr.com:80

so now what am I doing wrong?

p.s. in googling, I found Matt Holt’s comment “the address in the Caddyfile needs to match that in the URL” so thinking that the actual url might include the trailing / (since, after all, I’m GETting /), I tried graph.speakr.com/ - but that didn’t work either

p.p.s. here’s how I’m running Caddy:

sudo ./caddy -conf ../Caddyfile -agree -email x@x.com -log stdout

and the output I see from it:

Activating privacy features… done.
https://graph.speakr.com/
2017/03/14 20:04:37 https://graph.speakr.com/
http://graph.speakr.com
2017/03/14 20:04:37 http://graph.speakr.com
WARNING: File descriptor limit 1024 is too low for production servers. At least 8192 is recommended. Fix with “ulimit -n 8192”.
2017/03/14 20:04:42 [INFO] - No such site at :80 (Remote: 107.198.136.173, Referer: )

from which I gather that asking for a document (/) from http://graph.speakr.com should work

I also tried testing the 443:

$ openssl s_client -connect graph.speakr.com:443

CONNECTED(00000003)
59291:error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version:/BuildRoot/Library/Caches/com.apple.xbs/Sources/OpenSSL098/OpenSSL098-59.60.1/src/ssl/s23_clnt.c:593:

$ openssl s_client -connect graph.speakr.com:443 -ssl3

CONNECTED(00000003)
59316:error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number:/BuildRoot/Library/Caches/com.apple.xbs/Sources/OpenSSL098/OpenSSL098-59.60.1/src/ssl/s3_pkt.c:300:

$ openssl s_client -connect graph.speakr.com:443 -tls1

CONNECTED(00000003)
59366:error:1409442E:SSL routines:SSL3_READ_BYTES:tlsv1 alert protocol version:/BuildRoot/Library/Caches/com.apple.xbs/Sources/OpenSSL098/OpenSSL098-59.60.1/src/ssl/s3_pkt.c:1145:SSL alert number 70
59366:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:/BuildRoot/Library/Caches/com.apple.xbs/Sources/OpenSSL098/OpenSSL098-59.60.1/src/ssl/s3_pkt.c:566:

on the server end I see:

2017/03/14 20:25:05 http: TLS handshake error from 107.198.136.173:55223: tls: unsupported SSLv2 handshake received
2017/03/14 20:25:36 http: TLS handshake error from 107.198.136.173:55297: tls: client offered an unsupported, maximum protocol version of 300
2017/03/14 20:26:35 http: TLS handshake error from 107.198.136.173:55448: tls: client offered an unsupported, maximum protocol version of 301

and I don’t get a prompt so I can’t actually make a request

ok, some more progress. IF I define the Caddyfile like this:

graph.speakr.com:80 {
  proxy / localhost:8000 {
    transparent
  }
}

AND I make the request with HTTP 1.1

$ printf "GET / HTTP/1.1\nHOST: graph.speakr.com\n\n" |nc graph.speakr.com 80

HTTP/1.1 200 OK
Content-Length: 8
Content-Type: text/html; charset=utf-8
Date: Tue, 14 Mar 2017 20:42:51 GMT
Etag: W/“8-nkAJZlx2jOXdTDuvHL7ID4JMl6o”
Server: Caddy
X-Powered-By: Express

– reached listener –

it works! but, of course, now my attempts to connect on 443 fail:

$ openssl s_client -connect graph.speakr.com:443

connect: Connection refused
connect:errno=61

so… why can’t I just declare graph.speakr.com (as I’ve seen done everywhere else) instead of having to declare graph.speakr.com:80, graph.speakr.com:443?

incidentally, I’m running Caddy v0.9.5 on Ubuntu and the client (OpenSSL 0.9.8zh 14 Jan 2016) is OSX 10.11.6 (El Capitan)

some more progress. after reading about issues with protocol negotiation using certain clients it occurred to me to upgrade my openssl library. I use brew on OSX and discovered that /usr/bin/openssl is left unaffected by brew upgrades. the correct location is /usr/local/Cellar/openssl/1.0.2k/bin/openssl and using the new version I get a proper connection:

$ printf "GET / HTTP/1.1\nHOST: graph.speakr.com\n\n" |/usr/local/Cellar/openssl/1.0.2k/bin/openssl s_client -connect graph.speakr.com:443

CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let’s Encrypt, CN = Let’s Encrypt Authority X3
verify return:1
depth=0 CN = graph.speakr.com
verify return:1

Certificate chain
0 s:/CN=graph.speakr.com
i:/C=US/O=Let’s Encrypt/CN=Let’s Encrypt Authority X3
1 s:/C=US/O=Let’s Encrypt/CN=Let’s Encrypt Authority X3
i:/O=Digital Signature Trust Co./CN=DST Root CA X3

Server certificate
-----BEGIN CERTIFICATE-----
MIIFBDCCA+ygAwIBAgISAzB8/Hyxo/vEo+GJ/XA3PHvcMA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xNzAzMTAwMTAwMDBaFw0x
NzA2MDgwMTAwMDBaMBsxGTAXBgNVBAMTEGdyYXBoLnNwZWFrci5jb20wggEiMA0G
CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCZM0z+r3dS/Rr+rEMOo7KcIG1dlXFX
MZ/jZvXQkFE8vlHRSktx03qyYQ0uUCJe5K9aYTBp40/QQ6AtSRgElS8X8p2GhZA3
INif55fevDxUxvdD//ioHZF0ZfPQX+l/q3XmfAUfPL/Sh52J9S3w17nnkVmfe3af
IS2gkqzqVwgJVYMVhKN+LFxxxspEOmszx6+YlYCsbIIh7vL9o/vNHFE9m8yylCmf
crWUeUPePWsIHkPpNdFaM/of3yFmdaF2YaMfHs+JA8HYSvRflzeiBYhzOZgnfQyl
mPXf0aShGjfL2BdqSRSKeuVRc89iDjogoOXT1+bpz7AXHYz1UMq00c+fAgMBAAGj
ggIRMIICDTAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsG
AQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFNj47RfL/zUvfcuQp/uFFDsT
HJHbMB8GA1UdIwQYMBaAFKhKamMEfd265tE5t6ZFZe/zqOyhMHAGCCsGAQUFBwEB
BGQwYjAvBggrBgEFBQcwAYYjaHR0cDovL29jc3AuaW50LXgzLmxldHNlbmNyeXB0
Lm9yZy8wLwYIKwYBBQUHMAKGI2h0dHA6Ly9jZXJ0LmludC14My5sZXRzZW5jcnlw
dC5vcmcvMBsGA1UdEQQUMBKCEGdyYXBoLnNwZWFrci5jb20wgf4GA1UdIASB9jCB
8zAIBgZngQwBAgEwgeYGCysGAQQBgt8TAQEBMIHWMCYGCCsGAQUFBwIBFhpodHRw
Oi8vY3BzLmxldHNlbmNyeXB0Lm9yZzCBqwYIKwYBBQUHAgIwgZ4MgZtUaGlzIENl
cnRpZmljYXRlIG1heSBvbmx5IGJlIHJlbGllZCB1cG9uIGJ5IFJlbHlpbmcgUGFy
dGllcyBhbmQgb25seSBpbiBhY2NvcmRhbmNlIHdpdGggdGhlIENlcnRpZmljYXRl
IFBvbGljeSBmb3VuZCBhdCBodHRwczovL2xldHNlbmNyeXB0Lm9yZy9yZXBvc2l0
b3J5LzANBgkqhkiG9w0BAQsFAAOCAQEAFYaLV9uDLqtHIRTN978xpuGs212rs23E
1qcnUdVBogHJW9KWRn/7YxbY4NDyCFik56mztkBfVych3UkedUMkrce9qQ09vzkU
dVxfcg13m11oGGHvY9et5EZrQ6pxp3iSOFxXF4mda54yUrj2YOJ4wIFAvslXz6o9
2wV8VB40Xc1+VplHrFfKlWPUcybgZ5jX56z8RwQxgYNDhV+Q0f1DHpHUnu3eSU/d
rZakASNzD3eRP0f0/975ipHKrE6JoiyK+4pRkqJeB4ioa3dh0MGsq9gP9wF5Z/CX
/gRXoh0Yi0uLfrUrSawEvGzcteaGcQ1Q0Po9ZlNm/4K+6bxVHQB+dw==
-----END CERTIFICATE-----
subject=/CN=graph.speakr.com
issuer=/C=US/O=Let’s Encrypt/CN=Let’s Encrypt Authority X3

No client certificate CA names sent
Peer signing digest: SHA384
Server Temp Key: ECDH, P-256, 256 bits

SSL handshake has read 3071 bytes and written 434 bytes

New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: 6566ECB6BDFA30057F94D6807F13B8CF2520BDC3EE291E0F6AEC7F8B6FA3F1E0
Session-ID-ctx:
Master-Key: 8D9E538C2432F1DF2B334C23603AA99EB31A2365115EF33DCFAC557CDA73CD693C8B118CD161D081FCB833B4A1E1D47B
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket:
0000 - e4 eb 8e d8 ee 07 8b c8-ef 77 18 cc 80 3f 7c 9e …w…?|.
0010 - 6a 9d e5 7d 16 64 14 c7-42 ee fb 65 48 71 9b 31 j…}.d…B…eHq.1
0020 - d2 89 87 56 2f 43 42 42-fb 50 75 ba 14 bf 4b 67 …V/CBB.Pu…Kg
0030 - 4e 0e 3a 9e 72 fc 86 d5-3d 61 ce 74 45 de 0d ee N.:.r…=a.tE…
0040 - 1a e8 4e 1f 9c 74 c9 1d-fc ae 6c df 89 85 04 e7 …N…t…l…
0050 - 17 1a ae 1d 7d fb 86 70-b4 4f d1 09 bc cf 1e 8a …}…p.O…
0060 - 05 54 63 fc 6e 7c a0 71-05 0e 62 d5 7a a6 10 9e .Tc.n|.q…b.z…
0070 - 18 41 89 7e 63 26 12 a3- .A.~c&…

Start Time: 1489525903
Timeout   : 300 (sec)
Verify return code: 0 (ok)

DONE

however, I thought piping a command to openssl would actually send the request. nowhere in the output do I see the document I requested, but if I manually type it in, I get routed properly (so it works).

so my question at this point is: why do I have to declare both ports (80 & 443) for this to work instead of just declaring the host name as I was doing originally?

if I (making the calls properly with HTTP 1.1) call it when it’s just defined with the name, I get:

HTTP/1.1 301 Moved Permanently
Connection: close
Location: https://graph.speakr.com/
Server: Caddy
Date: Tue, 14 Mar 2017 21:20:22 GMT
Content-Length: 60
Content-Type: text/html; charset=utf-8

Moved Permanently.

Wow, you’ve been busy! I’ll try answer what I can.

That 503 from Apache implies to me that your upstream is misconfigured in Apache or it is otherwise unable to communicate with Caddy (at all). If Apache could connect to Caddy, but Caddy was playing up, Apache would have given you a status 502 instead, I believe.

Not to put too fine a point on it, no, that’s not strictly required. All that needs to happen is that a request to example.com's IP address, for example.com, needs to be responded to by Caddy. You can absolutely put a proxy in between, as long as the proxy faithfully forwards the request to Caddy and returns Caddy’s response to the client.

Again, not necessarily - see previous. As long as the proxy is set up correctly and faithfully (and transparently edit: actually, transparency is not required either; all LetsEncrypt actually cares about is that the well-known token they request matches what they told Caddy to put there) proxying, it should work.

With this command, you’re not actually indicating the server name (graph.speakr.com) to the server, you’re just connecting to the server (by IP address as nc simply resolves the FQDN) on port 80 and sending GET / HTTP/1.0. Caddy doesn’t have a matching server name, as you have not configured a default server ("" != graph.speakr.com - you would need to set up a blanket vhost :80 to catch these kinds of requests). Instead of printf | nc, try curl -I https://graph.speakr.com, which will correctly indicate the server name to Caddy.

Well and good, but you’re testing a few times using unsupported protocols. The tls directive can be used to extend the protocol support range. Caddy will close the connection on unsupported protocols.

The effective change here is that you’re indicating the server name; you could revert your Caddyfile change and make this request again, and the expected behaviour would be a 301 in response (telling you to try HTTPS on port 443 instead), with A HTTPS request returning status 200.

Again, I recommend relying on curl -I instead of piping printf to nc.

You absolutely can. Specifying :80 will disable Automatic HTTPS and only serve HTTP, specifying :443 will disable HTTP (but leave certificate management running), and specifying both separately is a common method of getting fine grained control over the redirection process where required (by default it simply 301’s to HTTPS).

This is expected behaviour, which is good! Automatic HTTPS with just the name declared will set up the listeners on :80 (where it will issue 301 redirects to HTTPS) as well as :443 (where it will serve the actual site, as per your server directives).

2 Likes

To clarify some expected Caddy behaviours for different Caddyfile formats:

example.com {
    root /var/www/html
}

Request: http://example.com (on port 80)
Expected behaviour: 301 redirect to https://example.com

Request: https://example.com (on port 443)
Expected behaviour: Serve /var/www/html/index.html (or index.php, or index.txt, etc.)

example.com:80 {
    root /var/www/html
}

Request: http://example.com (on port 80)
Expected behaviour: Serve /var/www/html/index.html (or index.php, or index.txt, etc.)

Request: https://example.com (on port 443)
Expected behaviour: No response (no listener on port 443)

example.com:443 {
    root /var/www/html
}

Request: http://example.com (on port 80)
Expected behaviour: No response (no listener on port 80)

Request: https://example.com (on port 443)
Expected behaviour: Serve /var/www/html/index.html (or index.php, or index.txt, etc.)

example.com:80, example.com:443 {
    root /var/www/html
}

Request: http://example.com (on port 80)
Expected behaviour: Serve /var/www/html/index.html (or index.php, or index.txt, etc.)

Request: https://example.com (on port 443)
Expected behaviour: Serve /var/www/html/index.html (or index.php, or index.txt, etc.)

1 Like

Strongly recommend going through the Caddyfile docs, as well as the details in the Automatic HTTPS docs, both of which have a number of great examples.

1 Like

ok. I’ve gotten lost in so much change and this thread has gotten too long. I’m going to start a new thread as a new attempt to start from scratch. I hope you can help me resolve this