V2: ACME errors about agree

1. My Caddy version (caddy version):

v2.0.0-beta.19 h1:6kbQ5jf/lWjD+o3uuq7rnfrvw+x5UU3tuwGpZsLKr7M=

2. How I run Caddy:

Followed this guide exactly: https://caddyserver.com/docs/install
Did the manually install as a service

a. System environment:

Ubuntu 16.04, using systemd

b. Command:

/usr/bin/caddy run --config /etc/caddy/Caddyfile --resume --environ

c. Service/unit/compose file:

# This service file requires the following:
#
# 1) Group named caddy:
#      $ groupadd --system caddy
#
# 2) User named caddy, with a writeable home folder:
#      $ useradd --system \
#           --gid caddy \
#           --create-home \
#           --home-dir /var/lib/caddy \
#           --shell /usr/sbin/nologin \
#           --comment "Caddy web server" \
#           caddy
#
# 3) Caddyfile at /etc/caddy/Caddyfile that is
#    readable by the caddy user
#

[Unit]
Description=Caddy Web Server
Documentation=https://caddyserver.com/docs/
After=network.target

[Service]
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --config /etc/caddy/Caddyfile --resume --environ
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

d. My complete Caddyfile or JSON config:

http://met-zoom-test.bu.edu:80 {
        reverse_proxy / localhost:9009
}

3. The problem I’m having:

At first I tried the Caddyfile without the http:// and port 80. However, whenever I would try going to the domain, nothing would load. But I checked and port 80 was open so I figured the issue was with Caddy. Somewhere along the way Caddy started auto switching to HTTPS, and this was an issue as the certificate never went through because of the ACME error. So first I tried disabling https in the caddyfile, but that did nothing and kept trying to do https (I flushed the DNS on my side to make sure it wasn’t my computer). Now I’m completely stuck as I can’t get it to stop forcing HTTPS, and I keep getting errors about must agree to terms.

4. Error messages and/or full log output:

Mar 25 12:28:42 MET-ZOOM caddy[10297]: 2020/03/25 12:28:42 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed :: must agree to terms of seMar 25 12:28:43 MET-ZOOM caddy[10297]: 2020/03/25 12:28:43 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed :: must agree to terms of seMar 25 12:28:44 MET-ZOOM caddy[10297]: 2020/03/25 12:28:44 [ERROR] attempt 1: [met-zoom-test.bu.edu] Obtain: acme: error: 400 :: POST :: https://acme-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed :: must agMar 25 12:29:11 MET-ZOOM caddy[10297]: 2020/03/25 12:29:11 http: TLS handshake error from 168.122.70.188:59635: no certificate available for 'met-zoom-test.bu.edu'
Mar 25 12:29:14 MET-ZOOM caddy[10297]: 2020/03/25 12:29:14 http: TLS handshake error from 168.122.70.188:59644: no certificate available for 'met-zoom-test.bu.edu'
Mar 25 12:29:17 MET-ZOOM caddy[10297]: 2020/03/25 12:29:17 http: TLS handshake error from 168.122.70.188:59653: no certificate available for 'met-zoom-test.bu.edu'
Mar 25 12:29:37 MET-ZOOM caddy[10297]: 2020/03/25 12:29:37 http: TLS handshake error from 168.122.70.188:59786: no certificate available for 'met-zoom-test.bu.edu'
Mar 25 12:29:45 MET-ZOOM caddy[10297]: 2020/03/25 12:29:45 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme-staging-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed :: must agree to terMar 25 12:29:46 MET-ZOOM caddy[10297]: 2020/03/25 12:29:46 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme-staging-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed :: must agree to terMar 25 12:29:47 MET-ZOOM caddy[10297]: 2020/03/25 12:29:47 [ERROR] attempt 2: [met-zoom-test.bu.edu] Obtain: acme: error: 400 :: POST :: https://acme-staging-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed ::Mar 25 12:30:11 MET-ZOOM caddy[10297]: 2020/03/25 12:30:11 http: TLS handshake error from 168.122.70.188:59907: no certificate available for 'met-zoom-test.bu.edu'
Mar 25 12:30:34 MET-ZOOM caddy[10297]: 2020/03/25 12:30:34 http: TLS handshake error from 168.122.70.188:59970: no certificate available for 'met-zoom-test.bu.edu'
Mar 25 12:30:52 MET-ZOOM caddy[10297]: 2020/03/25 12:30:52 http: TLS handshake error from 168.122.70.188:60023: no certificate available for 'met-zoom-test.bu.edu'
Mar 25 12:31:47 MET-ZOOM caddy[10297]: 2020/03/25 12:31:47 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme-staging-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed :: must agree to terMar 25 12:31:48 MET-ZOOM caddy[10297]: 2020/03/25 12:31:48 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme-staging-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed :: must agree to terMar 25 12:31:49 MET-ZOOM caddy[10297]: 2020/03/25 12:31:49 [ERROR] attempt 3: [met-zoom-test.bu.edu] Obtain: acme: error: 400 :: POST :: https://acme-staging-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed ::Mar 25 12:33:49 MET-ZOOM caddy[10297]: 2020/03/25 12:33:49 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme-staging-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed :: must agree to terMar 25 12:33:50 MET-ZOOM caddy[10297]: 2020/03/25 12:33:50 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme-staging-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed :: must agree to terMar 25 12:33:51 MET-ZOOM caddy[10297]: 2020/03/25 12:33:51 [ERROR] attempt 4: [met-zoom-test.bu.edu] Obtain: acme: error: 400 :: POST :: https://acme-staging-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed ::Mar 25 12:38:51 MET-ZOOM caddy[10297]: 2020/03/25 12:38:51 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme-staging-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed :: must agree to terMar 25 12:38:53 MET-ZOOM caddy[10297]: 2020/03/25 12:38:53 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme-staging-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed :: must agree to terMar 25 12:38:54 MET-ZOOM caddy[10297]: 2020/03/25 12:38:54 [ERROR] attempt 5: [met-zoom-test.bu.edu] Obtain: acme: error: 400 :: POST :: https://acme-staging-v02.api.letsencrypt.org/acme/new-acct :: urn:ietf:params:acme:error:malformed ::Mar 25 12:41:11 MET-ZOOM systemd[1]: Stopping Caddy Web Server...

5. What I already tried:

I’ve tried googling this error for hours, and everything I’ve found has been for Caddy v1 (tls off, or -agree flag, etc.) I have no clue where to go from here other than switching to another web server. (I do remember at some point trying caddy v1 but some other issues came up that I don’t remember too clearly, honestly after looking at this for 2 days I just need some clear guidance before I try anything else)

Please upgrade to beta 20, it is fixed. :slight_smile: Thank you!

Thanks! I’m no longer receiving any issues from caddy logs, but the application doesn’t seem to be working. It’s a nodejs application on port 9009, and for caddy v1 I would just do reverse proxy transparent. I tried doing the v2 equivalent but I just get blank pages with response 200. This is my caddyfile right now:

met-zoom-test.bu.edu {
        reverse_proxy / localhost:9009 {
                header_up Host {http.request.host}
                header_up X-Real-IP {http.request.remote}
                header_up X-Forwarded-For {http.request.remote}
                header_up X-Forwarded-Port {http.request.port}
                header_up X-Forwarded-Proto {http.request.scheme}
        }
}

Am I missing anything?

In v2, path matchers are exact match instead of prefix matches. This means that your config is only handling requests to / and not to anything else. Instead, you should use * to match all paths, or you can even omit the path matcher because * is implicit as long as the next directive argument does not start with /.

Also, you can omit header_up Host, this is already done for you in v2. You can also shorten {http.request.remote} to {remote} (Caddyfile offers shorter versions of some placeholders for readability).

Altogether, it would look like this:

met-zoom-test.bu.edu {
        reverse_proxy localhost:9009 {
                header_up X-Real-IP {remote}
                header_up X-Forwarded-For {remote}
                header_up X-Forwarded-Port {port}
                header_up X-Forwarded-Proto {scheme}
        }
}
2 Likes

Here’s some docs about matchers: https://caddyserver.com/docs/caddyfile/concepts#matchers

1 Like

I updated my Caddyfile to match that, but when I tried systemctl reload caddy I got this error:

Job for caddy.service failed because the control process exited with error code. See "systemctl status caddy.service" and "journalctl -xe" for details.

Checking the status right after showed

Process: 11366 ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile (code=exited, status=1/FAILURE)

I stopped, disabled, enabled, and started caddy to make sure the new config would load and I still get a 200 code with an empty response (I tried emptying my browser cache too and no change). Am I just not waiting for things to propagate?

Ah, that’s because you’re telling Caddy to resume the prior configuration:

ExecStart=/usr/bin/caddy run --config /etc/caddy/Caddyfile --resume --environ

See the --resume flag?

(This is a safety feature to avoid losing data.)

Take out the resume flag, or use the API to update the config (recommended).

But if you take out the resume flag and use the API, you risk losing those changes if you restart or reload your server.

Perfect, it’s working now! Thank you so much!

1 Like

Incidentally, this was the actual mistake – not the --resume flag. Again, I recommend leaving the resume flag there, because it prevents data loss if you use the API.

Don’t stop the server to reload config. Doing so will incur downtime. :slight_smile:

Instead, use caddy reload (or systemctl reload caddy if using our service file) to reload config.

Glad you got it working!

Please see https://caddyserver.com/docs/command-line – all of this is documented. The right way to reload configuration is to use reload, not stop/start.

1 Like

Huh, so was this error not actually an issue?

Job for caddy.service failed because the control process exited with error code. See "systemctl status caddy.service" and "journalctl -xe" for details.

Depends. What was the error?

I’m not really sure, checking journalctl shows this around the reload:

Mar 25 14:23:57 MET-ZOOM systemd[1]: Reloading Caddy Web Server.
Mar 25 14:23:57 MET-ZOOM caddy[11366]: {"level":"info","ts":1585160637.862006,"msg":"using provided configuration","config_filMar 25 14:24:27 MET-ZOOM caddy[11366]: reload: sending configuration to instance: performing request: Post "http://localhost:2Mar 25 14:24:27 MET-ZOOM systemd[1]: caddy.service: Control process exited, code=exited status=1
Mar 25 14:24:27 MET-ZOOM systemd[1]: Reload failed for Caddy Web Server.

That’s weird, the log lines are truncated. Are you copying+pasting from the terminal? Annoyingly, journalctl is designed to be interactive, so copy+pasting from it isn’t sufficient, you’ll have to get the full output somehow.

Ah I think I see the issue now:

Mar 25 14:23:57 MET-ZOOM systemd[1]: Reloading Caddy Web Server.
Mar 25 14:23:57 MET-ZOOM caddy[11366]: {"level":"info","ts":1585160637.862006,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
Mar 25 14:24:27 MET-ZOOM caddy[11366]: reload: sending configuration to instance: performing request: Post "http://localhost:2019/load": dial tcp [::1]:2019: connect: connection refused
Mar 25 14:24:27 MET-ZOOM systemd[1]: caddy.service: Control process exited, code=exited status=1
Mar 25 14:24:27 MET-ZOOM systemd[1]: Reload failed for Caddy Web Server.

Something about the connection being refused. So I didn’t setup this server, it was given to me at work to setup this application on. However there are pretty stringent firewall rules on it, so my guess is that port 2019 is blocked and I need to allow traffic on it before Caddy reload will work?

1 Like

Hmm, possibly, can you find the logs prior to that too? Like when you ran the server before the reload? I’m curious of the precise network address the admin interface is listening on; the full logs will have it.

I’m not sure if this is what you mean by full logs, but here are the logs from the start most recent to the reload:

Mar 25 14:22:55 MET-ZOOM systemd[1]: Started Caddy Web Server.
Mar 25 14:22:55 MET-ZOOM caddy[11346]: caddy.HomeDir=/var/lib/caddy
Mar 25 14:22:55 MET-ZOOM caddy[11346]: caddy.AppDataDir=/var/lib/caddy/.local/share/caddy
Mar 25 14:22:55 MET-ZOOM caddy[11346]: caddy.AppConfigDir=/var/lib/caddy/.config/caddy
Mar 25 14:22:55 MET-ZOOM caddy[11346]: caddy.ConfigAutosavePath=/var/lib/caddy/.config/caddy/autosave.json
Mar 25 14:22:55 MET-ZOOM caddy[11346]: runtime.GOOS=linux
Mar 25 14:22:55 MET-ZOOM caddy[11346]: runtime.GOARCH=amd64
Mar 25 14:22:55 MET-ZOOM caddy[11346]: runtime.Compiler=gc
Mar 25 14:22:55 MET-ZOOM caddy[11346]: runtime.NumCPU=3
Mar 25 14:22:55 MET-ZOOM caddy[11346]: runtime.GOMAXPROCS=3
Mar 25 14:22:55 MET-ZOOM caddy[11346]: runtime.Version=go1.14.1
Mar 25 14:22:55 MET-ZOOM caddy[11346]: os.Getwd=/
Mar 25 14:22:55 MET-ZOOM caddy[11346]: LANG=en_US.UTF-8
Mar 25 14:22:55 MET-ZOOM caddy[11346]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Mar 25 14:22:55 MET-ZOOM caddy[11346]: HOME=/var/lib/caddy
Mar 25 14:22:55 MET-ZOOM caddy[11346]: LOGNAME=caddy
Mar 25 14:22:55 MET-ZOOM caddy[11346]: USER=caddy
Mar 25 14:22:55 MET-ZOOM caddy[11346]: SHELL=/usr/sbin/nologin
Mar 25 14:22:55 MET-ZOOM caddy[11346]: {"level":"info","ts":1585160575.8528292,"msg":"resuming from last configuration","autosave_file":"/var/lib/caddy/.config/caddy/autosave.json"}
Mar 25 14:22:55 MET-ZOOM caddy[11346]: {"level":"info","ts":1585160575.8541002,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["localhost:2019"]}
Mar 25 14:22:55 MET-ZOOM caddy[11346]: {"level":"info","ts":1585160575.8568852,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
Mar 25 14:22:55 MET-ZOOM caddy[11346]: {"level":"info","ts":1585160575.85716,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
Mar 25 14:22:55 MET-ZOOM caddy[11346]: 2020/03/25 14:22:55 [INFO][cache:0xc0003dcaa0] Started certificate maintenance routine
Mar 25 14:22:55 MET-ZOOM caddy[11346]: {"level":"info","ts":1585160575.8592014,"logger":"tls","msg":"cleaned up storage units"}
Mar 25 14:22:55 MET-ZOOM caddy[11346]: {"level":"info","ts":1585160575.859533,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["met-zoom-test.bu.edu"]}
Mar 25 14:22:55 MET-ZOOM caddy[11346]: {"level":"info","ts":1585160575.8715506,"msg":"autosaved config","file":"/var/lib/caddy/.config/caddy/autosave.json"}
Mar 25 14:22:55 MET-ZOOM caddy[11346]: {"level":"info","ts":1585160575.8718345,"msg":"serving initial configuration"}
Mar 25 14:23:57 MET-ZOOM systemd[1]: Reloading Caddy Web Server.
Mar 25 14:23:57 MET-ZOOM caddy[11366]: {"level":"info","ts":1585160637.862006,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
Mar 25 14:24:27 MET-ZOOM caddy[11366]: reload: sending configuration to instance: performing request: Post "http://localhost:2019/load": dial tcp [::1]:2019: connect: connection refused
Mar 25 14:24:27 MET-ZOOM systemd[1]: caddy.service: Control process exited, code=exited status=1
Mar 25 14:24:27 MET-ZOOM systemd[1]: Reload failed for Caddy Web Server.
Mar 25 14:25:41 MET-ZOOM systemd[1]: Stopping Caddy Web Server...
Mar 25 14:25:41 MET-ZOOM caddy[11346]: {"level":"info","ts":1585160741.7427309,"msg":"shutting down apps then terminating","signal":"SIGTERM"}
Mar 25 14:25:41 MET-ZOOM caddy[11346]: 2020/03/25 14:25:41 [INFO][cache:0xc0003dcaa0] Stopped certificate maintenance routine
Mar 25 14:25:41 MET-ZOOM caddy[11346]: {"level":"info","ts":1585160741.7433062,"msg":"shutdown done","signal":"SIGTERM"}
Mar 25 14:25:41 MET-ZOOM systemd[1]: Stopped Caddy Web Server.
Mar 25 14:25:41 MET-ZOOM systemd[1]: caddy.service: Unit entered failed state.
Mar 25 14:25:41 MET-ZOOM systemd[1]: caddy.service: Failed with result 'exit-code'.

Do you have IPv6 in your hosts file for localhost but don’t accept IPv6 connections maybe?

Firewall probably shouldn’t affect this one. Most don’t block from localhost. But who knows. Can you send this thread to your sysadmin to investigate on their end maybe?

2 Likes

Will do, thanks!

1 Like