Caddy hangs when doing stop

Help
1

1. Caddy version (caddy version):

v2.4.3 h1:Y1FaV2N4WO3rBqxSYA8UZsZTQdN+PwcoOcAiZTM8C0I=

2. How I run Caddy:

I have caddy installed as a binary (/usr/bin/caddy) and I goto the folder with my Caddyfile and run

$ caddy start

a. System environment:

Ubuntu 18.04.5, linux 4.15.0-147-generic

b. Command:

$ caddy stop

d. My complete Caddyfile or JSON config:

getcroc.schollz.com {
        file_server * {
                root /www/croc/src/install
                index default.txt
        }
        log {
                output file logs/getcroc.schollz.com.log
                format single_field common_log
        }
}

3. The problem I’m having:

There are two problems, and they may be related.

  1. The server stops serving ~1/week (several times documented here).

  2. I goto restart the server by stopping it with caddy stop and it hangs. If I just run caddy stop it won’t stop. Each time I end up having to kill it with pkill -9 caddy.

I can even run caddy start and then later run caddy stop in the same terminal and it still hangs:

$ caddy stop
2021/09/02 16:51:41.594 INFO    admin.api       received request        {"method": "POST", "host": "localhost:2019", "uri": "/stop", "remote_addr": "127.0.0.1:52832", "headers": {"Accept-Encoding":["gzip"],"Content-Length":["0"],"Origin":["localhost:2019"],"User-Agent":["Go-http-client/1.1"]}}
2021/09/02 16:51:41.594 WARN    admin.api       exiting; byeee!! 👋

The terminal never comes back and I have to Ctl+C. Then I can see that caddy never actually exited:

$  ps aux | grep caddy
root     22023  2.7  1.0 739836 83176 pts/0    Sl   16:43   0:17 caddy run --pingback 127.0.0.1:40007

The only way to actually kill is to use pkill -9 caddy.

4. Error messages and/or full log output:

This is where I need help! Where should I find the error messages for caddy? I do have the log file for the website and it is pretty weird…it just shows 404 errors! This is when caddy was running but was unresponsive to requests. At the same time it is unresponsive to caddy stop.

...
xxx - - [02/Sep/2021:11:51:44 +0000] "GET / HTTP/2.0" 404 0
xxx - - [02/Sep/2021:14:13:28 +0000] "GET / HTTP/2.0" 404 0
xxx - - [02/Sep/2021:16:13:33 +0000] "GET / HTTP/2.0" 404 0

until I restart Caddy (pkill -9 caddy; caddy start) and then is responsive to requests, but unresponsive to caddy stop:

xx - - [02/Sep/2021:16:49:01 +0000] "GET / HTTP/2.0" 200 25248

5. What I already tried:

I tried searching about caddy hanging but didn’t find anything that quite fit. LMK if I missed!

6. Links to relevant resources:

N/A

2

$ caddy will run and then exit immediately because it is missing a subcommand.

In the stderr of the terminal where you ran caddy.

1 Like
3

Whoops, meant to say caddy start is how I run caddy. Updated above.

4

In that case these are the only error messages:

2021/09/02 17:49:47.644 INFO    admin.api       received request        {"method": "POST", "host": "localhost:2019", "uri": "/stop", "remote_addr": "127.0.0.1:56624", "headers": {"Accept-Encoding":["gzip"],"Content-Length":["0"],"Origin":["localhost:2019"],"User-Agent":["Go-http-client/1.1"]}}
2021/09/02 17:49:47.644 WARN    admin.api       exiting; byeee!! 👋

and then it hangs.

Although I’ve been trying it over and over this morning and one time it actually completed a stop:

2021/09/02 17:49:21.788 INFO    admin.api       received request        {"method": "POST", "host": "localhost:2019", "uri": "/stop", "remote_addr": "127.0.0.1:55934", "headers": {"Accept-Encoding":["gzip"],"Content-Length":["0"],"Origin":["localhost:2019"],"User-Agent":["Go-http-client/1.1"]}}
2021/09/02 17:49:21.788 WARN    admin.api       exiting; byeee!! 👋
2021/09/02 17:49:26.490 INFO    tls.cache.maintenance   stopped background certificate maintenance  {"cache": "0xc0001be930"}
2021/09/02 17:49:26.491 INFO    admin   stopped previous server {"address": "tcp/localhost:2019"}
2021/09/02 17:49:26.491 INFO    admin.api       shutdown complete       {"exit_code": 0}

The stop completed after 5 seconds. But in my other tries I waited several minutes and it never actually stopped. (I did not try waiting longer than a few minutes…should I try that?)

5

FWIW I also tried with two other versions - the very latest has the same problem for me (v2.4.4 h1:QBsN1jXEsCqRpKPBb8ebVnBNgPxwL50HINWWTuZ7evU=) and then I tried an older version (v2.0.0 h1:pQSaIJGFluFvu8KDGDODV8u4/QRED/OPyIR+MWYYse8=) and the problem ceased. I can try a git bisect style investigation if needed.

6

Can you explain what is going on more precisely? It sounds like there are a few issues here, and so far I suspect something weird about your OS, hardware, network, or I/O.

That’s all it shows? From the very beginning? That means a file is not found. No index file in the root directory, /www/croc/src/install.

You’re welcome to do a git bisect – more info is better than less, but I’m still not clear on what the problem is in the first place. You seem to be, so go ahead and do that if you know what you’re looking for.

7

This topic was automatically closed after 30 days. New replies are no longer allowed.