Newbie question: Caddy file vs. cli, documentation is misleading

[benibilme]

Hello,

I am complete newbie in caddy. I am running caddy version v2.2.1 h1:Q62GWHMtztnvyRU+KPOpw6fNfeCD3SkwH7SfT1Tgt2c= on manjaro system. I am following documentation with a sample configuration.

I have a simple index.html in a directory and following Caddyfile in the same directory

localhost:2015
file_server 

when I run caddy run in the directory, I receive

2021/01/17 09:00:05.463 INFO    using adjacent Caddyfile 
2021/01/17 09:00:05.465 INFO    admin   admin endpoint started  {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127.0.0.1:2019"]}
2021/01/17 09:00:05.466 INFO    http    enabling automatic HTTP->HTTPS redirects        {"server_name": "srv0"}
2021/01/17 09:00:05.466 INFO    tls.cache.maintenance   started background certificate maintenance      {"cache": "0xc0004709a0"}
2021/01/17 09:00:05.498 INFO    tls     setting internal issuer for automation policy that has only internal subjects but no issuer configured  {"subjects": ["localhost"]}
run: loading initial config: loading new config: http app module: start: tcp: listening on :80: listen tcp :80: bind: permission denied

However if I give the command via CLI like caddy run file_server -listen :2015, everything works as expected.

I validated the file with caddy validate. I activated debug global option, I receive following output.

2021/01/17 09:02:13.146 INFO    using adjacent Caddyfile  
2021/01/17 09:02:13.148 INFO    admin   admin endpoint started  {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127.0.0.1:2019"]}
2021/01/17 09:02:13.149 INFO    http    enabling automatic HTTP->HTTPS redirects        {"server_name": "srv0"}
2021/01/17 09:02:13.149 INFO    tls.cache.maintenance   started background certificate maintenance      {"cache": "0xc0003ca9a0"}
2021/01/17 09:02:13.182 INFO    tls     setting internal issuer for automation policy that has only internal subjects but no issuer configured  {"subjects": ["localhost"]}
2021/01/17 09:02:13.183 INFO    tls     cleaned up storage units
2021/01/17 09:02:13.329 INFO    pki.ca.local    root certificate is already trusted by system   {"path": "storage:pki/authorities/local/root.crt"}
2021/01/17 09:02:13.329 DEBUG   http    starting server loop    {"address": "[::]:2015", "http3": false, "tls": true}
run: loading initial config: loading new config: http app module: start: tcp: listening on :80: listen tcp :80: bind: permission denied

I have disabled auto_https via global section. it removed the permission error, that removed port 80 permission error but the site is not being served as expected. index.html file is not being served.

I believe documentation is missing or outdated. The simple Caddyfile does not work as described in the documentation or I misunderstood something.

Any help much appreciated.

[benibilme]

My problem persist, however if I change the localhost:2015 to just :2015 in caddyfile everything works beautifully.

This does not work

localhost:2015  {
      file_server 
}

This works

:2015  {
      file_server 
}

[matt (Matt Holt)]

When you say “works” or “does not work” what do you mean exactly?

@benibilme

I believe documentation is missing or outdated. The simple Caddyfile does not work as described in the documentation

Which documentation?

[francislavoie]

@benibilme

run: loading initial config: loading new config: http app module: start: tcp: listening on :80: listen tcp :80: bind: permission denied

This error is typically because you have another web server already running, using port 80.

How did you install Caddy? You didn’t fill out the help thread template, so we’re missing that piece of context. You may already have Caddy running automatically as a service depending on how you installed it (which is all noted in the docs). Or maybe you have a server like Apache running.

[benibilme]

Hello,

Thank you for the response, the documentation below shows all options of address entry which includes localhost:port

Caddyfile Concepts - Caddy Documentation

Works means, as stated in my initial post, not getting

run: loading initial config: loading new config: http app module: start: tcp: listening on :80: listen tcp :80: bind: permission denied

error and serving the index.html file in the same directory in which I started caddy via caddy run command.

According to documentation, the Caddyfile with the following content in the directory where caddy run is called

localhost:2021
file_server 

should work. But does not.

However

:2021
file_server 

works.

I have found Arch Caddy Wiki following notation which also works.

{
   http_port 2020
}

localhost:2020
file_server

If this is not a bug, the documentation should clear this issue. localhost:port does not work in my manjaro system. I tested in two different manjaro system.

[benibilme]

Hello,

I have installed it via manjaro/arch package manager pacman. I have not done any configuration or building etc. I have tried tens of times and before posting this message, I tried again. No it is not because of another process is using that port. I am not running web server other than caddy. I have checked if there is any other running caddy process or other any application is using that port via sudo lsof -i -P -n | grep LISTEN . I have also tried many other ports just to be sure.

Thank you for the response.

[francislavoie]

@benibilme

I am not running web server other than caddy. I have checked if there is any other running caddy process or other any application is using that port via sudo lsof -i -P -n | grep LISTEN

Right, it’s because you’re running Caddy as a user that doesn’t have permissions to bind to low ports, then. You need to have the CAP_NET_BIND_SERVICE capability set for the program you want to run.

@benibilme

According to documentation

localhost:2021
file_server 

should work. But does not.

However

:2021
file_server 

works.

That’s because of these Auto HTTP activation rules:

Automatic HTTPS - Caddy Documentation

[benibilme]

But the documentation does not point out that. The documentation just proposes using higher ports with localhost:port structure, if you get permission denied error. I have no idea about CAP_NET_BIND_SERVICE. I have not seen that in caddy documentation as well. I almost read all documentation including the link you provided before posting my initial message. I have already tried disabling auto_https etc. which did not serve the index.html as stated in my initial post.

I just want to point that the documantation is not clear about localhost:port configuration and what must go with it.

It is clear that :port is different localhost:port. There are some peculiar behaviour differences not clearly documented. localhost:port requires additional settings as stated in arch wiki and You.

Again thank you for the response.

[francislavoie]

@benibilme

I have no idea about CAP_NET_BIND_SERVICE. I have not seen that in caddy documentation as well.

Honestly, that falls under the category of “know how to use your computer”. The Caddy docs can’t teach you everything you need to know about linux, that’s not its purpose.

Like I said, the Automatic HTTPS docs do describe the behaviour you’re seeing, in that Caddy attempts to enable Automatic HTTPS when you use a domain such as localhost. Part of that is enabling a HTTP->HTTPS redirect, which requires binding to port 80 so that http://localhost will properly redirect you to https://localhost.

The docs also say that if you only want HTTP, you can prefix your site address with http:// to tell Caddy “I don’t want HTTPS”, or you can add the auto_https off global option to turn off that behaviour altogether.

[benibilme]

Again thank you for the response.

I have done all the things you said before my initial post, disabled auto_https. but at that time the site did not work properly.

I have been linux user for almost 20 years, I am around 50. Before that I used solaris. But I was just user. I have been using only linux in last 15 years in my labtop, server and desktop at home. I running my own nas, configured everything. That “know how to use your computer” did not come up all those years. I am really offended about what you said. I can throw many things that you must have not heard. I really wonder how long have been solely using linux?

I am not really into web servers, this is not my thing. I am only here just because caddy claims to be easy to use, that’s why I am here. Such a simple configuration should work out of the box or the documentation should point out it clearly. Arch linux wiki can do that but caddy documentation can not? I accept, my stupidity that I read almost all dam documentation and tried many things instead of looking at just arch wiki. Thank you for teaching me.

As I stated in the beginning of my post, I believe the documentation must clearly state the requirements you stated here in the getting started caddy file section and provide a full working example for use of caddy web server with caddy file and with unprivileged ports which is cleary missing in the documentation.

[benibilme]

I posted a bug or feature report in github to resolve this issue. This should have been cleared out long ago. Newcomers must at least should be able to find the answer and must not lose the time I lost.

Github bug report.

[matt (Matt Holt)]

Just getting back from a holiday weekend…

@benibilme

Works means, as stated in my initial post, not getting

run: loading initial config: loading new config: http app module: start: tcp: listening on :80: listen tcp :80: bind: permission denied

Hmm, I think that’s an unreasonable reading of those docs. That page describes the address/port combinations that Caddy accepts, it doesn’t guarantee you won’t get an error from the operating system. There are lots of factors that can cause binding a socket to fail. In your case, the operating system is not giving your process permission to bind a port as low as 80, as @francislavoie has already pointed out.

@benibilme

But the documentation does not point out that. The documentation just proposes using higher ports with localhost:port structure, if you get permission denied error. I have no idea about CAP_NET_BIND_SERVICE. I have not seen that in caddy documentation as well.

Sorry you did not know about this. But again, I second Francis. We cannot document every possible operating system or environment. It is not feasible. Our docs can only describe how Caddy works, not how to use your computer.

@benibilme

It is clear that :port is different localhost:port. There are some peculiar behaviour differences not clearly documented.

We document the automatic HTTPS behavior very thoroughly on this page, as you’ve probably read already: Automatic HTTPS — Caddy Documentation

@benibilme

but at that time the site did not work properly.

Can you be more specific about what “did not work” means? I can’t improve the documentation otherwise.

@benibilme

That “know how to use your computer” did not come up all those years. I am really offended about what you said.

I am sorry for providing you free software that you are invited to collaborate on, rather than asked to pay for. I know it wasn’t much of a thing 20 years ago, but in open source, we collaborate rather than complain. It’s more productive and helps others. Since open source is free, collaboration is the primary way things get done.

@benibilme

I am only here just because caddy claims to be easy to use, that’s why I am here.

Caddy is easy to use, but one must know how web servers work and how to administer a system before that. This is true of all advanced software.

@benibilme

Such a simple configuration should work out of the box

Well, it does:

https://twitter.com/wesbos/status/1348751941623570434

@benibilme

As I stated in the beginning of my post, I believe the documentation must clearly state the requirements you stated here in the getting started caddy file section and provide a full working example for use of caddy web server with caddy file and with unprivileged ports which is cleary missing in the documentation.

The permissions error you got is actually clearly explained in our tutorial: https://caddyserver.com/docs/caddyfile-tutorial

If the HTTP and HTTPS ports (80 and 443, respectively) are privileged ports on your OS, you will either need to run with elevated privileges or use a higher port. To use a higher port, just change the address to something like localhost:2015 and change the HTTP port using the http_port Caddyfile option.

I think our docs are very good at explaining what Caddy does – and many others have also told us this too. That is the purpose of the docs. Beyond that, administering your system or learning how to troubleshoot errors your OS gives you are beyond the scope of our documentation. You’ll have to look into the Arch docs or file a bug with Arch if you believe it is a bug (since, for example, your basic config works fine on macOS without permissions errors or needing to set the http_port option.).

@benibilme

Arch linux wiki can do that but caddy documentation can not?

The arch linux wiki is the right place to document Arch linux. Not the Caddy docs.

This is apples and oranges: comparing a wiki to official documentation? We have a wiki – you are welcome to contribute a tutorial that explains how to set up Arch linux so that you don’t get permissions errors.