This is because curl doesn’t trust the certificates Caddy issued, because they’re issued by Caddy’s internal CA like I said earlier, which are not publicly trusted.
TLS involves trust. Clients need to trust that the server that encrypted the request used a set of keys that are validated/authorized to do so.
Systems ship a list of certificate authority roots which are trusted (as in the companies/entities issuing them have been vetted to be doing it properly, and securely; such as Let’s Encrypt for example, and many others, majority of which are non-free).
Caddy can ask Let’s Encrypt to issue certificates for your domains, but only if your server is publicly accessible (public DNS, and with ports 80/443 reachable).
For names like localhost or IP addresses, Caddy won’t attempt to get a publicly trusted certificate, because localhost can only ever resolve to 127.0.0.1, i.e. “this same machine”. So to deal with that, Caddy sets up it’s own internal CA to issue certificates with, but those are not publicly trusted.
It shouldn’t, and I haven’t seen any evidence that it does. Make absolutely sure that you’re running Caddy with the config you think you are, that it has properly reloaded after your changes by looking at the logs. You can also run caddy adapt --config /etc/caddy/Caddyfile to make sure that your Caddyfile properly adapts to a valid JSON config.
ok - my head is spinning - so the bottom line is … I will never be able to use Caddy to serve a page to my domain - either HTTP or HTTPS - because ( as I mentioned in first post on this thread) , my server has 80 and 443 available outbound , but inbound only ports above 5000 are allowed (though can be, and are being, redirected to 443 and 80 in router before getting to the server) ,
The higher port numbers e.g. 5000 are being redirected to 80 and 443.
(Inbound) Port 9991 is redirected to 443.
(Inbound) Port 9992 is redirected to 80.
Obovuosly their restriction is Inbound only… redirectiions work for everything else… i can use port 22 (ssh) port (9 WOL ) port 3389 (RDP) inbound … simply by portforwarding in the router… But Caddy cannot work with that it seems. . The “Caddy Works!” page gave me premature hope…
When it gets to the Caddy Server - Caddy picks it up on ports 443 and port 80. …
I’m sorry, I’ve been trying to follow this thread, but there are so many things wrong that it’s hard to make any sense of it. Let’s take this back to square 1.
I’m glad you’re not using containers; that helps simplify things a bit. As far as I can tell according to my experiments, Caddy does work properly for your given configs. This config:
:80 {
respond "Yahaha! You found me!"
}
outputs this:
$ go run -exec ./setcap.sh main.go run
2021/07/18 18:07:15.432 INFO using adjacent Caddyfile
2021/07/18 18:07:15.434 INFO admin admin endpoint started {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127.0.0.1:2019"]}
2021/07/18 18:07:15.434 INFO http server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server {"server_name": "srv0", "http_port": 80}
2021/07/18 18:07:15.434 INFO tls.cache.maintenance started background certificate maintenance {"cache": "0xc000450690"}
2021/07/18 18:07:15.434 DEBUG http starting server loop {"address": "[::]:80", "http3": false, "tls": false}
2021/07/18 18:07:15.434 INFO tls cleaning storage unit {"description": "FileStorage:/home/matt/.local/share/caddy"}
2021/07/18 18:07:15.434 INFO autosaved config (load with --resume flag) {"file": "/home/matt/.config/caddy/autosave.json"}
2021/07/18 18:07:15.434 INFO serving initial configuration
2021/07/18 18:07:15.465 INFO tls finished cleaning storage units
and it behaves like this:
$ curl -v "http://127.0.0.1"
* Trying 127.0.0.1:80...
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: 127.0.0.1
> User-Agent: curl/7.74.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: Caddy
< Date: Sun, 18 Jul 2021 18:07:24 GMT
< Content-Length: 21
<
* Connection #0 to host 127.0.0.1 left intact
Yahaha! You found me!
But you’re reporting that it’s issuing a redirect. Ok, I believe you’re seeing a redirect. But I’m not. So how can I reproduce the behavior you’re seeing?
These are common requirements for any basic production website, not just Caddy. The main difference is to set your DNS records properly before running Caddy so it can provision certificates.
If your domain’s A/AAAA records point to your server,
ports 80 and 443 are open externally,
Caddy can bind to those ports (or those ports are forwarded to Caddy),
Yes I see that . What specifically ? From my server running Caddy I can outbound on 80 and 443 … and that server running Caddy receives inbound on 80 and 443. Like I said - other services SSH (22) and WOL (9) and RDP (3389) all take inbound traffic that has been port-forwarded in the router.
So I will put that in the Caddyfile, would you like the auto_https off or commented out ?
And then curl to my address… you can use the IP or the domain name.
Caddy only needs to be able to make requests to ACME CAs to set up Automatic HTTPS – that’s its only outbound concern.
The main concern is that ports 80 and 443 need to be open for inbound connections, because those are the ports necessary to for the ACME HTTP and TLS-ALPN challenges, i.e. the HTTP and HTTPS ports respectively.
See the list below, ports 80 and 443 are well-known ports for HTTP and HTTPS:
auto_https off is only relevant if your config actually meets the activation requirements as listed in the Automatic HTTPS docs. A site block with :80 does not meet the activation requirements, so you do not need that. auto_https off basically means “ignore the activation requirements and just never activate Automatic HTTPS features”
so when it receives something on 80 , it will not try to redirect it to 443/https ??? Thats good to know !! That was the reason I was using auto_https off … to prevent it from trying to redirect it (as I assumed it would fail ). But you are saying even when not using that option it still will not even try to redirect it … even it received the traffic on port 80.
Correction: when Caddy is configured to use the HTTP port (80 by default), it will not redirect to HTTPS, since you explicitly configured HTTP. That’s explained in the post above by Francis and our docs.
Only add something to a config if you need it.
Related to my correction of your statement above, receiving traffic on a port is different from being configured to use a port.
By default, Caddy will redirect requests received on the HTTP port to the HTTPS port, but if you explicitly configure to use HTTP, those redirects won’t happen.
BUT when I change the Caddyfile back to original state, it serves up the “Caddy works!” page.
I think it gets the incoming request ( on port 80 ) and then tries to redirect it to https… That why I turned off the option before … But am told that the option makes no difference
You think, or you’re sure? Because my request definitely isn’t even connecting to anything. You’ve got some network misconfiguration somewhere with your ISP or your home network.
You cannot reliably test your edge configuration from within your network unless you understand exactly how your router works. Some do what’s called hairpin NAT, for example, which won’t even send your traffic to the outside world if its destination is inside your own network. An easy test is to turn off your phone’s WiFi and try loading your site over your cell data connection. I bet it will time out. If so, that’s the first thing you need to fix (and it has nothing to do with Caddy).
Makes no difference in your case because there’s nothing in your config that would activate automatic HTTPS.
its almost 3am here and I need to work in the morning … Sorry my wording is bad as it is late. I will check it tomorrow. But - like this - with no change to the router/network etc, and the unchanged Caddyfile (pristine after installation) , I get the “Caddy works!” page in a browser pointing to : ` http://…:9992 … That say to me that the Caddy server gets the traffic ( on port 80 ) and replies , and the reply gets all the way to the external web browser.
IF the auto -ssl is not going to ever work - surely I can load in certs manually ? Do I really need to toss out Caddy ? I hope not… good night
That is - from outside - I can show the “Caddy Works!” in a browser (pointing to port 9992 ) … But nothing I have tried will show a page of my own html or a curl response from outside…
And how do you know you’re connecting from outside your home network? Are you using your cell phone over cellular/LTE? Are you physically in a different location on a different internet connection? Are you remotely controlling some other machine from which you’re testing the requests?
Because when Matt and I try to make requests to emupay.me:9992, we can’t connect.
Please be as specific as possible. We’re going in circles here because we’re unclear on a lot of details of your setup that aren’t being clearly explained.
Post your current config at every step, the logs you’re seeing, etc.
[Mon Jul 19] root@batopi: /etc/caddy$ cat Caddyfile
# The Ca3ddyfile is an easy way to configure your Caddy web server.
:80 {
}
[Mon Jul 19] root@batopi: /etc/caddy$
try this: http://194.124.229.155:9992 ← edited typo is fixed
By the way - I really appreciate your help… I mean it is really amazing ! and I thank you for every minute you are helping me solve my problems here. I cannot say in words how happy and appreciative I am about it… just ty, ty, ty ! (ps I know my language not always so clear and I am sorry about that). I already tried for hours/days of fighting this… and only with your help in last 24 hours do I feel I make some progress
