Subdomains not redirecting from HTTP

1. Caddy version (caddy version):

v2.2.1 h1:Q62GWHMtztnvyRU+KPOpw6fNfeCD3SkwH7SfT1Tgt2c=

2. How I run Caddy:

Via systemd with Caddyfile

a. System environment:

Ubuntu 20.04
systemd 245 (245.4-4ubuntu3.3)

b. Command:

systemctl start caddy

c. Service/unit/compose file:

N/A

d. My complete Caddyfile or JSON config:

{
        experimental_http3
}
(logging) {
        log {
                output file /var/log/caddy/caddy.log {
                        roll_size 1gb
                        roll_keep 5
                }
        }
}
:80 {
        root * /var/www/html
        file_server
}
www.telesphoreo.me {
        import logging
        redir https://telesphoreo.me{uri}
}
telesphoreo.me {
        import logging
        root * /var/www/telesphoreo.me
        file_server browse
        encode gzip zstd
        php_fastcgi unix//run/php/php7.4-fpm.sock
        respond /assets/ 403
        respond /files/ 403
        respond /files/theme/ 403
        respond /files/theme/fonts/ 403
        respond /files/theme/images/ 403
        respond /uploads/2/6/3/9/26395298/ 403
        respond /uploads/2/6/3/9/26395298/background-images/ 403
        respond /nitrogen/ 403
        respond /wave/ 403
}
db.telesphoreo.me {
        import logging
        root * /usr/share/phpmyadmin
        file_server
        php_fastcgi unix//run/php/php7.4-fpm.sock
}
i.telesphoreo.me {
        import logging
        root * /var/www/i.telesphoreo.me
        file_server
}
panel.telesphoreo.me {
        reverse_proxy https://panel.telesphoreo.me:4431 # Apache
}
updater.telesphoreo.me {
        import logging
        root * /var/www/updater.telesphoreo.me
        file_server browse
}
packs.smokes-crystal.rocks {
        import logging
        redir https://i.telesphoreo.me{uri}
}
smokes-crystal.rocks {
        import logging
        root * /var/www/smokes-crystal.rocks
        file_server
}
always-loses.net {
        import logging
        root * /var/www/html
        file_server
}
www.always-loses.net {
        import logging
        redir https://always-loses.net{uri}
}

3. The problem I’m having:

So when I had Apache, I had a failsafe that would redirect any site that didn’t have a vhost to a page saying not found. This works as if I go to the site by the IP I get the page. However, if you go to a subdomain with HTTP, you get this page as well. For example, going to http://i.telesphoreo.me redirects to invalid vhost, but https://i.telesphoreo.me does not. I want to force HTTPS on all sites, but there are some subdomains that simply don’t have a vhost (and don’t need https) and redirect to the page. I can’t find anything on how to actually do this, I only see the conditions to disable automatic https (which I don’t want for the sites that do exist). Firefox automatically changes HTTP to HTTPS which solves the problem, but other browsers do not. Is there any way I could achieve HTTPS for everything except the IP of my server?
Also I guess I’ll ask if there’s a way to block listing of all directories AND directories within the directory but make all the files still accessible? I don’t want to manually have to tell every directory and directory within a directory to not list, but still have access to individual files.

4. Error messages and/or full log output:

5. What I already tried:

The way that I’ve thought of solving this is to specify http:// on a separate block for each domain and use a redir to https. This is exactly why I left Apache and I don’t want to have to have the same hacks I had for Apache all over again.
I also tried putting this in each block in my Caddyfile:

automatic_https {
        disable false
}

from JSON Config Structure - Caddy Documentation but this doesn’t work (I know it’s JSON but I thought I could still use it in the Caddyfile)

Edit: I realize that caddy adapt exists. I tried converting my Caddyfile to JSON which at a glance seems to have worked (although its massive) but I’m not sure how to start Caddy with JSON over a Caddyfile

6. Links to relevant resources:

If you look at your stdout logs (which you haven’t posted as asked by the thread template), you’ll see that it warns that having a :80 site block will override all HTTP → HTTPS redirects.

There’s been a lot of discussion about this, which you can see in this github thread:

A workaround would be to add this to your config:

http://telesphoreo.me,
http://i.telesphoreo.me,
<etc...> {
    redir https://{hostport}{uri}
}

Yeah, the Caddyfile is a separate config language altogether. It adapts to JSON. You can use the caddy adapt command to see the underlying JSON for your Caddyfile.

What you were trying to do there is the auto_https off global option, but that won’t have the effect you’re looking for; it will turn off certificate management altogether, but there’s the disable_redirects option which only turns off the HTTP → HTTPS redirects which is also not what you want.

You’d edit your systemd unit file with systemctl edit caddy to use your caddy.json file and remove the --adapter caddyfile option

Directories are only listed if you use file_server browse. If you remove browse, directory listing doesn’t happen.

I have no idea how to quote on here but I’ll do my best to reply. I’m aware that having :80 disables HTTP → HTTPS redirects. That’s the point of the thread. I want to know if I can have a failsafe for the IP without disabling the redirects. Seems like it should be pretty basic functionality. The IP of the server has nothing to do with the domain redirects, so it shouldn’t be affecting that functionality and yet it is. I realize that I can either remove it or add every subdomain as HTTP and force it to redirect to HTTPS as you suggested. but that’s kinda why I ditched Apache and complicated vhosts for that kind of stuff.

Thanks for the systemd info, didn’t know how to switch it but I’ll take a look.

Last part about the directories. I know removing the browse will deny listing, but that’s not what I want. I want to allow listing on the telesphoreo.me domain but not for every subfolder. I was wondering if there was a way to deny listing for many subfolders in without having to specify every directory like:
/1/ /1/2/ /1/2/3/ /1/2/3/4/ /1/2/3/4/5/. I know * exists but that denies access to all the files as well. I don’t want access to the files denied, only listing of those specific directories, which looks like it’s not possible without doing it manually. That’s what I figured but might as well have asked

Just select some text, and a “Quote” button should appear. Click it, and it’ll paste in a quote snippet into your reply box.

Discourse has a dumb bug though that causes it to sometimes not work if what you select crosses the boundary of some HTML elements. They basically slapped us with a “wont-fix” because they couldn’t replicate :roll_eyes:

See the github issue I linked. It’s not that simple, ultimately. A solution might be implemented in a later version, but unfortunately, at this time, what I suggested is the simplest solution.

Maybe something like this?

file_server / browse
file_server

That way only requests to the root will show the file listing, everything else won’t. Or if you need to only allow a certain few directories, then:

@browsable path /foo/* /bar/* /baz/*
file_server @browsable browse
file_server

Or to allow most but deny some:

@notBrowsable path /foo/* /bar/* /baz/*
file_server @notBrowsable
file_server browse

You’re right, it does look like it’s a bug

In my best effort to keep things simple, I’ve decided to just bin the failsafe and will just deal with the ERR_SSL_PROTOCOL_ERROR whenever someone accesses by IP.

The second solution code block worked exactly for my needs, thanks. Now all directories are blocked except the ones I want to allow listing too (which is fewer than blocking all of them).

While I still have you, I’ll ask one more question. Is there any way to enforce specific things globally. For example, is there any way I could bin the whole “import logging” in every file and just have it as a given for every block. I ask this because I also want to do error handling. I see it’s possible to do this:

handle_errors {
    @404 {
        expression {http.error.status_code} == 404
    }
    rewrite @404 /404.html
}

But I don’t want to have to add this to every webserver block. With Apache I had a .htacess in /var/www that had all of the ErrorDocuments that came before all of the vhosts. Also, is it possible to have one spot for all the error pages. Could I make a folder called /var/www/internal/errors or /etc/caddy/errors and have the error pages live there rather than duplicating the errors folder in every vhost.

Snippets (which you’re already using for logging) are your best bet for reuse. You can’t get around needing to import a snippet, in the best-case.

If you want just one specific location for your error pages for all sites, just change root in your handle_errors block:

handle_errors {
	root * /var/www/internal/errors
	@404 expression {http.error.status_code} == 404
	rewrite @404 /404.html
}

Also note that with snippets, you can provide arguments to generalize your snippets. There’s an example in the docs:

Just did that and it looks like Caddy likes it. But here’s the weird thing. It doesn’t work because every request is being accepted as 200. Even if you enter something random into the URL, it will return 200 and just a blank page. Removing the import errors makes everything go back to displaying the proper error codes. Not sure what the issue is, I’m using the code you posted above

Edit: using the code from Custom 404 Error Pages With Caddy V2 - RichJ plus the root * /var/www/internal/errors above seems to work for me, thanks for all your help.

Edit 2: So it looks like that code only works for error 404, not 403. I have a feeling that’s because the stuff I hid with the file_server code above actually hid it, not forbid it

This topic was automatically closed after 30 days. New replies are no longer allowed.