"Templates" doesn't work for "index.shtml"

1. The problem I’m having:

I have locally saved mine old web-site. This site was built with SSI, CGI (Perl scripts) technologies. I already successfully adapt it for running with lighttpd. Now I want to adapt this site for running with Caddy2.

So, for replacing original SSI technology I need to use Caddy’s templates with include. Goal is to include one HTML file into another.

Example of my site’s file structure:
web-site’s root/index.shtml
web-site’s root/left.html

“index.shtml” includes “left.html” by this string:
<!--#include virtual="left.html"-->

But for now, I am using these test files with default delimiters “{{” “}}”:
web-site’s root/index.html
web-site’s root/index.shtml
web-site’s root/include.htm

Now my tryouts with defaults:

  1. If index file is index.html curl -v http://example.com/ - include works
  2. If directly get “index.html” curl -v http://example.com/index.html - include works
  3. If use as index file “index.shtml” curl -v http://example.com/index.shtml - include do not work nor with indirect access, nor with direct access

My bet that problem is somehow connected with MIME. Docs:

…any responses that do not have a qualifying Content-Type will not be evaluated as templates

But, as testing “index.shtml” I am using simply renamed same “index.html”:

<html>
<body>
{{include "./include.htm"}}
<br>
{{.RemoteIP}}
</body>
</html>

Now I am absolutely do not know what to do else. Thanks for any help!

2. Error messages and/or full log output:

...
2023/12/14 10:17:28.489 DEBUG   http.auto_https adjusted config {"tls": {"automation":{"policies":[{}]}}, "http": {"servers":{"srv0":{"listen":[":80"],"routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"vars","root":"/srv"},{"encodings":{"gzip":{}},"handler":"encode","prefer":["gzip"]},{"delimiters":["{{","}}"],"handler":"templates","mime_types":["text/html","text/plain","text/markdown"]}]},{"handle":[{"executable":"/usr/local/bin/test.sh","handler":"cgi"}],"match":[{"path":["/test"]}]},{"handle":[{"browse":{},"handler":"file_server","hide":[".git","/etc/caddy/Caddyfile"],"index_names":["index.shtml"]}]}]}],"terminal":true}],"automatic_https":{"disable":true}}}}}
...
2023/12/14 10:17:49.762 DEBUG   http.handlers.file_server       sanitized path join     {"site_root": "/srv", "request_path": "/", "result": "/srv"}
2023/12/14 10:17:49.762 DEBUG   http.handlers.file_server       located index file      {"filename": "/srv/index.shtml"}
2023/12/14 10:17:49.762 DEBUG   http.handlers.file_server       opening file    {"filename": "/srv/index.shtml"}

curl output:

clear && curl -vL http://my.example.com/index.shtml

*   Trying MY_IP:80...
* Connected to my.example.com (MY_IP) port 80 (#0)
> GET /index.shtml HTTP/1.1
> Host: my.example.com
> User-Agent: curl/7.81.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Content-Length: 77
< Etag: "s5nk8s25"
< Last-Modified: Thu, 14 Dec 2023 10:38:52 GMT
< Server: Caddy
< Date: Thu, 14 Dec 2023 10:40:19 GMT
<
<html>
<body>
{{include "./include.htm"}}
<br>
{{.RemoteIP}}
</body>
</html>
* Connection #0 to host my.example.com left intact

3. Caddy version:

docker exec affectionate_lamport caddy version:
v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A=

4. How I installed and ran Caddy:

a. System environment:

Ubuntu Server 22, x86_64, Docker.

b. Command:

build.sh:

docker build --rm --tag "mycaddy:latest" ./
docker run --rm --tty --publish 80:80 --publish 443:443 mycaddy:latest

Run:

nano Caddyfile && ./build.sh

c. Service/unit/compose file:

Dockerfile:

FROM caddy:builder AS builder

RUN xcaddy build \
    --with github.com/aksdb/caddy-cgi/v2

FROM caddy:latest

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

COPY ./Caddyfile /etc/caddy/Caddyfile
COPY ./site /srv
COPY ./site/cgi-bin/test.sh /usr/local/bin/

EXPOSE 80
EXPOSE 443

VOLUME /data
VOLUME /config

d. My complete Caddy config:

Caddyfile:

{
        debug
        order cgi before respond
}

http://my.example.com {
        file_server {
                index index.shtml
                hide .git
                browse
                }
        root * /srv
        cgi /test /usr/local/bin/test.sh
        templates * {
                mime "text/html" "text/plain" "text/markdown"
                between "{{" "}}"
# "<!--#" "-->"
                }
        encode gzip

#       @shtml path *.shtml
#       header @shtml Content-Type text/html
}

Looks like /usr/share/mime/globs2 on ubuntu is read first, and this file doesn’t have shtml in it.

It can be added using Modifying MIME types

Or you can add this to your Caddyfile:

@shtml path *.shtml
header @shtml Content-Type "text/html; charset=utf-8"
2 Likes

It just hit me: I should check MIME types support INSIDE DOCKER CONTAINER! :slight_smile:
And inside docker - it’s Alpine OS. Ubuntu is my host OS.

In Alpine’s “/etc/mime.types” there is really no SHTML.

Solution: add in Dockerfile RUN echo "text/html shtml" >> /etc/mime.types.

1 Like

But I have no luck with setting up Caddyfile still…

I have this directory structure:

.
├── include.htm
├── index.shtml
└── subdir
    ├── include.htm
    └── index.shtml

Where “index.shtml” not usual files with direct access, but index file with in-direct access (there is no filename with extension in request path).

And adding this to Caddyfile:

@shtml path *.shtml
header @shtml Content-Type "text/html; charset=utf-8"

Does not work for indexes curl -v --insecure https://example.com/.

But works while direct access like curl -v --insecure https://example.com/index.shtml.

If I change matcher in Caddyfile with adding “/”:

@shtml path / *.shtml

Only root’s index starting to work, but index in subdirectory does not work.

If I try another change of matcher:

@shtml path /* *.shtml

All indexes are working, including ones in subdirs, but that ^^ header starting to add to any file, not only with SHTML extension. For example, I have file “/files/image.png” - and it sending to client with header “text/html; charset=utf-8”.

So, I have 2 questions about writing right matchers:

  1. How to, with 100% sure and accuracy, detect any file by it’s extension in any place/path? (I would need it not only for SHTML, but later for my CGI scripts)
  2. How to do ^^ the same, but for indexes.

You shouldn’t need this at all if you were able to fix the mime types in your container.

Not really possible because the file_server directive does its own thing based on OS environnement.

For your CGI scripts you should just set any response headers as appropriate from those scripts.

I know. Just want to implement second variant also as a solution - do not like have unfinished job without result :slight_smile:

So, there is no way to detect any file by it’s extension in any place/path? I simply could not catch all request with path containing, for example, *.sh files?

I don’t have a solution for you.

Still, I recommend you rename all your HTML files to remove the S. Not difficult with a quick recursive bash loop using a pattern.

1 Like

Yeap, it was probably fastest solution in my case.

But, I am that rare and strange person, who does not want fish, but wants fish pole :slight_smile:
In other words - I want to understand how and why so things are working, not just get some solution. Ofc, in adequate amount of time :slight_smile: Not full hacking of every system (it would require too much time).

Sorry, I don’t have the time to go down that rabbit hole with you. From my perspective, you should just do it the right way (rename files, or update system mime mappings) and call it a day.

The problem is directives like file_server have their own logic and trying to write config that match that logic for mapping to index files and such is counterproductive, you’d be doubling the amount of system calls and complicating your config needlessly.

It’s Okay. You already helped a lot, thank you!

You are right. There is misunderstanding: I thought, that Caddyfile way was ALSO A RIGHT way. Now you tell me that it is NOT - and I am happy knowing that it is dead end. So, no need to spend time and break head at this wall :slight_smile:

1 Like

Yeah, setting the content type in Caddyfile is a hack, not a real solution.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.