Templates: Do not include if file doesn't exist

1. Caddy version (caddy version):

v2.3.0 h1:fnrqJLa3G5vfxcxmOH/+kJOcunPLhSBnjgIvjXV/QTA=

2. How I run Caddy:

From docker-compose using the official images on docker hub

a. System environment:

  • Ubuntu 20.04.1 LTS bare metal server.
  • Docker version 20.10.1, build 831ebea
  • docker-compose version 1.25.0, build unknown

b. Command:

From a docker-compose perspective: docker-compose start caddy ; docker launches caddy as caddy --run --config /etc/caddy/Caddyfile --adapter caddyfile

c. Service/unit/compose file:

docker-compose.yml:

  caddy:
    container_name: "caddy"
    image: caddy:2
    restart: unless-stopped
    environment:
      - TZ=Americas/Toronto
    ports:
      - target: 80
        published: 81
        mode: host
      - target: 443
        published: 444
        mode: host
    networks:
      - caddy
   volumes:
      - /srv/docker/caddy/Caddyfile:/etc/caddy/Caddyfile
      - /srv/docker/caddy/site:/srv
      - /srv/docker/caddy/caddy_data:/data
      - /srv/docker/caddy/caddy_config:/config

Note: i’m using unusual external ports as (other) production servers uses 80/443. I’m setting up and testing caddy using temporary ports to set up things.

d. My complete Caddyfile or JSON config:

{
         debug
}

http://md.bilange.duckdns.org {
        handle_errors {
                @404 {
                        expression {http.error.status_code} == 404
                }
                handle @404 {
                        rewrite * /html/404.html
                        file_server
                }
        }

        respond /favicon.ico 404 {
                body "404"
                close
        }

        import common
        root * /srv/md.bilange.duckdns.org
        templates
        encode gzip

        file_server * browse {
                hide .git *.ini *.sh *.swp
        }

        @markdown {
                not path /html/*
                path_regexp md /.*\.md$
        }
        rewrite @markdown /html/template.html

        @dir {
                not path /html/*
                path_regexp dir (.*)/$
        }
        redir @dir {http.regexp.dir.1}/index.md
}

html/template.html (minimal example):

{{$pathParts := splitList "/" .OriginalReq.URL.Path}}
{{$markdownFilename := default "index" (slice $pathParts 2 | join "/")}}
{{$markdownFilePath := printf "/%s.md" $markdownFilename}}        
        
{{$markdownFile := (include .OriginalReq.URL.Path | splitFrontMatter)}}

{{$title := default $markdownFilename $markdownFile.Meta.title}}
{{$body := $markdownFile.Body}}
{{/*not shown: multiples searches and replaces on $body before outputting, should be irrevelant for this issue at this point */}}

<!DOCTYPE html>
<html>
        <head>
           <!-- skipped for brevity, if needed I can provide them -->
        </head>
        <body>
                <div class="container">
		        <div class="file-header">
		                <span class="header-title">
		                        <span class="octicon octicon-file"></span> {{$markdownFilename}}
		                </span>
		        </div>
		        <div class="markdown-body">
		                {{markdown $body}}
		        </div>
                </div>
        </body>
</html>

3. The problem I’m having:

This config snippet above is 99% working, except in one situation: when attempting to access a markdown file that doesn’t exist on disk, caddy will still pass control to the template file, and the `{{include…}}`` line will fail with this error log (below). So I need a solution to either catch them from the caddyfile or from the template, which I am not sure how to proceed.

4. Error messages and/or full log output:

Client-side:

>> curl -i 'http://md.bilange.duckdns.org:81/notfound.md'
HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Length: 4892
Content-Type: text/html; charset=utf-8
Etag: "qokwxu3rw"
Last-Modified: Mon, 15 Feb 2021 16:39:30 GMT
Server: Microsoft-IIS/10.0
Date: Mon, 15 Feb 2021 16:43:47 GMT

curl: (18) transfer closed with 4892 bytes remaining to read

Server-side (caddy logs):

{"level":"debug","ts":1613407559.1065104,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"66.187.114.18:58508","proto":"HTTP/1.1","method":"GET","host":"md.bilange.duckdns.org:81","uri":"/notfound.md","headers":{"User-Agent":["curl/7.68.0"],"Accept":["*/*"]}},"method":"GET","uri":"/html/template.html"}
{"level":"debug","ts":1613407559.1067774,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/srv/md.bilange.duckdns.org","request_path":"/html/template.html","result":"/srv/md.bilange.duckdns.org/html/template.html"}
{"level":"debug","ts":1613407559.106876,"logger":"http.handlers.file_server","msg":"opening file","filename":"/srv/md.bilange.duckdns.org/html/template.html"}
{"level":"error","ts":1613407559.1133502,"logger":"http.log.error","msg":"template: /html/template.html:16:21: executing \"/html/template.html\" at <include .OriginalReq.URL.Path>: error calling include: open /srv/md.bilange.duckdns.org/notfound.md: no such file or directory","request":{"remote_addr":"66.187.114.18:58508","proto":"HTTP/1.1","method":"GET","host":"md.bilange.duckdns.org:81","uri":"/notfound.md","headers":{"User-Agent":["curl/7.68.0"],"Accept":["*/*"]}},"duration":0.006944929,"status":500,"err_id":"atem08igx","err_trace":"templates.(*Templates).executeTemplate (templates.go:315)"}

5. What I already tried:

On the template side, I havent seen any functions either from caddy or sprig that permits me to check for files existence. Trying to pass {{if (os.Stat($markdownFilename))}} says that os is undefined (on afterthought, I am not sure opening up golang that much is really secure anyway :-). I think it’s possible from Caddyfile to check for file existence (try_files?), but using try_files on an existing file sends the raw markdown to the client, without using the configured template.

6. Links to relevant resources:

  • The first lines of the template is actually from this thread. The goal of my project is: I wanted to have a small HTTP server serving markdown files as converted HTML files.

7. Notes

  • If you need to try the server mentioned in the config, IIRC I believe it’s firewalled so only a few static IPs can connect. Let me know if you actually need to test it and it doesn’t respond.
  • I’ve been starting using Caddy very recently (~a week ago), prior knowledge of Golang is minimal AND rusty to say the least :slight_smile: Do not hesitate to kindly point me to the right direction if I missed some piece of documentation I should have read.

Thanks for your assistance!

You can’t write Go code or use Caddyfile directives in templates. Template actions are documented here: Modules - Caddy Documentation

Additionally, the latest HEAD has a fileExists template action you can use:

(it’s not released yet, so you’ll have to download build artifacts or build from source).

You could probably use that before including a file you’re not sure exists.

Wow, thank you for the fast response!

I’m glad support for that is coming up soon, I will definitely tinker with source if time permits.

After re-reading for the Nth time my initial post, an idea popped up: since the HTTP response seems to be abuptly ended (from curl’s point of view), is there a way to catch that on the fly (maybe for example redirecting to an HTTP 500) from the configuration? I am under the impression that the HTTP response headers has been already been sent out at the moment of trying to render the template, but who knows, maybe you have ideas for later for that as well :slight_smile:

No, templates are buffered until they are fully rendered precisely because we don’t know if they will error or not.

So yeah, ideally your template would return an HTTP error that Caddy could then handle with its error handling config, or return a default error response to the browser. Unfortunately, an open bug in the Go standard library is blocking this.

See commit:

https://github.com/caddyserver/caddy/commit/14f50d9dfb6c0da0da636829175a37b1864d23cf

As you can see, the godoc comment says an example might be: {{if not (fileExists $includeFile)}}{{httpError 404}}{{end}}.

Relevant bug:

https://github.com/golang/go/issues/34201

Should be coming in Go 1.17 though.

This topic was automatically closed after 30 days. New replies are no longer allowed.