Setup handling templates in proper way

Help
1

1. Caddy version (caddy version):

./caddy version
v2.4.6 h1:HGkGICFGvyrodcqOOclHKfvJC0qTU7vny/7FhYp9hNw=

./caddy list-modules
admin.api.load
admin.api.metrics
admin.api.reverse_proxy
caddy.adapters.caddyfile
caddy.config_loaders.http
caddy.listeners.tls
caddy.logging.encoders.console
caddy.logging.encoders.filter
caddy.logging.encoders.filter.delete
caddy.logging.encoders.filter.ip_mask
caddy.logging.encoders.filter.replace
caddy.logging.encoders.json
caddy.logging.encoders.single_field
caddy.logging.writers.discard
caddy.logging.writers.file
caddy.logging.writers.net
caddy.logging.writers.stderr
caddy.logging.writers.stdout
caddy.storage.file_system
http
http.authentication.hashes.bcrypt
http.authentication.hashes.scrypt
http.authentication.providers.http_basic
http.encoders.gzip
http.encoders.zstd
http.handlers.acme_server
http.handlers.authentication
http.handlers.encode
http.handlers.error
http.handlers.file_server
http.handlers.headers
http.handlers.map
http.handlers.metrics
http.handlers.push
http.handlers.request_body
http.handlers.reverse_proxy
http.handlers.rewrite
http.handlers.static_response
http.handlers.subroute
http.handlers.templates
http.handlers.vars
http.matchers.expression
http.matchers.file
http.matchers.header
http.matchers.header_regexp
http.matchers.host
http.matchers.method
http.matchers.not
http.matchers.path
http.matchers.path_regexp
http.matchers.protocol
http.matchers.query
http.matchers.remote_ip
http.matchers.vars
http.matchers.vars_regexp
http.precompressed.br
http.precompressed.gzip
http.precompressed.zstd
http.reverse_proxy.selection_policies.cookie
http.reverse_proxy.selection_policies.first
http.reverse_proxy.selection_policies.header
http.reverse_proxy.selection_policies.ip_hash
http.reverse_proxy.selection_policies.least_conn
http.reverse_proxy.selection_policies.random
http.reverse_proxy.selection_policies.random_choose
http.reverse_proxy.selection_policies.round_robin
http.reverse_proxy.selection_policies.uri_hash
http.reverse_proxy.transport.fastcgi
http.reverse_proxy.transport.http
pki
tls
tls.certificates.automate
tls.certificates.load_files
tls.certificates.load_folders
tls.certificates.load_pem
tls.certificates.load_storage
tls.handshake_match.remote_ip
tls.handshake_match.sni
tls.issuance.acme
tls.issuance.internal
tls.issuance.zerossl
tls.stek.distributed
tls.stek.standard

  Standard modules: 83

exec
http.authentication.providers.authorizer
http.handlers.authenticator
http.handlers.exec
http.handlers.rate_limit
http.handlers.realip
http.handlers.s3proxy
security

  Non-standard modules: 8

  Unknown modules: 0

2. How I run Caddy:

./caddy run

a. System environment:

lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.4 LTS
Release:	20.04
Codename:	focal

b. Command:

# initial config
curl localhost:8081/config/ -H "Content-Type: application/json" -d @hello-world.json
# template config
curl localhost:8081/config/apps/http/servers/example2 -H "Content-Type: application/json" -d @second-http-server.json

# normal user call.
curl -v localhost:2016/print-headers.tmpl

c. Service/unit/compose file:

# no used

d. My complete Caddyfile or JSON config:

Caddyfile

{
	debug
	admin 127.0.0.1:8081
}

hello-world.json

{
    "admin": {
        "listen": "127.0.0.1:8081"
    },
	"apps": {
		"http": {
            "http_port":8081,
            "https_port": 8443,
            "grace_period": "30m",
			"servers": {
				"example": {
					"listen": [":2015"],
					"routes": [
						{
							"handle": [{
                                "@id":"stat-resp",
								"handler": "static_response",
								"body": "I can do hard things."
							}]
						}
					]
				}
			}
		}
	},
    "logging": {
        "logs": {
          "default": {
            "level": "DEBUG"
          }
        }
      }    
}

second-http-server.json

{
    "listen": [":2016"],
    "routes": [
      {
        "handle": [
            {"handler": "encode"},
            {"handler": "templates"},
            {"handler": "file_server"}
        ]
      }
    ]
  }

print-headers.tmpl

X-Forwarded-For:   {{.Req.Header "X-Forwarded-For"}}
X-Forwarded-Host:  {{.Req.Header "X-Forwarded-Host"}}
X-Forwarded-Port:  {{.Req.Header "X-Forwarded-Port"}}
X-Forwarded-Proto: {{.Req.Header "X-Forwarded-Proto"}}
Forwarded: {{.Req.Header "Forwarded"}}

UA: {{.Req.Header.Get "User-Agent"}}

All Headers:

{{range $field, $val := .Req.Header}}
    {{$field}}: {{$val}}
{{end}}

3. The problem I’m having:

I face the problem that the template handler is not used and the response is the plain template file without any substitution.

curl -v localhost:2016/print-headers.tmpl
*   Trying 127.0.0.1:2016...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 2016 (#0)
> GET /print-headers.tmpl HTTP/1.1
> Host: localhost:2016
> User-Agent: curl/7.68.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Content-Length: 379
< Etag: "rathjcaj"
< Last-Modified: Sun, 24 Apr 2022 00:04:24 GMT
< Server: Caddy
< Date: Sun, 24 Apr 2022 00:04:36 GMT
< 
X-Forwarded-For:   {{.Req.Header.Get "X-Forwarded-For"}}
X-Forwarded-Host:  {{.Req.Header.Get "X-Forwarded-Host"}}
X-Forwarded-Port:  {{.Req.Header.Get "X-Forwarded-Port"}}
X-Forwarded-Proto: {{.Req.Header.Get "X-Forwarded-Proto"}}
Forwarded: {{.Req.Header.Get "Forwarded"}}

UA: {{.Req.Header.Get "User-Agent"}}

All Headers:

{{range $field, $val := .Req.Header}}
    {{$field}}: {{$val}}
{{end}}
* Connection #0 to host localhost left intact

4. Error messages and/or full log output:

./caddy run
2022/04/24 00:02:26.909	INFO	using adjacent Caddyfile
2022/04/24 00:02:26.910	INFO	admin	admin endpoint started	{"address": "tcp/127.0.0.1:8081", "enforce_origin": false, "origins": ["localhost:8081", "[::1]:8081", "127.0.0.1:8081"]}
2022/04/24 00:02:26.910	INFO	autosaved config (load with --resume flag)	{"file": "/home/alex/.config/caddy/autosave.json"}
2022/04/24 00:02:26.910	INFO	serving initial configuration

# configure caddy via api
2022/04/24 00:02:31.727	INFO	admin.api	received request	{"method": "POST", "host": "localhost:8081", "uri": "/config/", "remote_addr": "127.0.0.1:45944", "headers": {"Accept":["*/*"],"Content-Length":["546"],"Content-Type":["application/json"],"User-Agent":["curl/7.68.0"]}}
2022/04/24 00:02:31.728	INFO	admin	admin endpoint started	{"address": "tcp/127.0.0.1:8081", "enforce_origin": false, "origins": ["localhost:8081", "[::1]:8081", "127.0.0.1:8081"]}
2022/04/24 00:02:31.728	INFO	tls.cache.maintenance	started background certificate maintenance	{"cache": "0xc000405ab0"}
2022/04/24 00:02:31.728	DEBUG	http	starting server loop	{"address": "[::]:2015", "http3": false, "tls": false}
2022/04/24 00:02:31.728	INFO	tls	cleaning storage unit	{"description": "FileStorage:/home/alex/.local/share/caddy"}
2022/04/24 00:02:31.728	INFO	tls	finished cleaning storage units
2022/04/24 00:02:31.728	INFO	autosaved config (load with --resume flag)	{"file": "/home/alex/.config/caddy/autosave.json"}
2022/04/24 00:02:31.730	INFO	admin	stopped previous server	{"address": "tcp/127.0.0.1:8081"}
2022/04/24 00:02:37.503	INFO	admin.api	received request	{"method": "POST", "host": "localhost:8081", "uri": "/config/apps/http/servers/example2", "remote_addr": "127.0.0.1:45946", "headers": {"Accept":["*/*"],"Content-Length":["199"],"Content-Type":["application/json"],"User-Agent":["curl/7.68.0"]}}
2022/04/24 00:02:37.503	INFO	admin	admin endpoint started	{"address": "tcp/127.0.0.1:8081", "enforce_origin": false, "origins": ["localhost:8081", "[::1]:8081", "127.0.0.1:8081"]}
2022/04/24 00:02:37.503	INFO	tls.cache.maintenance	started background certificate maintenance	{"cache": "0xc0004084d0"}
2022/04/24 00:02:37.503	DEBUG	http	starting server loop	{"address": "[::]:2016", "http3": false, "tls": false}
2022/04/24 00:02:37.504	DEBUG	http	starting server loop	{"address": "[::]:2015", "http3": false, "tls": false}
2022/04/24 00:02:37.506	INFO	tls.cache.maintenance	stopped background certificate maintenance	{"cache": "0xc000405ab0"}
2022/04/24 00:02:37.506	INFO	autosaved config (load with --resume flag)	{"file": "/home/alex/.config/caddy/autosave.json"}
2022/04/24 00:02:37.507	INFO	admin	stopped previous server	{"address": "tcp/127.0.0.1:8081"}
2022/04/24 00:02:44.991	INFO	admin.api	received request	{"method": "GET", "host": "localhost:8081", "uri": "/config/", "remote_addr": "127.0.0.1:45948", "headers": {"Accept":["*/*"],"User-Agent":["curl/7.68.0"]}}

# made the normal user request
2022/04/24 00:03:34.776	DEBUG	http.handlers.file_server	sanitized path join	{"site_root": ".", "request_path": "/print-headers.tmpl", "result": "print-headers.tmpl"}
2022/04/24 00:03:34.776	DEBUG	http.handlers.file_server	opening file	{"filename": "print-headers.tmpl"}
2022/04/24 00:04:36.159	DEBUG	http.handlers.file_server	sanitized path join	{"site_root": ".", "request_path": "/print-headers.tmpl", "result": "print-headers.tmpl"}
2022/04/24 00:04:36.159	DEBUG	http.handlers.file_server	opening file	{"filename": "print-headers.tmpl"}

5. What I already tried:

I followed the doc and the recommendation for the handler order.
this is the current caddy config.

curl -s localhost:8081/config/|jq
{
  "admin": {
    "listen": "127.0.0.1:8081"
  },
  "apps": {
    "http": {
      "grace_period": "30m",
      "http_port": 8081,
      "https_port": 8443,
      "servers": {
        "example": {
          "listen": [
            ":2015"
          ],
          "routes": [
            {
              "handle": [
                {
                  "@id": "stat-resp",
                  "body": "I can do hard things.",
                  "handler": "static_response"
                }
              ]
            }
          ]
        },
        "example2": {
          "listen": [
            ":2016"
          ],
          "routes": [
            {
              "handle": [
                {
                  "handler": "encode"
                },
                {
                  "handler": "templates"
                },
                {
                  "handler": "file_server"
                }
              ]
            }
          ]
        }
      }
    }
  },
  "logging": {
    "logs": {
      "default": {
        "level": "DEBUG"
      }
    }
  }
}

6. Links to relevant resources:

2

I think the problem is that there’s no Content-Type attached to the response written by file_server, so templates ignores it – by default, it only processes responses with text/plain, text/markdown, and text/html.

It’s probably because you used .tmpl as your file extension, and that doesn’t match to any extension in mailcap, so it remains empty. If you use .txt or .html or .md, it should work.

1 Like
3

Thanks @francislavoie for your help, that was it. :slightly_smiling_face: :+1:

curl -v localhost:2016/print-headers-tmpl.txt
*   Trying 127.0.0.1:2016...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 2016 (#0)
> GET /print-headers-tmpl.txt HTTP/1.1
> Host: localhost:2016
> User-Agent: curl/7.68.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Length: 175
< Content-Type: text/plain; charset=utf-8
< Server: Caddy
< Date: Sun, 24 Apr 2022 08:05:19 GMT
< 
X-Forwarded-For:   
X-Forwarded-Host:  
X-Forwarded-Port:  
X-Forwarded-Proto: 
Forwarded: 

UA: curl/7.68.0

All Headers:


    Accept: [*/*]

    User-Agent: [curl/7.68.0]

* Connection #0 to host localhost left intact

You are pointed me to the right path as we can see in the code.

I tried now to set '*" to mime_types and got the following message.

curl localhost:8081/config/apps/http/servers/example2 -H "Content-Type: application/json" -d @second-http-server.json 
{"error":"loading new config: loading http app module: provision http: server example2: setting up route handlers: route 0: loading handler modules: position 1: loading module 'templates': decoding module config: http.handlers.templates: json: cannot unmarshal string into Go struct field Templates.mime_types of type []string"}

{
    "listen": [":2016"],
    "routes": [
      {
        "handle": [
            {"handler": "encode"},
            {
                "handler": "templates",
                "mime_types": "*"
            },
            {"handler": "file_server"}
        ]
      }
    ]
  }

I haven’t see any other possibility to set mime types in caddy file_server or in caddy them self.

What’s now my mistake?

4

The mime_types field takes an array of strings, not a single string value.

Using "*" isn’t supported here though; the code just does a strings.Contains() to see if one of the configured mime types is found in the Content-Type header.

Maybe using "" i.e. empty string might work though because empty string is always contained in all strings.

But the defaults for mime types are probably fine, just use a known file extension.

5

Cool thanks for your patience.
I have now a setup which works but for this I have to switched from sysadmin brain to development brain :smiley:

The snipplet looks now like this

{
    "listen": [":2016"],
    "routes": [
      {
        "match": [
            {
                "path":["*.tmpl"]
            }
        ],
        "handle": [
            {"handler": "encode"},
            {"handler": "templates"},
            {
                "handler": "headers",
                "response": {
                    "set": {
                        "Content-Type": ["text/plain; charset=utf-8"]
                    }
                }
            },
            {"handler": "file_server"}
        ]
      }
    ]
  }

As in JSON Config Structure - Caddy Documentation documented are the handlers like middleware in different go routers implementation. I have “just” to understand what this implies.

For the archive and the other community members let me explain how this config works now.

Now the snipplet sets’s in the “way back” the Content-Type because the suffix tmpl have no mime type mapping.
Because of that recognizes the template handler that it should be executed and e voila the substitution will be done. :heart_eyes:

Here now the full caddy json config requested via curl -s localhost:8081/config/|jq

{
  "admin": {
    "listen": "127.0.0.1:8081"
  },
  "apps": {
    "http": {
      "grace_period": "30m",
      "http_port": 8081,
      "https_port": 8443,
      "servers": {
        "example": {
          "listen": [
            ":2015"
          ],
          "routes": [
            {
              "handle": [
                {
                  "@id": "stat-resp",
                  "body": "I can do hard things.",
                  "handler": "static_response"
                }
              ]
            }
          ]
        },
        "example2": {
          "listen": [
            ":2016"
          ],
          "routes": [
            {
              "handle": [
                {
                  "handler": "encode"
                },
                {
                  "handler": "templates"
                },
                {
                  "handler": "headers",
                  "response": {
                    "set": {
                      "Content-Type": [
                        "text/plain; charset=utf-8"
                      ]
                    }
                  }
                },
                {
                  "handler": "file_server"
                }
              ],
              "match": [
                {
                  "path": [
                    "*.tmpl"
                  ]
                }
              ]
            }
          ]
        }
      }
    }
  },
  "logging": {
    "logs": {
      "default": {
        "level": "DEBUG"
      }
    }
  }
}

Thank yo @francislavoie for your tips and answers.

1 Like
6

This topic was automatically closed after 30 days. New replies are no longer allowed.