Altering request URL header placeholders via plugin

1. The problem I’m having:

Hi all.

I’m trying to create simple plugin to perform string functions such as lower/upper case on placeholders. The use case is other plugins which accept templated keys such as Souin’s cache plugin, where I want to ensure it’s all lowercase. The problem I’m having is I’m unsure why certain internal placeholders are not available in my plugin. For example:

{http.request.host} and {http.request.uri.path} will work fine with my plugin, but {http.request.uri.query} and {http.request.header.x-custom-header} won’t. I have confirmed that without my .lower suffix, there is a value.

I went through Placeholder Support — Caddy Documentation and noted the need to use the request context Replacer to evaluate the placeholders, however, it’s unclear why it doesn’t work for all - but I assume some kind of timing issue?

This is the primary function which gets a map of all placeholders, identifies those with the correct suffix, and applies the needed strings function:

func (m *CaddyStrings) ServeHTTP(w http.ResponseWriter, r *http.Request, next caddyhttp.Handler) error {
	repl := r.Context().Value(caddy.ReplacerCtxKey).(*caddy.Replacer)
	
	mapStringCases(repl)

	return next.ServeHTTP(w, r)
}

and then:

func mapStringCases(repl *caddy.Replacer) {
	repl.Map(func(key string) (any, bool) {
		base := key
		lower := false
		upper := false

		if strings.HasSuffix(key, ".lower") {
			base = strings.TrimSuffix(key, ".lower")
			lower = true
		} else if strings.HasSuffix(key, ".upper") {
			base = strings.TrimSuffix(key, ".upper")
			upper = true
		}

		if lower || upper {
			val, found := repl.GetString(base)
			if !found {
				return nil, false
			}
			if lower {
				val = strings.ToLower(val)
			} else if upper {
				val = strings.ToUpper(val)
			}
			return val, true
		}

		return nil, false
	})
}

I initialize the directive via:

func init() {
	caddy.RegisterModule(CaddyStrings{})
	httpcaddyfile.RegisterHandlerDirective("strings", parseCaddyfile)
	httpcaddyfile.RegisterDirectiveOrder("strings", "before", "redir")
}

2. Error messages and/or full log output:

N/A

3. Caddy version:

v2.10.2

4. How I installed and ran Caddy:

/usr/bin/xcaddy build --with github.com/darkweak/souin/plugins/caddy --with github.com/darkweak/storages/badger/caddy --with github.com/BlueBox-WorldWide/caddy-strings-plugin

a. System environment:

NAME=“Amazon Linux”
VERSION=“2023”
ARM64

b. Command:

Standard Caddy run

c. Service/unit/compose file:

N/A

d. My complete Caddy config:

{
	debug
	cache {
		api {
			souin
			debug
		}
		timeout {
			backend 120s
		}
		allowed_additional_status_codes 202 400 401
		cache_name Cache-Handler
		disable_surrogate_key
		disable_coalescing
		default_cache_control no-cache, no-store, must-revalidate, max-age=0
	}
	order strings before rewrite
	order cache after strings
}

:443 {
	strings
	header -Server
	header -Via

	@proxied_api_path path /api/*
	handle @proxied_api_path {
		header >Cache-Control "no-cache, no-store, must-revalidate, max-age=0"
        @souin_cache `path('/api/v1/app/*')`
        cache @souin_cache {
			key {
				template KEY-{http.request.header.x-custom-header.lower}-{http.request.uri.path.lower}-q-{http.request.uri.query.lower}
				disable_vary
				hide
			}
		}

		reverse_proxy https://example.com {
			header_up Host example.com
			header_up X-Forwarded-Host {host}
		}
	}

	reverse_proxy https://another-example.com {
		header_up Host another-example.com
		header_up X-Forwarded-Host {host}
		header_up X-Forwarded-Path {path}
	}
}

5. Links to relevant resources:

Hmm. I think your best bet is to checkout the caddy repo and add some log.Printf() tracing throughout the Replacer code to see where it goes wrong. Check the logic in Get() with the order of providers and what they return.

You can build Caddy with your checkout + your plugin like this:

xcaddy build --with github.com/caddyserver/caddy/v2=./caddy --with github.com/BlueBox-WorldWide/caddy-strings-plugin=./caddy-strings-plugin
2 Likes

Thanks for the feedback. So I did that and believe I’ve identified why this doesn’t work. In the addHTTPVarsToReplacer function certain placeholders are dynamic such as headers, query param and cookies. From the looks of it, it just checks if the prefix matches such as http.request.header.*, and then any related string will be replaced by this function, even if the header doesn’t exist. Therefore it takes priority over my custom replacer. I’d say it should probably check if the variable actually exists, but not sure of the consequences of that.

In the interim, I will need to find a way to run my replacer higher up in the chain of providers before addHTTPVarsToReplacer. Will give it some thought, and appreciate any suggestions.

Edit: Created an issue on Github. I believe that the core provider should return false if the key doesn’t exist to allow custom providers the ability to alter those placeholders: Replacer's addHTTPVarsToReplacer should return false on unset dynamic placeholder keys · Issue #7373 · caddyserver/caddy · GitHub