Internal url shortener with analytics

[lughino (Luca Pau)]

1. The problem I’m having:

Hi there,

I’m very new to caddy, so apologise in advance if I ask stupid questions.
I’m writing an internal url shortener that will be used for intranet only use.
Looking at caddy’s functionalities, this seems very trivial.
However, I found multiple ways to achieve the same thing, like for example:

{
	"apps": {
		"http": {
			"servers": {
				"srv0": {
					"listen": [
						":80"
					],
					"routes": [
						{
							"match": [
								{
									"path": [
										"/qwerty"
									]
								}
							],
							"handle": [
								{
									"@id": "qwerty",
									"handler": "static_response",
									"headers": {
										"Location": [
											"https://google.com"
										]
									},
									"status_code": 302
								},
								{
									"handler": "metrics",
									"disable_openmetrics": false
								}
							]
						}
					]
				}
			}
		}
	}
}

or

{
	"apps": {
		"http": {
			"servers": {
				"srv0": {
					"listen": [
						":443",
						":80"
					],
					"routes": [
						{
							"match": [
								{
									"host": [
										"foo.local"
									]
								}
							],
							"handle": [
								{
									"handler": "subroute",
									"routes": [
										{
											"handle": [
												{
													"destinations": [
														"{redirect-uri}"
													],
													"handler": "map",
													"mappings": [
														{
															"input": "/tube",
															"outputs": [
																"https://youtube.com"
															]
														},
														{
															"input": "/photos",
															"outputs": [
																"https://google.com"
															]
														},
														{
															"input": "/rickrolled",
															"outputs": [
																"https://www.youtube.com/watch?v=dQw4w9WgXcQ"
															]
														}
													],
													"source": "{http.request.uri.path}"
												}
											]
										},
										{
											"handle": [
												{
													"handler": "static_response",
													"headers": {
														"Location": [
															"{redirect-uri}"
														]
													},
													"status_code": 302
												}
											],
											"match": [
												{
													"expression": "{redirect-uri} != \"\""
												}
											]
										},
										{
											"handle": [
												{
													"body": "Thas's an unknown short URL ... :(",
													"handler": "static_response"
												}
											]
										}
									]
								}
							],
							"terminal": true
						}
					]
				}
			}
		}
	}
}

From here I got the redirect working in both ways, but this raised me several questions:

1. Which one is the most appropriate way, considering that these short urls are dynamic and updated from another service constantly?
2. How can I build a “catch-all” handler that redirect to a specific page if users provide an unknown path?
3. I need to extract several metrics like unique users per link, remote ip, various data inferred from user-agent, etc., how can I achieve that? Can I send an event from caddy to a MQ instance or to a custom server that perform the aggregation of these metrics? If yes, how can I do that? Or what is the right way to accomplish that?

2. Error messages and/or full log output:

no errors

3. Caddy version:

2.7.6

4. How I installed and ran Caddy:

For now with brew

a. System environment:

mac os

[francislavoie]

I’m confused, your two configs do totally different things.

Your second one looks pretty good. Using a map handler makes sense for managing a list of path rules like that.

@lughino

					"listen": [
						":443",
						":80"
					],

You shouldn’t have a server with both :80 and :443 at the same time; Caddy can’t serve HTTP and HTTPS from the same server. You should only use :443 if you want HTTPS, and Caddy’s Automatic HTTPS routine will add the :80 server for you.

@lughino

How can I build a “catch-all” handler that redirect to a specific page if users provide an unknown path?

You’re already doing that, aren’t you? You have a matcher checking whether the map has a result and only redirecting if it does, falling back to a static response otherwise. Just change that last part to redirect somewhere else instead then

@lughino

I need to extract several metrics like unique users per link, remote ip, various data inferred from user-agent, etc., how can I achieve that? Can I send an event from caddy to a MQ instance or to a custom server that perform the aggregation of these metrics? If yes, how can I do that? Or what is the right way to accomplish that?

You can use the reverse_proxy handler pointing to your metrics server (HTTP), change the method to GET to skip consuming the request body, then use a handle_response matching on status 200 to “do nothing” and continue through to the next handler (to perform the redirect etc). You could put this after your map so that your map var is set, you can add the new target to an HTTP header or something on the proxy handler.

This is essentially the same pattern as the forward_auth directive in the Caddyfile, if you’re confused by what I mean, you can write a Caddyfile with that directive then adapt it to JSON with caddy adapt -p to give you a starting point.

[lughino (Luca Pau)]

Thanks!
Yeah, the two configs aren’t matching at all. I was playing to get the concept right.

You’re already doing that, aren’t you? You have a matcher checking whether the map has a result and only redirecting if it does, falling back to a static response otherwise. Just change that last part to redirect somewhere else instead then

Yes, but it seems not to work. When I try to link to a route that is not in the map, I get a blank page instead.

You can use the reverse_proxy handler pointing to your metrics server (HTTP), change the method to GET to skip consuming the request body, then use a handle_response matching on status 200 to “do nothing” and continue through to the next handler (to perform the redirect etc). You could put this after your map so that your map var is set, you can add the new target to an HTTP header or something on the proxy handler.

Perfect, this is exactly what I need, thanks!

Regarding the two configurations, considering that the map solution seems to be the right direction, do you have any tips on how to dynamically modify that map? Like I have to add/modify/delete some elements on that map, how can I target via API only the specific piece without rewriting the whole configuration all the time?

[francislavoie]

@lughino

"expression": "{redirect-uri} != \"\""

Ah, the problem is that the placeholder value is actually null, not empty string. So you can do != null.

Alternatively you can set defaults: [""] in the map handler to have it default to "" instead of null.

@lughino

Regarding the two configurations, considering that the map solution seems to be the right direction, do you have any tips on how to dynamically modify that map? Like I have to add/modify/delete some elements on that map, how can I target via API only the specific piece without rewriting the whole configuration all the time?

You can use PUT to append to the map array, and DELETE to remove them.

You can use Etag to ensure you safely make changes without clobbering changes from another process.