Logging per subdomain reverse proxies

[JustinWebDev (Justin WD)]

1. The problem I’m having:

I’d like to have log files for each subdomain that is proxied. I haven’t figured out how to do this. I want the Caddy log file for the Caddy-only stuff, then subdomain logs for the subdomain-specific stuff.

I’ve tried using a logging function that creates a log file based on a passed in filename, and I tried a log statement in the server block, but neither creates any specific log file.

2. Error messages and/or full log output:

na

3. Caddy version:

2.6.4

4. How I installed and ran Caddy:

a. System environment:

Win10 Pro x64

b. Command:

caddy run

d. My complete Caddy config:

{
	email myemail@email.net
	default_sni justinsdomain.com
	servers :8443 {
		protocols h2 h1
	}
	#log stdout
	log caddy-log {
		output file caddy.log	
	}
}
(headers) {
	header {
		Strict-Transport-Security "max-age=31536000; includeSubDomains;"
    		X-Xss-Protection "1; mode=block"
    		X-Content-Type-Options "nosniff"
    		X-Frame-Options "SAMEORIGIN"
    		Content-Security-Policy "upgrade-insecure-requests"
    		Referrer-Policy "strict-origin-when-cross-origin"
    		Cache-Control "public, max-age=15, must-revalidate"
	}
}
(logging) {
	log {
		output file "{args.0}.log"
	}
}
#file server 1
serv1.justinsdomain.com:8443 {
	log {
		output file caddy-Serv1.log
	}
	reverse_proxy :8080
	encode zstd gzip
	import headers
}
#file server 2
serv2.justinsdomain.com:8443 {
	import logging Serv2
	reverse_proxy :8081
	encode zstd gzip
	import headers
}

[francislavoie]

@JustinWebDev

output file "{args.0}.log"

You didn’t specify a path to the file, so Caddy would try to write it to the current working directory. Where that is depends on how you started Caddy.

Try using a full path, and not just a filename.

[JustinWebDev (Justin WD)]

I actually want the log files in the caddy.exe folder (where the current default log is written). I tried the below and it still won’t work.

I changed the logging snippet:

log {
		output file "{args.0}.log"
	}

to

log {
		output file "d:\\BIN\\caddy\\{args.0}.log"
	}

and likewise for the server block log directive example, but all the logging still goes into the default log file.

[francislavoie]

What do you mean by “all the logging”?

The log directive only configures access logs, not all general server logs.

[JustinWebDev (Justin WD)]

Maybe I should ask this way - is logging per subdomain (server) possible? Caddy doesn’t complain about the config file. Is my config valid? All I know is it doesn’t produce anything new or different than w/o the server block directives.

[francislavoie]

@JustinWebDev

is logging per subdomain (server) possible?

Yes, absolutely.

@JustinWebDev

All I know is it doesn’t produce anything new or different than w/o the server block directives.

The Caddy process probably doesn’t have permission to write to that location. I can’t really help with that part, permissions on Windows is not something I have much experience with.

@JustinWebDev

"d:\\BIN\\caddy\\{args.0}.log"

Try without double backslashes, maybe.

[JustinWebDev (Justin WD)]

Write permissions is not the problem. (If Caddy couldn’t write, I’d hope it would show some kind of error.)

I had to escape the backslash because Caddy would error if I didn’t. Even Caddy knows to do it:
redirected default logger {"from": "stderr", "to": "D:\\BIN\\caddy\\caddy.log"}

Then it sounds like something is broken, at least on Windows. Or maybe my expectations are incorrect. What would be logged in the per server log file?

[francislavoie]

@JustinWebDev

What would be logged in the per server log file?

Only access logs, i.e. the shape of each request that your site receives. See How Logging Works — Caddy Documentation

I’m on vacation but I can test things out on my Windows machine

[JustinWebDev (Justin WD)]

I see in another post you said " {args.0} is deprecated, you should use {args[0]}". I’ll update for that.

[francislavoie]

Sorry I took so long I had this on my TODO list but never got around to it until now.

I’m testing on Windows with a config like this:

(logging) {
	log {
		output file "{args.0}.log"
	}
}

:8881 {
	respond "WELCOME"
	import logging foo
}

I run Caddy with .\caddy.exe run --config Caddyfile, then make a request with curl -v http://localhost:8881. It creates a log file foo.log in the PWD (the dir I ran the command), and it has the access log written to it.

I also tried with import logging E:\foo to write the log file to a separate disk, and that worked too.

I also added log caddy-log like you had in your original post and it also wrote a caddy.log file, with foo.log still containing access logs.

@JustinWebDev

I had to escape the backslash because Caddy would error if I didn’t.

I can’t replicate that. I use output file E:\test\foo.log and it writes there without issue as well.

FWIW, I’m on Windows 11, you said you’re on Windows 10. No idea if that makes a difference (I should hope not) but yeah

[JustinWebDev (Justin WD)]

Thanks for coming back to this.

I was using an actual app, like File Browser or Emby, to generate traffic. I’m also using an RP, which you did not use. Do you get a log with a RP?

It looks like it might be because of an external connection. I used this:

(logging) {
	log {
		output file "{args.0}.log"
	}
}
some.external.com:8443 {
	respond "I am 8081"
	import logging foo
}

:8081 {
	respond "I am 8081"
	import logging foo
}

curl -v http://localhost:8081 returns "I am 8081" and creates foo.log.
curl -v https://some.external.com:443 returns "I am 8081" but does not create foo.log. (executed from another network, not looped)

[francislavoie]

What version of Caddy are you using? Still 2.6.4? Can you try with the latest, v2.7.4?

[JustinWebDev (Justin WD)]

I downloaded 2.7.4 and tested with this config:

{
	email mygreatemail@att.net
	default_sni external.com
	log {
		output file caddy.log
		format console {
			time_local
			time_format wall
			level_format upper
		}
	}
}
(logging) {
	log {
		output file "{args.0}.log"
	}
}

some.external.com:8443 {
	respond "I am 443 EXTERNAL"
	import logging foo-ext
}

:8081 {
	respond "I am 8081 INTERNAL"
	import logging foo-int
}

local curl did produce foo-int.log
external curl did not produce foo-ext.log

A would also say that Caddy is a bit overzealous with this warning in the Caddy log 2X every second! This quickly grows the log file.
WARN caddyfile Placeholder {args.0} deprecated, use {args[0]} instead

[francislavoie]

“every second”? It should only log that when you adapt your config, i.e. when Caddy starts. I guess you must be using --watch? It tries to adapt and reload the config constantly.

@JustinWebDev

local curl did produce foo-int.log
external curl did not produce foo-ext.log

Ooooh I have a hunch now.

I think the problem has to do with the fact that your site address is some.external.com:8443 with a port number. If you adapt your config with caddy adapt -c Caddyfile -p you’ll see logger_names which includes the port number.

I think this is bug, because I think we try to match logger names by the hostname only and not the port, so we don’t find a matching logger name for that request and the access log never gets written.

I’ll play around with this a bit more but I think if you use the https_port 8443 global option for now and omit the port from site addresses, it might work as a workaround until I fix that bug.

[JustinWebDev (Justin WD)]

Yes! Using https_port 8443 worked.

And, yes, I am using --watch.

[francislavoie]

@JustinWebDev

Yes! Using https_port 8443 worked.

Good! I hope to fix that soon.

@JustinWebDev

And, yes, I am using --watch.

I recommend only having that on while you’re actively updating your config. It’s very wasteful to keep it on all the time.

[francislavoie]

Okay, got a fix

httpcaddyfile: Remove port from logger names

[JustinWebDev (Justin WD)]

Awesome - thanks!