Caddy errors with `mkdir caddy: read-only file systemread-only file system`

1. The problem I’m having:

Caddy does not start and gives errors about being in a read-only filesystem. It was working before, but then I think I messed up the file permsions by running sudo brew services start caddy instead of brew services start caddy. I can run caddy by just using caddy run --config /usr/local/etc/caddy/Caddyfile and I get no errors.

2. Error messages and/or full log output:

{"level":"info","ts":1702417423.4799092,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc00050c880"}
Error: loading initial config: loading new config: loading http app module: provision http: getting tls app: loading tls app module: provision tls: provisioning automation policy 1: loading TLS automation management module: position 0: loading module 'internal': provision tls.issuance.internal: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir caddy: read-only file system
{"level":"warn","ts":1702417433.524765,"msg":"unable to determine directory for user configuration; falling back to current directory","error":"$HOME is not defined"}
{"level":"info","ts":1702417433.526219,"msg":"using provided configuration","config_file":"/usr/local/etc/caddy/Caddyfile","config_adapter":""}
{"level":"info","ts":1702417433.531339,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1702417433.531646,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0005f0b00"}
{"level":"info","ts":1702417433.532754,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc0005f0b00"}
Error: loading initial config: loading new config: loading http app module: provision http: getting tls app: loading tls app module: provision tls: provisioning automation policy 1: loading TLS automation management module: position 0: loading module 'internal': provision tls.issuance.internal: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir caddy: read-only file system
{"level":"warn","ts":1702417443.579307,"msg":"unable to determine directory for user configuration; falling back to current directory","error":"$HOME is not defined"}
{"level":"info","ts":1702417443.5803258,"msg":"using provided configuration","config_file":"/usr/local/etc/caddy/Caddyfile","config_adapter":""}
{"level":"info","ts":1702417443.586103,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1702417443.58641,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000229200"}
{"level":"info","ts":1702417443.587548,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc000229200"}
Error: loading initial config: loading new config: loading http app module: provision http: getting tls app: loading tls app module: provision tls: provisioning automation policy 1: loading TLS automation management module: position 0: loading module 'internal': provision tls.issuance.internal: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir caddy: read-only file system
{"level":"warn","ts":1702417453.634177,"msg":"unable to determine directory for user configuration; falling back to current directory","error":"$HOME is not defined"}
{"level":"info","ts":1702417453.63552,"msg":"using provided configuration","config_file":"/usr/local/etc/caddy/Caddyfile","config_adapter":""}
{"level":"info","ts":1702417453.64098,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]}
{"level":"info","ts":1702417453.641263,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000408b00"}
{"level":"info","ts":1702417453.642445,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc000408b00"}
Error: loading initial config: loading new config: loading http app module: provision http: getting tls app: loading tls app module: provision tls: provisioning automation policy 1: loading TLS automation management module: position 0: loading module 'internal': provision tls.issuance.internal: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir caddy: read-only file system
{"level":"warn","ts":1702417463.6884859,"msg":"unable to determine directory for user configuration; falling back to current directory","error":"$HOME is not defined"}
{"level":"info","ts":1702417463.689966,"msg":"using provided configuration","config_file":"/usr/local/etc/caddy/Caddyfile","config_adapter":""}
{"level":"info","ts":1702417463.695403,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1702417463.695755,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0004eb800"}
{"level":"info","ts":1702417463.696915,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc0004eb800"}
Error: loading initial config: loading new config: loading http app module: provision http: getting tls app: loading tls app module: provision tls: provisioning automation policy 1: loading TLS automation management module: position 0: loading module 'internal': provision tls.issuance.internal: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir caddy: read-only file system
{"level":"warn","ts":1702417473.73346,"msg":"unable to determine directory for user configuration; falling back to current directory","error":"$HOME is not defined"}
{"level":"info","ts":1702417473.734858,"msg":"using provided configuration","config_file":"/usr/local/etc/caddy/Caddyfile","config_adapter":""}
{"level":"info","ts":1702417473.740309,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1702417473.740619,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000588580"}
{"level":"info","ts":1702417473.7417102,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc000588580"}
Error: loading initial config: loading new config: loading http app module: provision http: getting tls app: loading tls app module: provision tls: provisioning automation policy 1: loading TLS automation management module: position 0: loading module 'internal': provision tls.issuance.internal: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir caddy: read-only file system
{"level":"warn","ts":1702417483.787861,"msg":"unable to determine directory for user configuration; falling back to current directory","error":"$HOME is not defined"}
{"level":"info","ts":1702417483.7895548,"msg":"using provided configuration","config_file":"/usr/local/etc/caddy/Caddyfile","config_adapter":""}
{"level":"info","ts":1702417483.794925,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1702417483.795254,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0001b6380"}
{"level":"info","ts":1702417483.796416,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc0001b6380"}
Error: loading initial config: loading new config: loading http app module: provision http: getting tls app: loading tls app module: provision tls: provisioning automation policy 1: loading TLS automation management module: position 0: loading module 'internal': provision tls.issuance.internal: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir caddy: read-only file system
{"level":"warn","ts":1702417493.843434,"msg":"unable to determine directory for user configuration; falling back to current directory","error":"$HOME is not defined"}
{"level":"info","ts":1702417493.8447201,"msg":"using provided configuration","config_file":"/usr/local/etc/caddy/Caddyfile","config_adapter":""}
{"level":"info","ts":1702417493.850249,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1702417493.850566,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000096180"}
{"level":"info","ts":1702417493.851651,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc000096180"}
Error: loading initial config: loading new config: loading http app module: provision http: getting tls app: loading tls app module: provision tls: provisioning automation policy 1: loading TLS automation management module: position 0: loading module 'internal': provision tls.issuance.internal: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir caddy: read-only file system
{"level":"warn","ts":1702418006.9935691,"msg":"unable to determine directory for user configuration; falling back to current directory","error":"$HOME is not defined"}
{"level":"info","ts":1702418006.995017,"msg":"using provided configuration","config_file":"/usr/local/etc/caddy/Caddyfile","config_adapter":""}
{"level":"warn","ts":1702418006.997278,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/usr/local/etc/caddy/Caddyfile","line":16}
{"level":"info","ts":1702418007.000478,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1702418007.000729,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000589800"}
{"level":"info","ts":1702418007.0007339,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1702418007.0008051,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"warn","ts":1702418007.00084,"logger":"http.auto_https","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv1","http_port":80}
{"level":"warn","ts":1702418007.0014741,"logger":"tls","msg":"unable to get instance ID; storage clean stamps will be incomplete","error":"open caddy/instance.uuid: no such file or directory"}
{"level":"info","ts":1702418007.001556,"logger":"http.log","msg":"server running","name":"srv1","protocols":["h1","h2","h3"]}
{"level":"info","ts":1702418007.0016289,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"error","ts":1702418007.001627,"logger":"tls","msg":"could not clean default/global storage","error":"unable to acquire storage_clean lock: creating lock file: open caddy/locks/storage_clean.lock: no such file or directory"}
{"level":"info","ts":1702418007.0017002,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1702418007.0018291,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1702418007.001847,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["ssh.awesomesheep48.ca","awesomesheep48.ca"]}
{"level":"error","ts":1702418007.002037,"msg":"unable to create folder for config autosave","dir":"caddy","error":"mkdir caddy: read-only file system"}
{"level":"info","ts":1702418007.002055,"msg":"serving initial configuration"}
{"level":"warn","ts":1702418007.002064,"msg":"$HOME environment variable is empty - please fix; some assets might be stored in ./caddy"}
{"level":"error","ts":1702418007.002202,"logger":"tls","msg":"job failed","error":"ssh.awesomesheep48.ca: obtaining certificate: failed storage check: mkdir caddy: read-only file system - storage is probably misconfigured"}
{"level":"error","ts":1702418007.00224,"logger":"tls","msg":"job failed","error":"awesomesheep48.ca: obtaining certificate: failed storage check: mkdir caddy: read-only file system - storage is probably misconfigured"}
{"level":"info","ts":1702418050.627368,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
{"level":"warn","ts":1702418050.627423,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
{"level":"info","ts":1702418050.627467,"logger":"http","msg":"servers shutting down with eternal grace period"}
{"level":"info","ts":1702418050.627681,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
{"level":"info","ts":1702418050.627726,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}

3. Caddy version:

v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A=

4. How I installed and ran Caddy:

Using macOS homebrew

a. System environment:

macOS 10.15, x86_64

b. Command:

brew services start caddy

c. Service/unit/compose file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>KeepAlive</key>
	<true/>
	<key>Label</key>
	<string>homebrew.mxcl.caddy</string>
	<key>LimitLoadToSessionType</key>
	<array>
		<string>Aqua</string>
		<string>Background</string>
		<string>LoginWindow</string>
		<string>StandardIO</string>
		<string>System</string>
	</array>
	<key>ProgramArguments</key>
	<array>
		<string>/usr/local/opt/caddy/bin/caddy</string>
		<string>run</string>
		<string>--config</string>
		<string>/usr/local/etc/caddy/Caddyfile</string>
	</array>
	<key>RunAtLoad</key>
	<true/>
	<key>StandardErrorPath</key>
	<string>/usr/local/var/log/caddy.log</string>
	<key>StandardOutPath</key>
	<string>/usr/local/var/log/caddy.log</string>
</dict>
</plist>

d. My complete Caddy config:

awesomesheep48.ca, rebeccas-imac.local:80 {
	root * /usr/local/var/www/AwesomeSheep48s-Landing
	file_server browse
	encode zstd gzip
	import snippets/global
	log {
		output file /usr/local/var/log/caddy_log
		format json
	}
}

snippets/global:

templates #for the 'handle errors' section below
uri strip_suffix .html
try_files {path}.html {path}
header {
	Strict-Transport-Security "max-age=31536000; includeSubdomains; preload"
	X-Frame-Options DENY
	X-XSS-Protection "1; mode=block"
	Referrer-Policy no-referrer
	Permissions-Policy interest-cohort=()
	X-Content-Type-Options nosniff
}
handle_errors {
	respond "{http.error.status_code} {http.error.status_text}"
}
encode zstd gzip #zstd is better, but not every browser supports it

5. Links to relevant resources:

I fixed this issue by deleting the log file, I must have forgotten to check it’s permissions. Leaving this up in case anyone else has issues.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.