Caddy not reading `storage s3` in Caddyfile correctly

1. The problem I’m having:

In short
I’m trying to use a s3 storage module with caddy

Example Caddyfile like

{
  storage s3 {
    bucket caddy
    region us-gov-west-1
  }
}

But I keep getting this error
Error: adapting config using caddyfile: parsing caddyfile tokens for 'storage': wrong argument count or unexpected line ending after <storage s3 line number>

2. Error messages and/or full log output:

Error: adapting config using caddyfile: parsing caddyfile tokens for 'storage': wrong argument count or unexpected line ending after <storage s3 line number>

Note: I’ve tried this with the s3 module by certmagic and it didn’t work either so I tried rolling my own and naming it sthree instead of s3 because I was worried about an issue with parsing numbers but that doesn’t seem to be the case either. My s3 module, heavily borrowed from certmagic’s, is here GitHub - owens3364/caddy-s3: Super basic dead simple S3 storage module for use with Caddy but I don’t believe that is the issue since I get the same error with both storage modules.

And my full Caddyfile is pasted below.

3. Caddy version:

Built with 2.10.2

sh-5.2$ ./caddy version
v2.10.2 h1:g/gTYjGMD0dec+UgMw8SnfmJ3I9+M2TdvoRL/Ovu6U8=

4. How I installed and ran Caddy:

a. System environment:

Linux arm64 t4g.micro AWS EC2 Instance with proper IAM roles and networking for S3 access.

b. Command:

I don’t use a run command, just a service, however
build command

# build-caddy.sh
#!/bin/bash
set -euxo pipefail

GOOS=linux GOARCH=arm64 ~/go/bin/xcaddy build \
  --with github.com/corazawaf/coraza-caddy/v2 \
  --with github.com/caddy-dns/route53@v1.6.0-beta.1 \
  --with github.com/caddyserver/cache-handler \
  --with github.com/darkweak/storages/otter/caddy \
  --with github.com/mholt/caddy-ratelimit \
  --with github.com/owens3364/caddy-s3@cbaf3df

Additional info - caddy binary lives at /usr/local/bin/caddy on the machine and Caddyfile and env vars are in /etc/caddy/Caddyfile and /etc/caddy/caddy.env, respectively.

c. Service/unit/compose file:

[Unit]
Description=Caddy Web Server
After=network.target

[Service]
EnvironmentFile=/etc/caddy/caddy.env
ExecStart=/usr/local/bin/caddy run --config /etc/caddy/Caddyfile
ExecReload=/usr/local/bin/caddy reload --config /etc/caddy/Caddyfile
Restart=always
User=root
Group=root
LimitNOFILE=1048576

[Install]
WantedBy=multi-user.target

d. My complete Caddy config:

{
	admin off
	email {$GHES_SUPPORT_EMAIL}
	order coraza_waf first
	order cache before rewrite
	log {
		output file /var/log/caddy/coraza.log
		format json
		level info
		include "http.handlers.waf"
	}
	storage sthree {
		bucket {$S3_BUCKET_NAME}
		region {$S3_BUCKET_REGION}
	}
}

(waf) {
	coraza_waf {
		load_owasp_crs
		directives `
		Include @coraza.conf-recommended
		Include @crs-setup.conf.example
		Include @owasp_crs/*.conf
		SecDefaultAction "phase:3,log,auditlog,pass"
		SecDefaultAction "phase:4,log,auditlog,pass"
		SecDefaultAction "phase:5,log,auditlog,pass"
		SecRuleEngine On
		SecAuditEngine On
		SecAuditLogParts ABIJDEFHZ
		SecAuditLog /var/log/caddy/modsec_audit.log
		SecAuditLogFormat json
		SecResponseBodyAccess On
		SecRequestBodyLimit 52428800
		SecRequestBodyNoFilesLimit 1048576
		SecRequestBodyInMemoryLimit 1048576
		SecRule REQUEST_COOKIES:_fi_sess ".*" "id:100001,phase:1,pass,nolog,ctl:ruleRemoveTargetById=942100;REQUEST_COOKIES:_fi_sess"
		SecRule GEO:COUNTRY_CODE "!@eq US" "id:100002,phase:1,deny,status:403,log,auditlog,msg:'Non-US Blocked'"
		SecRule REQUEST_URI "@beginsWith /git-upload-pack" "phase:2,nolog,pass,ctl:requestBodyLimit=262144000"
		`
	}

	handle_errors 403 {
		header X-Blocked "true"
		respond "Your request was blocked. Request ID: {http.request.uuid}"
	}
}
{$GHES_DOMAIN}, *.{$GHES_DOMAIN} {
	rate_limit {
		zone individual {
			key {remote_host}
			window 10s
			events 5000
		}
		zone overall {
			key static
			window 1m
			events 30000
		}
	}
	import waf
	tls {
		dns route53 {
			region {$GHES_DOMAIN_REGION}
		}
	}
	reverse_proxy {$GHES_PRIVATE_URL} {
		transport http {
			tls_insecure_skip_verify
		}
		header_up X-Forwarded-Proto https
	}
	handle {
		@assets host {$GHES_CACHEABLE_SUBDOMAIN}.{$GHES_DOMAIN}
		reverse_proxy {$GHES_PRIVATE_URL} {
			transport http {
				tls_insecure_skip_verify
			}
			header_up X-Forwarded-Proto https
		}
		cache {
			allowed_http_verbs GET
			otter
			ttl 24h
			timeout {
				backend 10s
				cache 10ms
			}
		}
	}
	header {
		Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
		X-Content-Type-Options "nosniff"
		X-Frame-Options "DENY"
		Referrer-Policy "no-referrer"
	}
}

Other Notes

  • I’ve validated that Caddy has all the env vars and have tried harcoding them with and without quotes.
  • I’ve tried all kinds of variations of storage s3 - putting it outside the globals, doing storage.s3, doing storage { s3 { // inner settings } } (with proper line formatting) - everything I could think of. Also tried sthree for all this instead of s3.

This seems to be a super weird error and I can’t figure out what’s causing it.

Caddy list-modules

sh-5.2$ ./caddy list-modules
admin.api.load
admin.api.metrics
admin.api.pki
admin.api.reverse_proxy
caddy.adapters.caddyfile
caddy.config_loaders.http
caddy.filesystems
caddy.listeners.http_redirect
caddy.listeners.proxy_protocol
caddy.listeners.tls
caddy.logging.cores.mock
caddy.logging.encoders.append
caddy.logging.encoders.console
caddy.logging.encoders.filter
caddy.logging.encoders.filter.cookie
caddy.logging.encoders.filter.delete
caddy.logging.encoders.filter.hash
caddy.logging.encoders.filter.ip_mask
caddy.logging.encoders.filter.query
caddy.logging.encoders.filter.regexp
caddy.logging.encoders.filter.rename
caddy.logging.encoders.filter.replace
caddy.logging.encoders.json
caddy.logging.writers.discard
caddy.logging.writers.file
caddy.logging.writers.net
caddy.logging.writers.stderr
caddy.logging.writers.stdout
caddy.network_proxy.none
caddy.network_proxy.url
caddy.storage.file_system
events
http
http.authentication.hashes.bcrypt
http.authentication.providers.http_basic
http.encoders.gzip
http.encoders.zstd
http.handlers.acme_server
http.handlers.authentication
http.handlers.copy_response
http.handlers.copy_response_headers
http.handlers.encode
http.handlers.error
http.handlers.file_server
http.handlers.headers
http.handlers.intercept
http.handlers.invoke
http.handlers.log_append
http.handlers.map
http.handlers.metrics
http.handlers.push
http.handlers.request_body
http.handlers.reverse_proxy
http.handlers.rewrite
http.handlers.static_response
http.handlers.subroute
http.handlers.templates
http.handlers.tracing
http.handlers.vars
http.ip_sources.static
http.matchers.client_ip
http.matchers.expression
http.matchers.file
http.matchers.header
http.matchers.header_regexp
http.matchers.host
http.matchers.method
http.matchers.not
http.matchers.path
http.matchers.path_regexp
http.matchers.protocol
http.matchers.query
http.matchers.remote_ip
http.matchers.tls
http.matchers.vars
http.matchers.vars_regexp
http.precompressed.br
http.precompressed.gzip
http.precompressed.zstd
http.reverse_proxy.selection_policies.client_ip_hash
http.reverse_proxy.selection_policies.cookie
http.reverse_proxy.selection_policies.first
http.reverse_proxy.selection_policies.header
http.reverse_proxy.selection_policies.ip_hash
http.reverse_proxy.selection_policies.least_conn
http.reverse_proxy.selection_policies.query
http.reverse_proxy.selection_policies.random
http.reverse_proxy.selection_policies.random_choose
http.reverse_proxy.selection_policies.round_robin
http.reverse_proxy.selection_policies.uri_hash
http.reverse_proxy.selection_policies.weighted_round_robin
http.reverse_proxy.transport.fastcgi
http.reverse_proxy.transport.http
http.reverse_proxy.upstreams.a
http.reverse_proxy.upstreams.multi
http.reverse_proxy.upstreams.srv
pki
tls
tls.ca_pool.source.file
tls.ca_pool.source.http
tls.ca_pool.source.inline
tls.ca_pool.source.pki_intermediate
tls.ca_pool.source.pki_root
tls.ca_pool.source.storage
tls.certificates.automate
tls.certificates.load_files
tls.certificates.load_folders
tls.certificates.load_pem
tls.certificates.load_storage
tls.client_auth.verifier.leaf
tls.ech.publishers.dns
tls.get_certificate.http
tls.get_certificate.tailscale
tls.handshake_match.local_ip
tls.handshake_match.remote_ip
tls.handshake_match.sni
tls.handshake_match.sni_regexp
tls.issuance.acme
tls.issuance.internal
tls.issuance.zerossl
tls.leaf_cert_loader.file
tls.leaf_cert_loader.folder
tls.leaf_cert_loader.pem
tls.leaf_cert_loader.storage
tls.permission.http
tls.stek.distributed
tls.stek.standard

  Standard modules: 127

admin.api.souin
cache
caddy.storage.sthree
dns.providers.route53
http.handlers.cache
http.handlers.rate_limit
http.handlers.waf
storages.cache.otter

  Non-standard modules: 8

  Unknown modules: 0

Lastly
If anyone wants to give feedback on my actual Caddyfile configuration I’d appreciate that. I’ve never done this before so while I believe it’s all right maybe there’s a better way.

Thanks for the assistance!

Your Caddyfile adapt code is wrong

You’re missing NextBlock() to read inside of { } braces. See Caddyfile Support — Caddy Documentation

3 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.