I’m trying to set up a Caddy proxy for various S3-compatible object storage services (Contabo, AWS S3, Storj). My current setup works fine for public buckets, but I’m struggling to authenticate access to private buckets. I want to authenticate privately with Caddy and use Caddy’s forward auth to handle my authentication logic from an external service. I need guidance on modifying my Caddyfile for this purpose.
2. Error messages and/or full log output:
No specific error messages yet, as I'm unsure how to proceed with the configuration for private bucket access.
I’m working on a project where I need to serve Cloud Optimized GeoTIFF (COG) files, which are TIFF files capable of self-serving from an S3 bucket, as well as Parquet files. My goal is to use Caddy to protect these files with Forward Authentication. This means only specific users can access specific files within my ecosystem.
I need to find a way to allow access to these files through a query parameter, enabling access for no-auth clients and browsers. An example URL is as follows:
?api_key=123456789 at the end of the url >> easy auth for non auth options clients
I want users to access these files similar to how it’s done on this COG viewer.
In your auth app, read the X-Forwarded-Uri header to see the original URI, you can parse it to grab the API key and check if it’s valid for that request path. If so, return a 200 status. If not, return a 401 or 403.
I’m currently exploring how to extend Caddy’s reverse proxy capabilities to handle forward_auth for private S3 buckets. While I have experience using Caddy’s forward_auth for other APIs, this specific task presents a new challenge. If an existing auth module doesn’t fit this purpose, I’m considering developing a custom one in Go. However, I would prefer to keep the solution within the Caddy ecosystem. I’m familiar with the extending Caddy documentation but require more practical examples or workshops to guide me.
I’m seeking advice from the community on integrating custom authentication logic into Caddy’s reverse proxy. Are there any existing modules or plugins I can look at for reference? Also, detailed examples or workshops related to extending Caddy, especially for authentication purposes, would be incredibly helpful. Any insights or resources would be greatly appreciated.
It seems there’s been a misunderstanding. I understand how to set up forward_auth. My issue is different. For example:
storage.example.com {
forward_auth :9089 {
uri /forward_auth_endpoint
}
reverse_proxy https://private-bucket-cog.s3.eu-central-1.amazonaws.com {
#here is the issue i am facing with how can i enter:
Access key ID secret
Secret access key secret
}
}
The S3 authentication mechanism described in their docs requires signature calculation based on the access and secret keys. If you have a service to do the calculation and prepare them, a call to the service can be ordered in the handler chain. The other approach is to create a custom middleware/handler module that calculates the header value and injects it in a var, then use that var value in header_up.
My issue has been resolved thanks to significant help from @Mohammed90 . I used the module caddy-aws-transport he developed to access the private S3 buckets.
