Help with Setting Up Private Bucket Access via Caddy Proxy for S3-Compatible Object Storage

1. The problem I’m having:

I’m trying to set up a Caddy proxy for various S3-compatible object storage services (Contabo, AWS S3, Storj). My current setup works fine for public buckets, but I’m struggling to authenticate access to private buckets. I want to authenticate privately with Caddy and use Caddy’s forward auth to handle my authentication logic from an external service. I need guidance on modifying my Caddyfile for this purpose.

2. Error messages and/or full log output:

No specific error messages yet, as I'm unsure how to proceed with the configuration for private bucket access.

3. Caddy version:

v2.7.4 h1:J8nisjdOxnYHXlorUKXY75Gr6iBfudfoGhrJ8t7/flI=

4. How I installed and ran Caddy:

a. System environment:

Running on Linux, arm64 architecture. No specific version of systemd or Docker involved yet.

b. Command:

caddy run --config /etc/caddy/Caddyfile

c. Service/unit/compose file:

Not applicable at the moment.

d. My complete Caddy config: {
    reverse_proxy {
        header_up Host

5. Links to relevant resources:


I don’t use S3, so I can’t help with that. I don’t know how auth for S3 works either.

You’ll need to ask a more specific question about the task you want to perform with Caddy. Explain what you need Caddy to do in terms of HTTP routing.

I’m working on a project where I need to serve Cloud Optimized GeoTIFF (COG) files, which are TIFF files capable of self-serving from an S3 bucket, as well as Parquet files. My goal is to use Caddy to protect these files with Forward Authentication. This means only specific users can access specific files within my ecosystem.

I need to find a way to allow access to these files through a query parameter, enabling access for no-auth clients and browsers. An example URL is as follows:

?api_key=123456789 at the end of the url >> easy auth for non auth options clients
I want users to access these files similar to how it’s done on this COG viewer.

Okay yeah, simple enough. The forward_auth directive will do what you need.

In your auth app, read the X-Forwarded-Uri header to see the original URI, you can parse it to grab the API key and check if it’s valid for that request path. If so, return a 200 status. If not, return a 401 or 403.

That’s pretty much all you need to do.

1 Like

I’m currently exploring how to extend Caddy’s reverse proxy capabilities to handle forward_auth for private S3 buckets. While I have experience using Caddy’s forward_auth for other APIs, this specific task presents a new challenge. If an existing auth module doesn’t fit this purpose, I’m considering developing a custom one in Go. However, I would prefer to keep the solution within the Caddy ecosystem. I’m familiar with the extending Caddy documentation but require more practical examples or workshops to guide me.

I’m seeking advice from the community on integrating custom authentication logic into Caddy’s reverse proxy. Are there any existing modules or plugins I can look at for reference? Also, detailed examples or workshops related to extending Caddy, especially for authentication purposes, would be incredibly helpful. Any insights or resources would be greatly appreciated.

You don’t need any modules for this. Just write a config like this: forward_auth (Caddyfile directive) — Caddy Documentation Just swap out the authelia upstream with your own.

It seems there’s been a misunderstanding. I understand how to set up forward_auth. My issue is different. For example: {
  forward_auth :9089 {
                uri /forward_auth_endpoint
  reverse_proxy {
                #here is the issue i am facing with how can i enter:
                Access key ID secret
                Secret access key secret 

Expected url will be :

I don’t understand what you mean by this.

What are you trying to set? HTTP headers? URI rewrite?

The S3 authentication mechanism described in their docs requires signature calculation based on the access and secret keys. If you have a service to do the calculation and prepare them, a call to the service can be ordered in the handler chain. The other approach is to create a custom middleware/handler module that calculates the header value and injects it in a var, then use that var value in header_up.

1 Like

My issue has been resolved thanks to significant help from @Mohammed90 . I used the module caddy-aws-transport he developed to access the private S3 buckets.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.