AWS S3 Bucket - Reverse Proxy

Help
1

1. Output of caddy version:

v2.6.2 h1:wKoFIxpmOJLGl3QXoo6PNbYvGW4xLEgo32GPBEjWL8o=

2. How I run Caddy:

a. System environment:

Ubuntu 22.04

b. Command:

sudo systemctl start caddy

c. Service/unit/compose file:

Service
[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target

[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

d. My complete Caddy config:

# The Caddyfile is an easy way to configure your Caddy web server.
#
# Unless the file starts with a global options block, the first
# uncommented line is always the address of your site.
#
# To use your own domain name (with automatic HTTPS), first make
# sure your domain's A/AAAA DNS records are properly pointed to
# this machine's public IP, then replace ":80" below with your
# domain name.

{
    on_demand_tls {
        interval 10m
        burst 10
    }
}

https:// {
    tls {
        on_demand
    }
    reverse_proxy {
        to http://bucketname.s3-website-us-east-1.amazonaws.com
    }
}

*.example.com {
    tls {
        dns route53 {
            access_key_id "AKIAzzzREPLACEME"
            secret_access_key "2GdiDzzzREPLACEME"
            max_retries 10
        }
    }
    reverse_proxy {
        to http://bucketname.s3-website-us-east-1.amazonaws.com
    }
}

3. The problem I’m having:

I have an S3 bucket configured as a static website, I can access it well through the URL provided by aws like as http://bucketname.s3-website-us-east-1.amazonaws.com or even through an address custom as app.example.com which is an “alias” on route53 for this bucket endpoint.

But when using Caddy as a reverse proxy for this target, I only get 404 NoSuchBucket error message.

If I change the target to any other site it works fine, but not for the bucket I need.

4. Error messages and/or full log output:

404 Not Found
Code: NoSuchBucket
Message: The specified bucket does not exist
BucketName: bucket_name
RequestId: T4BSHK2Dzzzzz
HostId: ZpAiCzdDvg3HVmE0ZRVdJ2Xzzzzzzzz=

5. What I already tried:

Tried using lindenlab/caddy-s3-proxy plugin

1- Access website Download Caddy
2- Switch platform to Linux amd64
3- Select caddy-dns/route53 and lindenlab/caddy-s3-proxy plugins
4- Right click on the Download button and copy this address (something like https://caddyserver.com/api/download?os=windows&arch=amd64&p=github.com%2Fcaddy-dns%2Froute53&p=github. com%2Flindenlab%2Fcaddy-s3-proxy&idempotency=35019229369139)

5- SSH into Ubuntu and run the command:
sudo systemctl stop caddy

6- Run the command
sudo wget -O /usr/bin/caddy “https://caddyserver.com/api/download?os=windows&arch=amd64&p=github.com%2Fcaddy-dns%2Froute53&p=github.com%2Flindenlab%2Fcaddy-s3-proxy&idempotency =35019229369139”

But I always get the error:
Resolving caddyserver.com (caddyserver.com)… 165.227.20.207, 2604:a880:2:d0::21b0:6001
Connecting to caddyserver.com (caddyserver.com)|165.227.20.207|:443… connected.
HTTP request sent, awaiting response…

And then the caddy no longer starts returning that your bin is invalid.

If I just repeat the process selecting only the route53 plugin everything works fine (but then I don’t have the S3 plugin installed and so I get the 404 error)

6. Links to relevant resources:

Community Topics

2

This topic was automatically closed after 30 days. New replies are no longer allowed.

3

Hi,
I managed to solve my problem, I share below what I did in case someone needs it too:

Note: I disregarded using the S3-proxy plugin, but I still use the Route53 plugin to generate wildcard certificates (this is reflected in my Caddyfile below).

1- I edited my Caddyfile from:

# The Caddyfile is an easy way to configure your Caddy web server.
#
# Unless the file starts with a global options block, the first
# uncommented line is always the address of your site.
#
# To use your own domain name (with automatic HTTPS), first make
# sure your domain's A/AAAA DNS records are properly pointed to
# this machine's public IP, then replace ":80" below with your
# domain name.

{
    on_demand_tls {
        interval 10m
        burst 10
    }
}

https:// {
    tls {
        on_demand
    }
    reverse_proxy {
        to http://bucketname.s3-website-us-east-1.amazonaws.com
    }
}

*.example.com {
    tls {
        dns route53 {
            access_key_id "AKIAzzzREPLACEME"
            secret_access_key "2GdiDzzzREPLACEME"
            max_retries 10
        }
    }
    reverse_proxy {
        to http://bucketname.s3-website-us-east-1.amazonaws.com
    }
}

2- To:

# The Caddyfile is an easy way to configure your Caddy web server.
#
# Unless the file starts with a global options block, the first
# uncommented line is always the address of your site.
#
# To use your own domain name (with automatic HTTPS), first make
# sure your domain's A/AAAA DNS records are properly pointed to
# this machine's public IP, then replace ":80" below with your
# domain name.

{
    on_demand_tls {
        interval 10m
        burst 10
    }
}

https:// {
    tls {
        on_demand
    }
    reverse_proxy {
        to http://bucketname.s3-website-us-east-1.amazonaws.com
        header_up Host {upstream_hostport}
    }
}

*.example.com {
    tls {
        dns route53 {
            access_key_id "AKIAzzzREPLACEME"
            secret_access_key "2GdiDzzzREPLACEME"
            max_retries 10
        }
    }
    reverse_proxy {
        to http://bucketname.s3-website-us-east-1.amazonaws.com
        header_up Host {upstream_hostport}
    }
}

So in summary, the “tip” was to add this line:

header_up Host {upstream_hostport}

And after that my bucket no longer returned the 404 error and started working :slight_smile: