Supabase storage proxying

Help
1

1. Caddy version (caddy version):

2.5.1

2. How I run Caddy:

a. System environment:

Docker with caddy:alpine image, using Traefik to route. Cloudflare for DNS (as well as Cloudflare SSL for SaaS for 3rd party domain names)

c. Service/unit/compose file:

  sites:
    image: 'caddy:alpine'
    deploy:
      labels:
        - traefik.enable=true
        - traefik.http.routers.sites-router.rule=PathPrefix(`/`)
        - traefik.http.routers.sites-router.priority=1
        - traefik.http.services.sites-service.loadbalancer.server.port=5555
        - traefik.http.routers.sites-router.middlewares=traefik-real-ip@file
      placement:
        constraints:
          - node.role == manager
    volumes:
      - /root/caddy/Caddyfile:/etc/caddy/Caddyfile
      - /root/caddy/config:/config
      - /root/caddy/data:/data

d. My complete Caddyfile or JSON config:

:5555, :443, :80 {
        encode gzip
        reverse_proxy https://[projectId].supabase.co
        rewrite * /storage/v1/object/public/sites/{host}{uri}
        log {
                output stdout
        }
}

3. The problem I’m having:

I’m trying to proxy a Supabase storage bucket through Traefik and Caddy. The setup provided above causes a long load time, then a 504 Gateway Timeout via Cloudflare.

I can see the requests going through Traefik, however I cannot see anything coming from Caddy.

4. Error messages and/or full log output:

{"level":"info","ts":1654493509.6411018,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"warn","ts":1654493509.641957,"msg":"Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":7}
{"level":"info","ts":1654493509.642667,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1654493509.6427753,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1654493509.642831,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000406690"}
{"level":"warn","ts":1654493509.6614385,"logger":"pki.ca.local","msg":"installing root certificate (you might be prompted for password)","path":"storage:pki/authorities/local/root.crt"}
{"level":"info","ts":1654493509.661951,"msg":"Warning: \"certutil\" is not available, install \"certutil\" with \"apt install libnss3-tools\" or \"yum install nss-tools\" and try again"}
{"level":"info","ts":1654493509.661962,"msg":"define JAVA_HOME environment variable to use the Java trust"}
{"level":"info","ts":1654493509.701497,"msg":"certificate installed properly in linux trusts"}
{"level":"debug","ts":1654493509.7018898,"logger":"http","msg":"starting server loop","address":"[::]:5555","http3":false,"tls":true}
{"level":"debug","ts":1654493509.7020652,"logger":"http","msg":"starting server loop","address":"[::]:80","http3":false,"tls":false}
{"level":"info","ts":1654493509.70208,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["localhost"]}
{"level":"warn","ts":1654493509.7025268,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [localhost]: no OCSP server specified in certificate","identifiers":["localhost"]}
{"level":"debug","ts":1654493509.7025504,"logger":"tls.cache","msg":"added certificate to cache","subjects":["localhost"],"expiration":1654528631,"managed":true,"issuer_key":"local","hash":"2c772d259453ec64edd5e5b16ee2981d4551d318e37f38c9a77bfca6ac991d34","cache_size":1,"cache_capacity":10000}
{"level":"info","ts":1654493509.7027164,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
{"level":"info","ts":1654493509.702759,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1654493509.702785,"msg":"serving initial configuration"}
{"level":"info","ts":1654493509.703011,"logger":"tls","msg":"finished cleaning storage units"}

5. What I already tried:

  • Simply using Traefik to route the requests; either I did it wrong, or it simply didn’t work.
  • Looked at another post here for proxying Google Cloud Buckets through Caddy

6. Links to relevant resources:

Supabase Storage Docs

2

Hi :wave:

I am not familiar with supabase, but perhaps I can provide some ideas anyway :slight_smile:

  1. Can you reach the caddy container from within traefik? You might have to join both docker-compose stacks into a common network, depending on how you set up traefik.
  2. Did you try to send requests to caddy directly? For example, via curl? Either from your host after exposing port :5555 or directly from within the caddy container (after running apk add curl)?
    curl http://example.com:5555 --resolve example.com:5555:127.0.0.1
## or
curl http://localhost:5555 -H "Host: example.com"

PS: You could also just use GitHub - lucaslorentz/caddy-docker-proxy: Caddy as a reverse proxy for Docker instead of traefik :^)

2 Likes
3

I could use that proxy you linked, however I’m implementing this into an existing environment so it’s not exactly feasible. Thanks for the suggestion though!

I just tried sending requests to caddy directly through the traefik container, didn’t seem to work (connection refused). It could be something with how Traefik does ports with Docker, but I’m not exactly sure. I’ll try forwarding the ports too.

4

Ah I tried the IP of the container in the Docker network, now it just loads forever. Caddy still doesn’t seem to see the requests.

5

Okay, this sounds very much like traefik and caddy aren’t in the same docker network.

You can check that for yourself by comparing the networks of both your containers via

docker inspect --format='{{range $k,$v := .NetworkSettings.Networks}}{{println $k}}{{end}}' your_container

## or if you have `jq` installed and want *all* network details, you could just
docker inspect --format "{{json .NetworkSettings.Networks}}" your_container | jq

## or if you are already familiar what you need to look for
docker inspect --format "{{.NetworkSettings.Networks}}" your_container

I can write a full explanation and tutorial if needed, just let me know.
Considering your system already has some docker-compose stacks that are being proxied by traefik, I would recommend looking at them first after reading Networking in Compose | Docker Documentation.

The really short form would be:

Your traefik stack might have something like

networks:
  proxy:
    name: proxy

and your other stack might need

networks:
  proxy:
    external: true

and for both traefik and the container you want to proxy, you’ll need to add networks:

services:
  sites:
    image: 'caddy:alpine'
    networks:
      - proxy

This way they share the same docker network proxy and can reach each other :slight_smile:

1 Like
6

Yup, just as I was about to respond saying I found that, you said it haha.

I also switched to just hosting the static sites on a server, as the sites were very tiny.

However, I’m now encountering an issue (no errors?) with using a dynamic file_server root.

Here’s my current Caddyfile:

{
        debug
}

http://localhost:2022 {
        encode gzip
        root * /sites/{host}
        file_server {
                index index.html
        }
        log {
                output stdout
        }
}

Requests do go through and are logged, except all that is shown is a blank page, so I think it’s something with the file_server directive.

If you can help me with that in this topic, that would be awesome. If not, I can make a new one for relevancy.

Thanks!

Edit: the /sites folder is populated with a folder the same as the hostname I’m testing with.

1 Like
7

Glad you learned a thing or too :slight_smile:

Ye, I would recommend opening another topic.
Some tips anyway :woman_shrugging:

2 Likes
8

the way it you set it up will only allow you to test /sites/localhost (because http://localhost:2022 will only catch localhost). You could try http://:2022 with the commands from Supabase storage proxying - #2 by IndeedNotJames to test others :innocent:

Well that was a simple fix haha. Thanks!

1 Like
9

This topic was automatically closed after 30 days. New replies are no longer allowed.