Reverse-Proxy and Google Storage Bucket

1. Caddy version (caddy version):

v2.3.0 h1:fnrqJLa3G5vfxcxmOH/+kJOcunPLhSBnjgIvjXV/QTA=

2. How I run Caddy:

a. System environment:

Using Docker on windows wsl2 ubuntu subsystem

b. Command:

caddy run --config Caddyfile

c. Service/unit/compose file:

Dockerfile:

FROM debian

RUN apt-get update && apt-get install -y debian-keyring debian-archive-keyring apt-transport-https curl && \
    curl -1sLf  'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | apt-key add - && \
    curl -1sLf  'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | tee -a /etc/apt/sources.list.d/caddy-stable.list && \
    apt-get update && \
    apt-get install caddy

CMD ["ping", "localhost"]

docker-compose.yml

version: '3.3'

services:
  caddy:
    build: .
    ports:
      - "2019:2019"
      - "9000:9000"
    restart: always
    volumes:
      - $PWD/.:/var/www

d. My complete Caddyfile or JSON config:

Caddyfile (mounted into container)

localhost:9000 {
        encode gzip
        reverse_proxy https://storage.googleapis.com
        rewrite * /<bucket>/<directory>/<subdirectory>{uri}
        log {
                output file /var/log/caddy/bucket.log
        }
}

3. The problem I’m having:

I want to use caddy as a reverse-proxy for static html, xml, js, css and image files in a google storage bucket. The bucket is public accesible.
If I request localhost:9000/index.html (or ommit index.html at all) I’d like to see the file https://storage.googleapis.com/<bucket>/<directory>/<subdirectory>/index.html
At the Moment I get, so some part of the proxy is working but not all. Does anybody manages it to successful reverse-proxying a google storage bucket?

<Error>
<Code>AccessDenied</Code>
<Message>Access denied.</Message>
<Details>
Anonymous caller does not have storage.objects.get access to the Google Cloud Storage object.
</Details>
</Error>

4. Error messages and/or full log output:

2021/03/05 19:59:01.812 error   http.log.access.log0    handled request {"request": {"remote_addr": "172.19.0.1:47048", "proto": "HTTP/2.0", "method": "GET", "host": "localhost:9000", "uri": "/", "headers": {"User-Agent": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0"], "Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"], "Accept-Language": ["de,en-US;q=0.7,en;q=0.3"], "Accept-Encoding": ["gzip, deflate, br"], "Dnt": ["1"], "Upgrade-Insecure-Requests": ["1"], "Cache-Control": ["max-age=0"], "Te": ["trailers"]}, "tls": {"resumed": false, "version": 772, "cipher_suite": 4865, "proto": "h2", "proto_mutual": true, "server_name": "localhost"}}, "common_log": "172.19.0.1 - - [05/Mar/2021:19:59:01 +0000] \"GET / HTTP/2.0\" 403 223", "duration": 1.2390266, "size": 223, "status": 403, "resp_headers": {"Date": ["Fri, 05 Mar 2021 19:59:01 GMT"], "Expires": ["Fri, 05 Mar 2021 19:59:01 GMT"], "Cache-Control": ["private, max-age=0"], "X-Guploader-Uploadid": ["ABg5-UyJ2I4ixCv11Ac67EcuA3Z1IyCXQHvL6v7Tia1V_fsRNZLwXlXHJmD8kC2dq-Eb1OgJTZDrJK5RLR_2EODAZg"], "Content-Type": ["application/xml; charset=UTF-8"], "Server": ["Caddy", "UploadServer"], "Content-Length": ["223"]}}
2021/03/05 19:59:01.947 info    http.log.access.log0    handled request {"request": {"remote_addr": "172.19.0.1:47048", "proto": "HTTP/2.0", "method": "GET", "host": "localhost:9000", "uri": "/favicon.ico", "headers": {"User-Agent": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0"], "Accept": ["image/webp,*/*"], "Accept-Language": ["de,en-US;q=0.7,en;q=0.3"], "Accept-Encoding": ["gzip, deflate, br"], "Dnt": ["1"], "Referer": ["https://localhost:9000/"], "Te": ["trailers"]}, "tls": {"resumed": false, "version": 772, "cipher_suite": 4865, "proto": "h2", "proto_mutual": true, "server_name": "localhost"}}, "common_log": "172.19.0.1 - - [05/Mar/2021:19:59:01 +0000] \"GET /favicon.ico HTTP/2.0\" 0 0", "duration": 0.0016936, "size": 0, "status": 0, "resp_headers": {"Server": ["Caddy"]}}

5. What I already tried:

I tried to get it working, but was not able to. I’m out of ideas, that’s why I’m asking for help here. Is the reverse-proxy the right thing here?

Why I’m trying this:
My goal ist to have multiple pages in the storage bucket in their own directorys and ship them out on different domains as well adding pages dynamically with the admin api.

Caddy has an official docker image, why not use that? Docker Hub

That said, you probably need to override the Host header when proxying, since you’re proxying to an external HTTPS server. See the example with header_up Host:

1 Like

Thank you for the hint with the docker image, I tried to get as close as possible to my server, that’s why I’m just using a base image and install caddy there. For testing purposes the official image is probably better, will use the next time :slight_smile:

With the header_up Host it works! Highly appreciate your answer!

One additional Question to the reverse_proxy and rewrite:
I want, that every request with a trailing / is going to the corresponding index.html

At the moment my Caddyfile looks like:

localhost:9000 {
	encode gzip
	reverse_proxy https://storage.googleapis.com {
	    header_up Host {http.reverse_proxy.upstream.hostport}
	}
	rewrite / /<bucket>/<directory>/<subdirectory>{uri}index.html
	rewrite * /<bucket>/<directory>/<subdirectory>{uri}
	log {
		output file /var/log/caddy/bucket.log
	}
}

With the first rewrite I get the index.html when I request localhost:9000. But how about localhost:9000/subpage/?
Using

rewrite */ /<bucket>/<directory>/<subdirectory>{uri}index.html

I get

run: adapting config using caddyfile: parsing caddyfile tokens for 'rewrite': Caddyfile:6 - Error during parsing: Wrong argument count or unexpected line ending after '/<bucket>/<directory>/<subdirectory>{http.request.uri}index.html'

What would be the right syntax here?

Last but not least, if I get any error code back and caddy is logging something like and the page is showing a storage bucket error I’d like to show instead the index.html or a dedicated 404.html.

2021/03/10 20:23:43.765 error   http.log.access.log0    handled request {"request": {"remote_addr": "172.19.0.1:39496", "proto": "HTTP/2.0", "method": "GET", "host": "localhost:9000", "uri": "/test/index.html", "headers": {"Dnt": ["1"], "Upgrade-Insecure-Requests": ["1"], "Pragma": ["no-cache"], "Cache-Control"
: ["no-cache"], "Te": ["trailers"], "User-Agent": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0"], "Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"], "Accept-Language": ["de,en-US;q=0.7,en;q=0.3"], "Accept-Encoding": ["gzip, deflate, br"]
}, "tls": {"resumed": true, "version": 772, "cipher_suite": 4865, "proto": "h2", "proto_mutual": true, "server_name": "localhost"}}, "common_log": "172.19.0.1 - - [10/Mar/2021:20:23:43 +0000] \"GET /test/index.html HTTP/2.0\" 404 216", "duration": 0.052911, "size": 216, "status": 404, "resp_headers": {"Date": [
"Wed, 10 Mar 2021 20:23:44 GMT"], "Expires": ["Wed, 10 Mar 2021 20:23:44 GMT"], "Cache-Control": ["private, max-age=0"], "Server": ["Caddy", "UploadServer"], "X-Guploader-Uploadid": ["ABg5-Ux7kDm5rNxyBBxZT1n-Aa4Lz8e2HGD_ST8_0bJRIT0kin-w2FqbKIp9ot06qGTAxVBSLI11ei_GLWzg_AAm5eM"], "Content-Type": ["application/xml
; charset=UTF-8"], "Content-Length": ["216"]}}

I tried with

handle_errors {
            respond "{http.error.status_code} {http.error.status_text}"
        }

inside the localhost:9000 {} block, but with no success. As a subdirective in reverse_proxy it does not work either. What could be the solution here?

You’ll need to use a named match for this. A matcher can only be something that starts with a @ (named matcher) with a / (path matcher) or is a * (match everything).

@isDirectory path */
rewrite @isDirectory /<bucket>/<directory>/<subdirectory>{uri}index.html

The problem is that the proxy is producing the error. The handle_errors directive only handles errors produced by other handlers (like basicauth or file_server), i.e. errors produced in Caddy itself.

What you’ll need is the WIP handle_response subdirective of reverse_proxy, which should land in v2.4.0:

If you want to try it out, you could grab a build from that PR (we have CI artifacts you can find at the bottom of the page here https://github.com/caddyserver/caddy/actions/runs/579734782), or in your current version, use JSON configuration to do the same thing (adapt your config to JSON, then modify it to use handle_response) since the feature was implemented earlier but just not available via Caddyfile: JSON Config Structure - Caddy Documentation.

2 Likes

This topic was automatically closed after 30 days. New replies are no longer allowed.