Reverse proxy for a frontend to private API (k8s)

Help
1

1. Output of caddy version:

caddy version
v2.6.2 h1:wKoFIxpmOJLGl3QXoo6PNbYvGW4xLEgo32GPBEjWL8o=

2. How I run Caddy:

When testing locally I use:

caddy run --config local-caddyfile --adapter caddyfile

In the container I used


CMD ["/bin/bash", "-c", "REACT_APP_API_URI=$API_URI REACT_APP_API_VERSION=$API_VERSION REACT_APP_REVERSE_PROXY=$REVERSE_PROXY npx react-inject-env set && \
caddy run --config /etc/caddy/Caddyfile --adapter caddyfile"]

a. System environment:

Kubernetes and the local environment

b. Command:

caddy run --config local-caddyfile --adapter caddyfile

c. Service/unit/compose file:

I use Docker CMD to start Caddy and point to the configuration file.

CMD ["/bin/bash", "-c", "REACT_APP_API_URI=$API_URI REACT_APP_API_VERSION=$API_VERSION REACT_APP_REVERSE_PROXY=$REVERSE_PROXY npx react-inject-env set && \
caddy run --config /etc/caddy/Caddyfile --adapter caddyfile"]

d. My complete Caddy config:

:8080 {
	root * /app/build/
	encode gzip
	try_files {path} /index.html
	file_server
	handle_path /api/* {
		reverse_proxy {$API_URI}
	}
	log {
		output file /var/log/caddy/caddy.log {
			roll_size 1gb
            roll_keep 2
            roll_keep_for 720h
		}
		format json {
            time_format wall
        }
	}
}

3. The problem I’m having:

I have a Kubernetes cluster that is hosting a DB, API, and frontend application. I have a loadbalancer in front of the EKS cluster. I want to use a reverse proxy inside the UI container to forward all /api/* requests to the private API container using the k8s hostname. However, because the API is private, I need all API request to go through the origin server, the UI container, and hit the API which is in the k8s cluster.

I have eliminated network connectivity as a possible issue as I can curl and hit the API from inside the UI container. The issue I am having is that the API requests are not getting routed to the reverse proxy.

For context, in the UI code, by default, all requests to the API point to http://0.0.0.0:3000 where the plan is to have the reverse proxy pick up the requests to /api/* and route them to the private API hostname.

4. Error messages and/or full log output:

The UI error

requests.js:32          GET http://0.0.0.0:3000/api/v1/counter net::ERR_CONNECTION_REFUSED

The Caddy server start log output

caddy run --config local-caddyfile --adapter caddyfile
2023/01/14 01:42:02.991	INFO	using provided configuration	{"config_file": "local-caddyfile", "config_adapter": "caddyfile"}
2023/01/14 01:42:02.992	WARN	Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies	{"adapter": "caddyfile", "file": "local-caddyfile", "line": 20}
2023/01/14 01:42:03.006	INFO	admin	admin endpoint started	{"address": "localhost:2019", "enforce_origin": false, "origins": ["//localhost:2019", "//[::1]:2019", "//127.0.0.1:2019"]}
2023/01/14 01:42:03.007	INFO	tls.cache.maintenance	started background certificate maintenance	{"cache": "0x140003571f0"}
2023/01/14 01:42:03.007	INFO	tls	cleaning storage unit	{"description": "FileStorage:/Users/karlcardenas/Library/Application Support/Caddy"}
2023/01/14 01:42:03.007	INFO	http.log	server running	{"name": "srv0", "protocols": ["h1", "h2", "h3"]}
2023/01/14 01:42:03.007	INFO	autosaved config (load with --resume flag)	{"file": "/Users/karlcardenas/Library/Application Support/Caddy/autosave.json"}
2023/01/14 01:42:03.007	INFO	serving initial configuration
2023/01/14 01:42:03.009	INFO	tls	finished cleaning storage units

5. What I already tried:

I can get this concept to work if I add this block :

:3000 {
	reverse_proxy {$API_URI}
}

But now I have to expose the upstream port 3000 and that’s something I would rather avoid. Some might ask, why not use the load balancer to route the requests to the private API, which would make things easier, but in this scenario I must have the requests go through the UI container.

To test this locally, I exposed the API publicly and I am now trying to hit it by using the browser and the reverse proxy but no luck.

I have also tried the following inside the main :8080 {} block

reverse_proxy /api/* {$API_URI}

Maybe I’m applying the wrong directives and concepts? I would not rule that out so apologies if this seems silly :sweat_smile:

6. Links to relevant resources:

3

So after some thinking, I now realize that I do indeed need the following block:

:3000 {
	reverse_proxy {$API_URI}
}

Something needs to handle the http://0.0.0.0:3000 requests. My issue now is that CORS is bitting me:

Access to fetch at 'http://0.0.0.0:3000/api/v1/counter' from origin 'http://a8e858acdaf444c0483195606c59da6b-1059349501.us-east-1.elb.amazonaws.com:8080' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
4

A problem with this config is that try_files runs before handle_path, so the path will get rewritten to /index.html for any path that doesn’t map to a file on disk, which means nothing ever reaches the API proxy.

You config should probably look something like this instead:

	encode gzip

	handle_path /api/* {
		reverse_proxy {$API_URI}
	}

	handle {
		root * /app/build
		try_files {path} /index.html
		file_server
	}

This way, the API handle will be hit first, and only run if the path starts with /api/, and anything else falls through to the second handle.

5

Hi @francislavoie thank you so much for replying.

I tried the following Caddyfile configuration.

:8080 {
	encode gzip
	handle_path /api/* {
		reverse_proxy ${API_URI}
	}

	handle {
		root * /app/build
		try_files {path} /index.html
		file_server	
	}

	log {
		output file /var/log/caddy/caddy.log {
			roll_size 1gb
            roll_keep 2
            roll_keep_for 720h
		}
		format json {
            time_format wall
        }
	}

}

Unfortunately, I still get the same behavior.

export API_URI= http://hello-universe-api-svc.hello-universe-api-ns.svc.cluster.local:3000

CleanShot 2023-01-16 at 10.58.55

It doesn’t seem like the request is getting sent to the {$API_URI} address. I also tried the following configuration to see if I could force the request to go inside the container and get sent to {$API_URI}

:8080 {
        encode gzip
        handle_path /api/* {
                reverse_proxy http://0.0.0.0:3000
        }

        handle {
                root * /app/build
                try_files {path} /index.html
                file_server
        }

        log {
                output file /var/log/caddy/caddy.log {
                        roll_size 1gb
            roll_keep 2
            roll_keep_for 720h
                }
                format json {
            time_format wall
        }
        }

}
:3000 {
        reverse_proxy {$API_URI}
}

I also don’t see anything in the logs of an attempt to send the request to the API URI address.

6

That’s not the right syntax for environment variables. The brace { goes before the $. Like this: {$API_URL}.

So, 0.0.0.0 as a dial address doesn’t really make sense, because that IP address is meant to mean “all addresses”. But most machines end up routing it as meaning localhost (e.g. 127.0.0.1). I recommend only using 0.0.0.0 as a listen address, and not as a dial address.

But trying to connect to localhost inside a container will not work, because localhost means “this container”, not “this machine”.

So you need to use the name of the container you want to proxy to, if that’s what you’re trying to do. That would use Docker’s built-in DNS resolver.

7

Whoops, sorry @francislavoie that was a typo from copying and pasting {$API_URI}. But I doubled checked an tried again and with this config (see below) I still get the same behavior

:8080 {
        encode gzip
        handle_path /api/* {
                reverse_proxy {$API_URI}
        }

        handle {
                root * /app/build
                try_files {path} /index.html
                file_server
        }

        log {
                output file /var/log/caddy/caddy.log {
                        roll_size 1gb
            roll_keep 2
            roll_keep_for 720h
                }
                format json {
            time_format wall
        }
        }

}

The value of the API_URI is something along these lines:

http://hello-universe-api-svc.hello-universe-api-ns.svc.cluster.local:3000

I also don’t see anything in the logs to indicate the reverse proxy is taking effect.
@francislavoie I don’t believe I mentioned this but what happens is that when the page loads. The landing page immediately makes an API request to http://0.0.0.0:3000/api/v1/someEndpoint. The idea was to have the reverse proxy pick up this request and send it to {$API_URI}.
I can confirm the UI container can reach the API as I can remote exec into the container and do a curl with $API_URI.

So I’m not really sure as to what I am doing wrong here :frowning_face:

8

Do you have any evidence that this actually reaches Caddy? Are you always running this on your local machine?

Because like I said, 0.0.0.0 effectively means “localhost” which means “this machine”.

We don’t have enough information to go on here. We need to see your logs, etc. Turn on the debug global option, show your Caddy container’s logs. Make requests with curl -v and show what you get back, etc.

9

@francislavoie I’m adding an architecture image to explain myself better. I understand what you are saying about 0.0.0.0, and I do indeed want the request to happen in the UI container because that’s the only way to access the private API in the cluster. So what I am wanting to achieve is that when the UI has to query the API /api/v1/myEndpoint that it sends it to the API container that is listening on port 3000. I’m also doing all my testing on the actual environment to remove issues related to local environment.

The moment the landing page / is presented, a request to the API is invoked.

I can see from Caddy logs that whenever I visit the UI URL that it’s serving the requests. But nothing in the logs displays an attempt to handle the /api/* path. I’m attaching the logs you mentioned above.

CleanShot 2023-01-18 at 09.27.58
CleanShot 2023-01-18 at 09.27.58995×940 61.6 KB

curl logs:

curl -v http://a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080
*   Trying 34.200.190.177:8080...
* Connected to a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com (34.200.190.177) port 8080 (#0)
> GET / HTTP/1.1
> Host: a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080
> User-Agent: curl/7.85.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Content-Length: 726
< Content-Type: text/html; charset=utf-8
< Etag: "rola6rk6"
< Last-Modified: Mon, 16 Jan 2023 17:35:15 GMT
< Server: Caddy
< Date: Wed, 18 Jan 2023 16:34:40 GMT
<
* Connection #0 to host a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com left intact
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="/favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1"/><meta name="theme-color" content="#000000"/><meta name="description" content="A Spectro Cloud demo application. Visit https://spectrocloud.com for more information."/><link rel="apple-touch-icon" href="/logo192.png"/><link rel="manifest" href="/manifest.json"/><script src="/env.js"></script><title>Spectro Cloud</title><script defer="defer" src="/static/js/main.3f926c94.js"></script><link href="/static/css/main.2dc3ef71.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>%

Caddy logs:

{"level":"info","ts":"2023/01/18 16:34:40","logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"10.0.208.128","remote_port":"15859","proto":"HTTP/1.1","method":"GET","host":"a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080","uri":"/","headers":{"User-Agent":["curl/7.85.0"],"Accept":["*/*"]}},"user_id":"","duration":0.000517349,"size":726,"status":200,"resp_headers":{"Server":["Caddy"],"Etag":["\"rola6rk6\""],"Content-Type":["text/html; charset=utf-8"],"Last-Modified":["Mon, 16 Jan 2023 17:35:15 GMT"],"Accept-Ranges":["bytes"],"Content-Length":["726"]}}
{"level":"info","ts":"2023/01/18 16:35:29","logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"10.0.208.128","remote_port":"26780","proto":"HTTP/1.1","method":"GET","host":"a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080","uri":"/","headers":{"Connection":["keep-alive"],"Upgrade-Insecure-Requests":["1"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,en;q=0.9"]}},"user_id":"","duration":0.000876755,"size":414,"status":200,"resp_headers":{"Vary":["Accept-Encoding"],"Server":["Caddy"],"Etag":["\"rola6rk6\""],"Content-Type":["text/html; charset=utf-8"],"Last-Modified":["Mon, 16 Jan 2023 17:35:15 GMT"],"Content-Encoding":["gzip"]}}
{"level":"info","ts":"2023/01/18 16:35:30","logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"10.0.208.128","remote_port":"26780","proto":"HTTP/1.1","method":"GET","host":"a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080","uri":"/env.js","headers":{"Referer":["http://a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080/"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,en;q=0.9"],"Connection":["keep-alive"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"],"Accept":["*/*"]}},"user_id":"","duration":0.000097755,"size":186,"status":200,"resp_headers":{"Accept-Ranges":["bytes"],"Content-Length":["186"],"Server":["Caddy"],"Etag":["\"roowni56\""],"Content-Type":["text/javascript; charset=utf-8"],"Last-Modified":["Wed, 18 Jan 2023 16:33:18 GMT"]}}
{"level":"info","ts":"2023/01/18 16:35:30","logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"10.0.208.128","remote_port":"26780","proto":"HTTP/1.1","method":"GET","host":"a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080","uri":"/static/css/main.2dc3ef71.css","headers":{"Referer":["http://a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080/"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,en;q=0.9"],"Connection":["keep-alive"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"],"Accept":["text/css,*/*;q=0.1"]}},"user_id":"","duration":0.000658718,"size":705,"status":200,"resp_headers":{"Server":["Caddy"],"Etag":["\"rola6r13a\""],"Content-Type":["text/css; charset=utf-8"],"Last-Modified":["Mon, 16 Jan 2023 17:35:15 GMT"],"Content-Encoding":["gzip"],"Vary":["Accept-Encoding"]}}
{"level":"info","ts":"2023/01/18 16:35:30","logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"10.0.208.128","remote_port":"56122","proto":"HTTP/1.1","method":"GET","host":"a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080","uri":"/static/js/main.3f926c94.js","headers":{"Connection":["keep-alive"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"],"Accept":["*/*"],"Referer":["http://a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080/"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,en;q=0.9"]}},"user_id":"","duration":0.09909016,"size":96022,"status":200,"resp_headers":{"Vary":["Accept-Encoding"],"Server":["Caddy"],"Etag":["\"rola6r5fv7\""],"Content-Type":["text/javascript; charset=utf-8"],"Last-Modified":["Mon, 16 Jan 2023 17:35:15 GMT"],"Content-Encoding":["gzip"]}}
{"level":"info","ts":"2023/01/18 16:35:30","logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"10.0.208.128","remote_port":"56122","proto":"HTTP/1.1","method":"GET","host":"a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080","uri":"/static/media/logo.c051b85956a838a03746c752d6254cf0.svg","headers":{"Accept":["image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8"],"Referer":["http://a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080/"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,en;q=0.9"],"Connection":["keep-alive"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"]}},"user_id":"","duration":0.000226276,"size":354,"status":200,"resp_headers":{"Server":["Caddy"],"Etag":["\"rola6rhp\""],"Content-Type":["image/svg+xml"],"Last-Modified":["Mon, 16 Jan 2023 17:35:15 GMT"],"Content-Encoding":["gzip"],"Vary":["Accept-Encoding"]}}
{"level":"info","ts":"2023/01/18 16:35:30","logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"10.0.208.128","remote_port":"56122","proto":"HTTP/1.1","method":"GET","host":"a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080","uri":"/favicon.ico","headers":{"Connection":["keep-alive"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"],"Accept":["image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8"],"Referer":["http://a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080/"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,en;q=0.9"]}},"user_id":"","duration":0.000212003,"size":14582,"status":200,"resp_headers":{"Content-Type":["image/x-icon"],"Last-Modified":["Mon, 16 Jan 2023 17:34:27 GMT"],"Accept-Ranges":["bytes"],"Content-Length":["14582"],"Server":["Caddy"],"Etag":["\"rola5fb92\""]}}
{"level":"info","ts":"2023/01/18 16:39:27","logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"10.0.208.128","remote_port":"34545","proto":"HTTP/1.1","method":"GET","host":"a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080","uri":"/","headers":{"Upgrade-Insecure-Requests":["1"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,en;q=0.9"],"Connection":["keep-alive"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"If-None-Match":["\"rola6rk6\""],"If-Modified-Since":["Mon, 16 Jan 2023 17:35:15 GMT"],"Cache-Control":["max-age=0"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"]}},"user_id":"","duration":0.000197508,"size":0,"status":304,"resp_headers":{"Server":["Caddy"],"Etag":["\"rola6rk6\""]}}
{"level":"info","ts":"2023/01/18 16:39:27","logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"10.0.208.128","remote_port":"34545","proto":"HTTP/1.1","method":"GET","host":"a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080","uri":"/env.js","headers":{"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"],"If-None-Match":["\"roowni56\""],"Accept-Encoding":["gzip, deflate"],"If-Modified-Since":["Wed, 18 Jan 2023 16:33:18 GMT"],"Connection":["keep-alive"],"Accept":["*/*"],"Referer":["http://a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080/"],"Accept-Language":["en-US,en;q=0.9"]}},"user_id":"","duration":0.000116058,"size":0,"status":304,"resp_headers":{"Server":["Caddy"],"Etag":["\"roowni56\""]}}

Caddy server logs

{"level":"info","ts":1674059600.0216062,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"warn","ts":1674059600.0231373,"msg":"Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":10}
{"level":"info","ts":1674059600.0239792,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1674059600.0303643,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000156e00"}
{"level":"info","ts":1674059600.1193013,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1674059600.1194797,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/home/appuser/.local/share/caddy"}
{"level":"info","ts":1674059600.1195548,"msg":"autosaved config (load with --resume flag)","file":"/home/appuser/.config/caddy/autosave.json"}
{"level":"info","ts":1674059600.1195657,"msg":"serving initial configuration"}
{"level":"info","ts":1674059600.1195931,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1674059943.8195412,"logger":"admin.api","msg":"received request","method":"POST","host":"localhost:2019","uri":"/load","remote_ip":"127.0.0.1","remote_port":"38466","headers":{"Accept-Encoding":["gzip"],"Content-Length":["1120"],"Content-Type":["application/json"],"Origin":["http://localhost:2019"],"User-Agent":["Go-http-client/1.1"]}}
{"level":"info","ts":1674059943.8203928,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1674059943.820684,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000614bd0"}
{"level":"debug","ts":1674059943.8208182,"logger":"http","msg":"starting server loop","address":"[::]:8080","tls":false,"http3":false}
{"level":"info","ts":1674059943.82085,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1674059943.82095,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc000156e00"}
{"level":"info","ts":1674059943.9192686,"msg":"autosaved config (load with --resume flag)","file":"/home/appuser/.config/caddy/autosave.json"}
{"level":"info","ts":1674059943.9193323,"logger":"admin.api","msg":"load complete"}
{"level":"info","ts":1674059943.9577994,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
{"level":"debug","ts":1674059967.4957237,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_ip":"10.0.208.128","remote_port":"34545","proto":"HTTP/1.1","method":"GET","host":"a876bdf3cb30749528538d3374d5c06c-834139808.us-east-1.elb.amazonaws.com:8080","uri":"/","headers":{"Accept-Language":["en-US,en;q=0.9"],"Connection":["keep-alive"],"Upgrade-Insecure-Requests":["1"],"Accept-Encoding":["gzip, deflate"],"If-Modified-Since":["Mon, 16 Jan 2023 17:35:15 GMT"],"Cache-Control":["max-age=0"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"If-None-Match":["\"rola6rk6\""]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1674059967.4957592,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/app/build","request_path":"/index.html","result":"/app/build/index.html"}
{"level":"debug","ts":1674059967.4957747,"logger":"http.handlers.file_server","msg":"opening file","filename":"/app/build/index.html"}
{"level":"debug","ts":1674059967.5952854,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/app/build","request_path":"/env.js","result":"/app/build/env.js"}
{"level":"debug","ts":1674059967.595322,"logger":"http.handlers.file_server","msg":"opening file","filename":"/app/build/env.js"}
10

So, to be clear, is it the client’s computer that’s originating the 0.0.0.0 request, or is it actually the UI container?

I would expect the UI container to just be serving files, not calling the API.

Either way, 0.0.0.0 will never work inside of your UI container, because it’s going to try to connect to itself. If you’re trying to connect to the API container, you need to use the API container’s address (via Docker/K8s built-in DNS resolver by service name).

If it’s coming from the client computer, the client computer would have to be the same computer that’s running the k8s stack, otherwise it will just try to connect to itself, which also would not work.

11

So the browser on the client is making the request to UI for the static assets. The browser is making the attempt to hit the API. This later part is what I want to force through UI container with Caddy. I want Caddy to intercept the requests to /api/v1/ and send them to the API container inside the cluster. The reason I want to force this behavior is that the API is unreachable from the clients computer due to not being publicly exposed.

My thought was that hardcoding http://0.0.0.0:3000 in the app code would be ignored because the following Caddyfile block would intercept the request and send it to the k8s hostname value that is stored in the env variable $API_URI

:8080 {
        handle_path /api/* {
                reverse_proxy {$API_URI}
        }

}

I also flirted with the idea of having the reverse proxy block send traffic 0.0.0.0:3000 and having caddy have another block for a reverse proxy to the k8s hostname.

:8080 {
        encode gzip
        handle_path /api/* {
                reverse_proxy http://0.0.0.0:3000
        }

        handle {
                root * /app/build
                try_files {path} /index.html
                file_server
        }

}
:3000 {
        reverse_proxy {$API_URI}
}

But everything I’ve tried does not seem to create the desired effect which is to force the request to /api/* to happen inside the UI container vs the users browser.

I know this is doable, as I have done this with Nginx and Consul listening on localhost:3000. Now I understand in my current setup, I don’t have Consul listening locally on the UI container’s port 3000 but my thinking was to have the extra Caddyfile block of :3000 { reverse_proxy {$API_URI}} replace Consul.

Here is that Nginx config I used with Consul in the past to force this behavior.

server {
  listen 8080;
  server_name localhost;

  server_tokens off;

  gzip on;
  gzip_proxied any;
  gzip_comp_level 4;
  gzip_types text/css application/javascript image/svg+xml;

  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection 'upgrade';
  proxy_set_header Host $host;

  location / {
    root   /usr/share/nginx/html;
    index  index.html index.htm;
  }

  location /api {
    proxy_pass http://localhost:3000;
  }
}

Hopefully this helps :sweat_smile: . It’s very possible I am applying the wrong Caddy directives ad should be using something else. I would much rater use Caddy over Nginx :smile:

12

No. Caddy is listening on port 8080 with that config, not on port 3000. And again, your address doesn’t make sense.

You can definitely have Caddy proxy the API requests, but you need to use the same hostname and port as the “UI” uses.

This config looks perfectly fine, (if you only want to support access over HTTP):

You probably want to change your “base API URL” to something like /api instead of http://0.0.0.0:3000 or whatever you currently have it set to, so that your frontend (JS code in the browser) doesn’t attempt to connect to “localhost” of the user’s own machine.

13

@francislavoie let me ask a different question. What would the equivalent of a Caddyfile look like for the Nginx config I shared? Maybe that’s the best place to start as I can get this to work with Nginx.

14

Probably this:

:8080 {
        encode gzip
        handle_path /api/* {
                reverse_proxy api:3000
        }

        handle {
                root * /app/build
                try_files {path} /index.html
                file_server
        }
}

But like I said, it’s important that your client (browser) uses the correct hostname to make the request (i.e. the same hostname as for your frontend, to reach Caddy) and not 0.0.0.0.

15

@francislavoie I got it working. You were right about the 0.0.0.0, so I replaced it with the load balancer’s URL. Thanks again for your patience and all your help :heart:

For anyone else, here is my final configuration:

ALB_URI = https://myALBaddress:3000
SVC_URL=http://hello-universe-api-svc.hello-universe-api-ns.svc.cluster.local:3000

{
        debug
}
:8080 {
        encode gzip
        handle_path /api/* {
                reverse_proxy {$ALB_URI}
        }

        handle {
                root * /app/build
                try_files {path} /index.html
                file_server
        }
	log {
		output file /var/log/caddy/caddy.log {
			roll_size 1gb
            roll_keep 2
            roll_keep_for 720h
		}
		format json {
            time_format wall
        }
	}

}
:3000 {
	reverse_proxy /api/* {
		header_up Authorization "Bearer {$TOKEN}"
		to {$SVC_URI}
	}
	log {
		output file /var/log/caddy/caddy.log {
			roll_size 1gb
            roll_keep 2
            roll_keep_for 720h
		}
		format json {
            time_format wall
        }
	}
}
16

This topic was automatically closed after 30 days. New replies are no longer allowed.