From Cli to Dockercompose

1. Output of caddy version:

v2.5.2 h1:eCJdLyEyAGzuQTa5Mh3gETnYWDClo1LjtQm2q9RNZrs=

2. How I run Caddy:

Caddy is in front of a Phoenix app run in a container. I want to serve a part of the static files: if all the static files are located at “priv/static”, I want to serve the subfolder “priv/static/react”.

When I run Phoenix in a container under compose (+ db) and Caddy outside of the network with caddy run Caddyfile, it works: the app renders.

If I integrate in the docker-compose, ie run Caddy in a container in the same network, it fails: “page not working” ,502 from server.

Phoenix runs at port 4000 and I put Caddy at 80.

a. System environment:

MacOS 12.4, Docker 20.10.17, 1.29

b. Command:

caddy run Caddyfile
docker-compose up

c. Service/unit/compose file:


caddy:
    image: caddy:2.5.2
    restart: unless-stopped
    ports:
      - "80:80"
      # - "443:443"
      # - "443:443/udp"
    volumes:
      - $PWD/Caddyfile:/etc/caddy/Caddyfile:ro
      - $PWD/priv/static/react/:/srv
      - caddy_data:/data
      - caddy_config:/config

d. My complete Caddy config:

http://localhost:80 {
    reverse_proxy  http://localhost:4000

    encode gzip
    file_server /react/* 

    log {
        output stdout
        format console
    }
}

3. The problem I’m having:

4. Error messages and/or full log output:

caddy_1  | {"level":"info","ts":1662508513.5174167,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
phx_1    | 23:55:14.608 [info] Access PhoenixReactWeb.Endpoint at http://localhost:4000
caddy_1  | {"level":"debug","ts":1662508539.5636141,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"localhost:4000","total_upstreams":1}
caddy_1  | {"level":"debug","ts":1662508539.5705636,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"localhost:4000","duration":0.006736875,"request":{"remote_ip":"172.22.0.1","remote_port":"56894","proto":"HTTP/1.1","method":"GET","host":"localhost","uri":"/","headers":{"Sec-Ch-Ua-Platform":["\"macOS\""],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"],"Sec-Fetch-Dest":["document"],"X-Forwarded-Proto":["http"],"Dnt":["1"],"Sec-Ch-Ua":["\"Google Chrome\";v=\"105\", \"Not)A;Brand\";v=\"8\", \"Chromium\";v=\"105\""],"Sec-Fetch-Site":["none"],"Accept-Encoding":["gzip, deflate, br"],"Upgrade-Insecure-Requests":["1"],"X-Forwarded-Host":["localhost"],"Accept-Language":["en"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Sec-Fetch-User":["?1"],"Sec-Fetch-Mode":["navigate"],"X-Forwarded-For":["172.22.0.1"],"Sec-Ch-Ua-Mobile":["?0"]}},"error":"dial tcp 127.0.0.1:4000: connect: connection refused"}

5. What I already tried:

I imagine a network discovery problem, Phoenix configuration. I used things like “http://caddy” or “http://phoenix” instead of localhost without success.

=> I posted here because I wanted to check that at least my Caddyfile and my Dockerfile are ok, Caddy wise; good for doing what I want, serving and caching some static files in front of Phoenix.

6. Links to relevant resources:

https://hub.docker.com/_/caddy
https://caddyserver.com/docs/quick-starts/static-files
https://caddyserver.com/docs/quick-starts/reverse-proxy

Inside of a container, localhost means “this current container”. You need to use the name of the other container you want to connect to in reverse_proxy, which will use Docker’s built-in DNS resolver to get the IP address of the other container.

Are you sure your pheonix app is in the same network? You didn’t post your full docker-compose.yml it looks like.

1 Like

oh how stupid, et voila, service name is “phoenix” WITH the port !.. Sorry & thks

http://localhost:80 {
    reverse_proxy  http://phoenix:4000

The rest of the docker-compose, almost nothing…

services:
  phoenix:
    build: .
    ports:
      - "4000:4000"
    env_file:
      - .env-docker

Just one point: how are you sure it’s Caddy that serves these files and not the webserver? The status code 304?

You’ll need to be more specific. I don’t know what you’re talking about. Show us logs, a request with curl -v, etc.

You don’t need this btw, because the connection happens within the docker network, so you don’t need to bind this container to a port on your host machine.

I had to add it in fact because I have a redirection from an API to locahost:4000, so if I don’t bind to the host, this does work. I don’t know how to get rid of this at the moment.

My question was how to make sure that the file is served by Caddy and not the webserver?

Again – please elaborate. Show us logs, example requests, etc. It’s not clear what you’re talking about.

Assets under the route “/react”, supposed to be served by Caddy

phx_1    | 22:11:06.894 request_id=FxKzlXHcKaz0dxYAAAMh [info] GET /react
phx_1    | 22:11:06.896 request_id=FxKzlXHcKaz0dxYAAAMh [info] Sent 200 in 1ms
caddy_1  | 1.6625886668976767e+09	info	http.log.access.log0	handled request	{"request": {"remote_ip": "172.27.0.1", "remote_port": "61180", "proto": "HTTP/1.1", "method": "GET", "host": "localhost", "uri": "/react", "headers": {"Sec-Fetch-Mode": ["navigate"], "Sec-Ch-Ua-Platform": ["\"Android\""], "Upgrade-Insecure-Requests": ["1"], "User-Agent": ["Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Mobile Safari/537.36"], "Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"], "Sec-Fetch-Site": ["same-origin"], "Connection": ["keep-alive"], "Cookie": [], "Sec-Ch-Ua": ["\"Google Chrome\";v=\"105\", \"Not)A;Brand\";v=\"8\", \"Chromium\";v=\"105\""], "Sec-Fetch-User": ["?1"], "Sec-Fetch-Dest": ["document"], "Referer": ["http://localhost/"], "Accept-Encoding": ["gzip, deflate, br"], "Accept-Language": ["en"], "Sec-Ch-Ua-Mobile": ["?1"], "Dnt": ["1"]}}, "user_id": "", "duration": 0.0047525, "size": 407, "status": 200, "resp_headers": {"Cache-Control": ["max-age=0, private, must-revalidate"], "Cross-Origin-Window-Policy": ["deny"], "Date": ["Wed, 07 Sep 2022 22:11:06 GMT"], "X-Download-Options": ["noopen"], "X-Content-Type-Options": ["nosniff"], "X-Permitted-Cross-Domain-Policies": ["none"], "Content-Encoding": ["gzip"], "Server": ["Caddy", "Cowboy"], "X-Request-Id": ["FxKzlXHcKaz0dxYAAAMh"], "Content-Type": ["text/html; charset=utf-8"], "Vary": ["Accept-Encoding"], "X-Xss-Protection": ["1; mode=block"], "X-Frame-Options": ["SAMEORIGIN"]}}

caddy_1  | 1.6625886669180763e+09	info	http.log.access.log0	handled request	{"request": {"remote_ip": "172.27.0.1", "remote_port": "61176", "proto": "HTTP/1.1", "method": "GET", "host": "localhost", "uri": "/react/assets/index.3fce1f81.css", "headers": {"Sec-Ch-Ua": ["\"Google Chrome\";v=\"105\", \"Not)A;Brand\";v=\"8\", \"Chromium\";v=\"105\""], "Accept": ["text/css,*/*;q=0.1"], "Sec-Fetch-Site": ["same-origin"], "Cookie": [], "If-None-Match": ["\"657A68E\""], "Connection": ["keep-alive"], "User-Agent": ["Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Mobile Safari/537.36"], "Sec-Ch-Ua-Mobile": ["?1"], "Sec-Fetch-Mode": ["no-cors"], "Referer": ["http://localhost/react"], "Accept-Encoding": ["gzip, deflate, br"], "Accept-Language": ["en"], "Dnt": ["1"], "Sec-Ch-Ua-Platform": ["\"Android\""], "Sec-Fetch-Dest": ["style"]}}, "user_id": "", "duration": 0.002887333, "size": 0, "status": 304, "resp_headers": {"Cache-Control": ["public, max-age = 31_536_00"], "Date": ["Wed, 07 Sep 2022 22:11:06 GMT"], "Etag": ["\"657A68E\""], "Server": ["Caddy", "Cowboy"], "Vary": ["Accept-Encoding"]}}

Detail of one file supposed to be served by Caddy

Assets not under "/react, supposed to be served by Cowboy only (Phoenix webserver)

Detail of one file supposed to be served by Cowboy only

If I docker exec -it <cont-id> sh into the Caddy container, I see the static files I am supposed to have copied. OK

So if I look at the logs, I believe Caddy serves them because I find all the URIs of the static files (with some patience but yes).
If I look at the time to render the assets between the first set and the second, I have 0ms (disk cache) for the supposed “non Caddy” whilst 10ms for assets supposed to be served by Caddy.

If I remove Caady and let Cowboy serve everything, I have for the “non /react” routes:

and the detail of one file

phx_1  | 21:57:39.932 [info] Running PhoenixReactWeb.Endpoint with cowboy 2.9.0 at :::4000 (http)
phx_1  | 21:57:40.020 [info] Access PhoenixReactWeb.Endpoint at http://phx
phx_1  | 21:57:49.526 request_id=FxKy28r-5PnIaEgAAAJE [info] GET /
phx_1  | 21:57:49.533 request_id=FxKy28r-5PnIaEgAAAJE [info] Sent 200 in 6ms
phx_1  | 21:57:52.219 request_id=FxKy3GuLKhWzJjUAAAAC [info] GET /react
phx_1  | 21:57:52.222 request_id=FxKy3GuLKhWzJjUAAAAC [info] Sent 200 in 3ms
phx_1  | 21:57:57.200 [info] CONNECTED TO PhoenixReactWeb.UserSocket in 5ms

It seems that if Caddy is serving them, it is slower. I don’t see a real advantage in terms of speed. That is why I wondering if they were really severed by Caddy because nothing in the browser shows me who serves the files ( it’s written Caddy / Cowboy in both cases). I hope this is more clear

Ah okay, there’s a problem with your config.

Directives in Caddy are sorted according to a predetermined directive order:

In your config, you have both reverse_proxy and file_server, but reverse_proxy always gets sorted first, and it’s a terminal handler so nothing ever falls past it.

So to fix it, I recommend using handle blocks to set up mutually exclusive routing. It would look like this:

:80 {
	log
	encode gzip

	handle /react/* {
		root * /srv
		file_server
	}

	handle {
		reverse_proxy phoenix:4000
	}
}
1 Like

ok, this makes sense, so indeed it didn’t work. Seems to work now, and websockets out of the box too. The only point is that I don’t see anymore the cascade of log output for each single static file but I image it is ok given what I see in the browser
https://caddyserver.com/docs/caddyfile/directives/handle#examples

Thks!

No redirection 304 but status 200 so better

only Caddy responds, so I imagine it is served by Caddy.