Issue with coordinating reverse proxy between three services

[coloradocolby (Colby Thomas)]

1. Caddy version:

v2.2.1

2. How I run Caddy:

a. System environment:

Mac OS Catalina 10.15.5

b. Command:

caddy start -config Caddyfile.development

c. Service/unit/compose file:

...
  hasura:
    image: hasura/graphql-engine:v1.3.0
    ports:
      - 8080:8080
    restart: always
    depends_on:
      - postgres
    volumes:
      - ./hasura/migrations:/hasura-migrations
      - ./hasura/metadata:/hasura-metadata
    env_file:
      - ./.env.development
...

d. My complete Caddyfile or JSON config:

localhost {
  reverse_proxy /api* localhost:8000
  reverse_proxy * localhost:3000
}

3. The problem I’m having:

I have an api running on localhost:8000, a React app on localhost:3000, and a Hasura GraphQL Engine on localhost:8080.

Ideally all I want from caddy is SSL on my app, but it’d be great to simplify these ports and make everything run seamlessly.

With my current Caddyfile, the server and react app run great. I’ve got HTTPS, and instead of sending my request to :8000/api/... I simply send them to /api/.... The issue I have is I’d like to also proxy requests to my graphql engine on 8080, but that’s where I’m falling short. I send both HTTPS and WSS request to 8080 through the :8080/v1/graphql route. Additionally, Hasura provides it’s own UI on http://localhost:8080 which I’d like to also be able to enter.

4. What I already tried:

I tried adding the following to the top of my Caddyfile (shown above):

localhost:8080 {
  reverse_proxy * localhost:8080
}

But caddy tells me port 8080 is in use. That makes sense because Hasura is running on that port, but isn’t that the job of a reverse proxy – to route requests to active ports among servers?

Is it possible for me to only proxy requests sent on port 443?

Also - beginner networking question: If I hit the following url https://example.com:9000, am I hitting port 443 AND 9000?

[francislavoie]

Just change - 8080:8080 in your docker-compose to something like - 8081:8080 so that the service is bound to a different port on your host machine, then Caddy will be able to listen on port 8080 and proxy to 8081 (which actually maps to 8080 inside of the docker network). Or just have Caddy serve on a different port number altogether for that service.

Also - beginner networking question: If I hit the following url https://example.com:9000, am I hitting port 443 AND 9000?

No, only 9000. Port 443 is the default when you specify https:// by convention, but when you specify :9000, you’re overriding that default. The fact you specify https:// though means that the browser/client will attempt to establish a TLS connection.

Is it possible for me to only proxy requests sent on port 443?

Yeah, just use https://localhost or localhost:443 or :443 as your site address.

[coloradocolby (Colby Thomas)]

@francislavoie thanks for your reply! I was able to get this working in my local environment, but when I deploy nothing is pulling up. As far as I can tell everything I’m doing should be fine though? Does anything in this docker-compose / Caddyfile look off to you?

version: '3.7'
services:
  caddy:
    image: caddy:latest
    restart: unless-stopped
    ports:
      - 80:80
      - 443:443
    volumes:
      - $PWD/Caddyfile.production:/etc/caddy/Caddyfile
      - $PWD/site:/srv
      - caddy_data:/data
      - caddy_config:/config

  postgres:
    image: postgres:12
    ports:
      - 5432:5432
    restart: unless-stopped
    volumes:
      - skrrrt_data:/var/lib/postgresql/data
    env_file:
      - ./.env.production

  hasura:
    image: hasura/graphql-engine:v1.3.0
    ports:
      - 8081:8080
    restart: unless-stopped
    depends_on:
      - postgres
    volumes:
      - ./hasura/migrations:/hasura-migrations
      - ./hasura/metadata:/hasura-metadata
    env_file:
      - ./.env.production

  app:
    build: ./
    ports:
      - 8001:8000
    depends_on:
      - postgres
      - hasura
    restart: unless-stopped
    env_file:
      - ./.env.production

volumes:
  skrrrt_data:
  caddy_data:
  caddy_config:

skrrrt.io:8080 {
  reverse_proxy * hasura:8081
}

skrrrt.io  {
  reverse_proxy * app:8001
}

[francislavoie]

    ports:
      - 80:80
      - 443:443

You’re only making Caddy accessible on ports 80 and 443, so requests to 8080 won’t reach it.

Also, since you’re running everything in Docker, you can stop binding those other services to the host machine, i.e. replace:

    ports:
      - 8081:8080

with

    expose:
      - 8080

And the same with your other services. This makes these ports only accessible from other services inside the docker network, which is what you want.

Caddy needs to be pointed to the internal docker ports, not the ones exposed to the host machine. So hasura:8081 will not have worked because you’re using the port bound on the host. Instead, use hasura:8080.

You didn’t specify that you were running Caddy in docker in your original post, leading to confusion. I assumed you were running it on the host machine.

[coloradocolby (Colby Thomas)]

Thank you so much for your help! Everything is working great. And yeah I didn’t mention docker initially because I was still troubleshooting locally, however you taught me a few things about docker in the process! Cheers!