How to get ngrok working with dockerised caddy

1. Output of caddy version:

I don’t know because I don’t know how to connect to the docker image that caddy is on using caddy:latest but presumably it’s the latest version.

2. How I run Caddy:

docker-compose up

a. System environment:

Mac OS 12.6 running docker desktop on M1 ARM architecture

b. Command:

docker-compose up

c. Service/unit/compose file:

version: "3.9"
networks:
  web-network:
services:
  caddy:
    image: caddy:latest
    restart: always
    volumes:
      - ./caddy/data:/data
      - ./caddy/config:/config
      - ./caddy/Caddyfile:/etc/caddy/Caddyfile
      - ./caddy/logs:/logs
      - ./mnr:/var/www/html
    ports:
      - "80:80"
      - "443:443"
    networks:
      - web-network
    # this bit allows caddy on docker to see the listener on the external (to docker) port 8080 (which listens locally on my laptop)
    extra_hosts:
      host.docker.internal: host-gateway
  php:
    build: ./php
    tty: true
    restart: always
    volumes:
      - ./mnr:/var/www/html
    networks:
      - web-network
    extra_hosts:
      host.docker.internal: host-gateway

  mysql:
    image: mysql/mysql-server:latest-aarch64
    ports:
      - "23306:3306"
    environment:
      MYSQL_ROOT_HOST: "%"
      MYSQL_ROOT_USER: root
      MYSQL_ROOT_PASSWORD: root
      MYSQL_DATABASE: mnr_be_dev
      MYSQL_USER: mnr_dev
      MYSQL_PASSWORD: slfid9fe898
    volumes:
      - $PWD/db/data:/var/lib/mysql
    networks:
      - web-network
    extra_hosts:
      host.docker.internal: host-gateway

d. My complete Caddy config:

{
    debug
}
fe.mnr.localhost {
    reverse_proxy host.docker.internal:8080
}
be.mnr.localhost {
    root * /var/www/html/mnr-be/webroot
    encode gzip
    php_fastcgi php:9000 
    file_server
}

3. The problem I’m having:

Thanks to this post (and the short novel it contains) – How to get dockerised Caddy to use self-signed certs for local dev with php-fpm SPA (VueJs) – I was able to get a dockerised https local dev environment working.

However that app also has integration points with 3rd party API’s (e.g. Xero). In order to test those, I need to make the back-end and front-end from the app available to the internet in order to test those integrations. This is necessary because those integrations have to be registered to a domain and secret and also have a callback URL which needs to be accessible.

4. Error messages and/or full log output:

ERROR:  Your configuration file must define at least one tunnel when using --all. To intentionally start no tunnels, use `ngrok start --none.`

5. What I already tried:

The usual way of enabling this is via https://ngrok.com. However I can’t seem to get that to work.

I’m using this command:

ngrok http http://fe.mnr.localhost

Which should work, as it sets up a forward to the local docker network… but it just presents a blank page via this proxy, even though the destination being proxied to is available on that localhost.

I found this: https://chriskirby.net/blog/using-ngrok-through-docker-for-local-service-development-on-mac but I’m not sure how to interpret that for this docker-compose environment, nor how that would integrate with the Caddy proxy that’s happening.

I tried adding this to the docker-compose.yml file:

  ngrok:
    image: ngrok/ngrok:alpine
    ports:
      - 4551:4551
    links:
      - caddy
    environment:
      - DOMAIN=caddy
      - PORT=80
      - NGROK_LOOK_DOMAIN = host.docker.internal
      - NGROK_AUTHTOKEN = <some auth token>
    extra_hosts:
      host.docker.internal: host-gateway

6. Links to relevant resources:

Hi :slight_smile:

Sorry for the late response.

ngrok does not change the Host: header by default.
See ngrok Secure Tunnels | ngrok Documentation

So when you open the by ngrok provided domain, e.g. https://example.eu.ngrok.io/, then ngrok will just pass Host: example.eu.ngrok.io to Caddy.
And Caddy uses that to decide which vhost you are trying to reach (in your case fe.mnr.localhost or be.mnr.localhost).
But example.eu.ngrok.io won’t match for them, so Caddy is like “nah, I don’t know that one, sorry”.

So you would have to use

ngrok http --host-header=rewrite https://fe.mnr.localhost

and if you want to proxy be.mnr.localhost at the same time, then you need another instance via

ngrok http --host-header=rewrite https://be.mnr.localhost

Note: You might run into your ngrok plans’ rate limits

But also note, that I replaced http:// with https://.
That is because Caddy serves all vhosts via https by default. That’s called Automatic HTTPS — Caddy Documentation.
ngrok mentions that briefly in their docs at ngrok Secure Tunnels | ngrok Documentation

3 Likes

Thanks, that took me a step forward but now Xero complains about TLS:

See my comments on StackOverflow: cakephp - Why is Xero OAUTH api is no longer returning JSON - Stack Overflow

Is this a limitation in caddy 2.0 or ngrok or what?

I can’t really help with Xero since I am not familiar with that.

So just to summarize, you are using ngrok to proxy the traffic to Caddy, which then serves your application (frontend & backend), right?

In which case, ngrok is the one deciding the minimum TLS version to serve to your clients.

ngrok state in their docs:

It is possible to specify the minimum TLS version that clients are required to use to talk to the ngrok edge for your tunnel.

Turns out Xero had to adjust their firewall to make it work. All ok now tx.

1 Like

This topic was automatically closed after 30 days. New replies are no longer allowed.