Inter-container routing

1. The problem I’m having:

I have 3 containers running on one machine :

  1. Authentik (Identity provider)
  2. Gitea (self-hosted GitHub-like)
  3. Caddy

Caddy has a configuration to use subdomains to route incoming requests to each container. I’m using self-signed certificates (mounted as volumes in the docker-compose). The Caddy file is as following

{
  local_certs
}
auth.example:443 {
  reverse_proxy authentik-server:9000
  tls /etc/ssl/servercert.pem /etc/ssl/serverkey.pem
}
git.example:443 {
  reverse_proxy gitea-server:3000
  tls /etc/ssl/servercert.pem /etc/ssl/serverkey.pem
}

From my machine I can access both of these domains (auth.example and git.example) and I end up into each container’s webpage. Fine.

The issue I face is when container 1 needs to talk to container 2 using their FQDN. For example, when I set Gitea to use Autentik for the identity provider. I have to setup the URL to reach to the “front-end” one (the FQDN auth.example). So I configure gitea to talk to auth.example, and I’d expect caddy to route this transparently and to forward the request to the Authentik container.

2. Error messages and/or full log output:

However this do not work as Gitea cannot access the Authentik container using its FQDN.

From the gitea container, I’ve tried to connect manually using netcat, and it fails.

3. Caddy version:

v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk=

4. How I installed and ran Caddy:

Docker-compose

a. System environment:

Docker

b. Command:

docker compose -f relevant-docker-compose.yml up

c. Compose file:

services:
  caddy:
    container_name: caddy
    image: caddy
    restart: always
    ports:
      - '80:80'
      - '443:443'
    volumes:
      - caddy-config:/config
      - caddy-data:/data
      - ./Caddyfile:/etc/caddy/Caddyfile
      - ./certs/test/:/etc/ssl/:ro
    networks:
      - corenet

volumes:
  caddy-config:
  caddy-data:

networks:
  corenet:
    name: corenet
    #external: false
    # external: true # created from another yml

Why not just have apps running in Docker connect to eachother’s containers directly via service name? What’s wrong with that?

1 Like

Below is probably way more verbose than it needs to be, and as @francislavoie suggests you probably can connect directly.

If you do need to route through Caddy such as for centralizing TLS, you can do so with a network alias for a service as shown below.

If you are having weird networking issues because of indirect connections (host IP / public DNS instead of direct container DNS), you may want to look over this reference table for docker network caveats.


Docker networks and container DNS records

Docker Compose will create a custom network (equivalent to docker network create) for each compose.yaml, and docker run will always start containers on the legacy docker bridge network (which has slightly different functionality, this network is discouraged vs custom networks).

If all your service containers are on the same network, they can reach each other by their service name in compose.yaml, their container_name if assigned, or their hostname too. You can also use links, extra_hosts, external_links, or the more useful one networks.<network-name-here>.aliases.

Compose will define the default network created for each compose.yaml as default internally, while the external name that is used can be found under docker network ls if you need to reference it. Alternatively you can add your own networks with a custom external name, and you can override the default one Compose generates by using the default network name.


Network alias

Now that we’ve clarified all that. When you want Gitea to connect to Authentik, it’s as simple as referencing any of the various sources mentioned above for valid DNS names to resolve (which the default embedded DNS that Docker provides will handle for you).

If you want traffic to route through Caddy for that service instead of a direct connection, just like you would without containers or were connecting via the browser, then network aliases are perfect for this.

You may only need this for Authentik btw. If you do need a separate FQDN like I did, your Caddyfile will also need to reference the alias as another site-address along with cert provisioned.

Here’s a rough example:

services:
  reverse-proxy:
    image: caddy:2.8
    container_name: caddy
    # If also wanting to reach from the host via `https://example.localhost`
    ports:
      - "127.0.0.1:80:80"
      - "127.0.0.1:443:443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
    # Containers on this network will resolve these aliases to the IP of this container:
    networks:
      default:
        aliases:
          - auth.example.test

Basic Caddyfile example:

{
  local_certs
}

# Auth service:
# I have two different TLDs here, neither is publicly resolvable:
# - `.localhost` for host / browser requires no custom DNS or /etc/hosts config
# - `.test` for containers DNS since they cannot use `.localhost`
# These both use the same cert, if you have LetsEncrypt for public but
# internal CA for non-public domains like `.test` you'd need separate site blocks?
auth.example.localhost, auth.example.test {
  reverse_proxy authelia:80
}

# Standard service like your gitea:
my-service-a.example.localhost {
  reverse_proxy my-service-a:80
}

# Forward auth example to require Authelia for login to access this service
my-service-b.example.localhost {
  forward_auth authelia:80 {
    uri /api/authz/forward-auth
    copy_headers Remote-User Remote-Groups Remote-Email Remote-Name
  }

  # After forward auth is successful, continues to intended service:
  reverse_proxy my-service-b:80
}

Nothing really special going on there in the Caddyfile, but note how I adjusted the site block for Authelia to recognize the Compose network alias added for Caddy. You can probably use the same custom TLD that you already have, mine was a workaround for .localhost usage on the host.

The distinction was helpful when I was learning about IdP integration as your services like gitea may be configured with both if they differ. Here’s an example for Roundcube:

// Browser URL related:
$config['oauth_auth_uri'] = 'https://auth.example.localhost/api/oidc/authorization';
// Internal only:
$config['oauth_token_uri'] = 'https://auth.example.test/api/oidc/token';
$config['oauth_identity_uri'] = 'https://auth.example.test/api/oidc/userinfo';

You could get it working without Caddy and do a direct container connection, some IdP are more strict however and it can be more convenient to centralize traffic + TLS through Caddy this way.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.