Reverse Proxy H2C Stoped to Function

1. Caddy version (caddy version):

Docker caddy:2.2.1-alpine

Also tested with v2.4.5

2. How I run Caddy:

Using docker

a. System environment:

Docker system on Ubuntu

b. Command:

docker-compose up

c. compose file:

docker-compose file:

version: "3.2"
    container_name: mapper-api
    image: docker-repo-/mapper-api:v1.2.3
      - /tmp:/host
      - 8008:8008
    restart: on-failure
    command: serve -dgAddr
      - api_net
    container_name: caddy
    image: caddy:2.2.1-alpine
      - /tmp:/host
      - ${PWD}/Caddyfile:/etc/caddy/Caddyfile:ro
      - ${PWD}/caddy_data:/data
      - ${PWD}/caddy_config:/config
      - 443:443
    restart: on-failure
      - api_net
    driver: bridge

d. My complete Caddyfile or JSON config:

# The Caddyfile is an easy way to configure your Caddy web server.
# Unless the file starts with a global options block, the first
# uncommented line is always the address of your site.
# To use your own domain name (with automatic HTTPS), first make
# sure your domain's A/AAAA DNS records are properly pointed to
# this machine's public IP, then replace the line below with your
# domain name.

log {
    level INFO

reverse_proxy {
    to h2c://api:8008
    transport http {
        versions h2c 2

# Refer to the Caddy docs for more information:

3. The problem I’m having:

This configuration worked since Jan 2021.
It listen to port 443 then using h2c it forward the request to the grpc server in clear-text.

Last day (Sep 30) around 5pm GMT grpc cli started to return error:

  "error": "14 UNAVAILABLE: failed to connect to all addresses"
  • I validated the cert and that the port is open (using firefox), and it is open and valid till end of Dec.
  • I opened the port to the insecure endpoint (skipping caddy) and the grpc server behaved correctly.
  • The dockers was not restarted or changed in the last month
  • I wiped the caddy data dir and restarted to get a new cert/reset/whatever. Didn’t helped.
  • Restarted other dockers / upgrade caddy to latest. None helped.

4. Error messages and/or full log output:

Reverse proxy to GRPC didn’t yield any logs whatsoever and that’s since I started to use Caddy.
Did tried to change Caddy log level ro DEBUG. nothing changed.
Maybe the log directive in my Caddy is incorrect???

6. Links to relevant resources:

My initial discuss on how to implament h2c:

Another discussion about h2c:

FYI you can simplify this to just this config:

reverse_proxy h2c://api:8008

The h2c:// scheme is a shortcut to setting versions.

Doesn’t sound like an issue with Caddy, then. If you didn’t change anything in Caddy, it can’t be a problem with Caddy.

You need to set the debug global option, not change the log directive’s level – the log directive only configures access logs. Additional debug logs are not emitted via access logging.

Add this at the top of your Caddyfile (before the site address):

1 Like

Thanks for the tips.

Turned out the the problem was with an outdated root certificate files that come as part of the grpc stack and is used by most grpc clients/servers implementation ( ref ).


Ah, that makes sense. Thanks for following up!

1 Like

This topic was automatically closed after 30 days. New replies are no longer allowed.