Configure Caddy to handle gRPC request

[Samir]

1. Caddy version:

2.6.4

2. How I installed, and run Caddy:

FROM caddy:2.6.4-alpine

a. System environment:

Docker

b. Command:

CMD ["caddy", "run", "--config", "/etc/caddy/Caddyfile"]

c. Service/unit/compose file:

FROM caddy:2.6.4-alpine
COPY Caddyfile /etc/caddy/Caddyfile
CMD ["caddy", "run", "--config", "/etc/caddy/Caddyfile"]

d. My complete Caddy config:

{
  debug
  skip_install_trust
  http_port 8080
  https_port 8443
  auto_https disable_redirects
}
:8443 {
  tls /etc/tls.crt /etc/tls.key

  @rlsRegex path_regexp rls ^~/test.v3.EmployeeService
  reverse_proxy @rlsRegex grpc://127.0.0.1:7710 {
        transport grpc 
    }
}

3. The problem I’m having:

I am using Caddy server as a reverse proxy to perform TLS termination. An API Gateway makes gRPC request to the Caddy server. The Caddy server performs the TLS termination and then make a gRPC request to the backend service.

In my case, I want to understand how to configure CaddyServer to accept gRPC requests and then make gRPC call to backend service.

In nginx, I used to perform this task easily using grpc_pass, however I’m unable to find an equivalent of it in Caddy server.

4. Error messages and/or full log output:

{"level":"info","ts":1676775638.139815,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
Error: adapting config using caddyfile: parsing caddyfile tokens for 'reverse_proxy': /etc/caddy/Caddyfile:13 - Error during parsing: getting module named 'http.reverse_proxy.transport.grpc': module not registered: http.reverse_proxy.transport.grpc

5. What I already tried:

I tried multiple options,

1. Create a custom build of Caddy using the tool xcaddy. I created a Dockerfile as below,

FROM caddy:2.6.4-alpine
RUN xcaddy build \
    --with github.com/caddyserver/grpc
COPY Caddyfile /etc/caddy/Caddyfile
CMD ["caddy", "run", "--config", "/etc/caddy/Caddyfile"]

But the build failed because github.com/caddyserver/grpc doesn’t exists anymore.

2. Tried replacing removing transport grpc sub-directive from the Caddyfile. That enabled the CaddyServer to launch successfully. However, now the API Gateway was not able to sent request to Caddy Server. The call failed with error message,

"error":"rpc error: code = Unknown desc = malformed header: missing HTTP content-type"

3. I tried to use the original nginx config file, but the nginx adapter was unable to handle many nginx directives like below,

{"level":"info","ts":1676773574.7542129,"msg":"using provided configuration","config_file":"/etc/nginx/nginx.conf","config_adapter":"nginx"}

{"level":"warn","ts":1676773574.7557411,"msg":"worker_processes: unrecognized or unsupported nginx directive","adapter":"nginx","file":"/etc/nginx/nginx.conf","line":1}

{"level":"warn","ts":1676773574.7557724,"msg":"error_log: unrecognized or unsupported nginx directive","adapter":"nginx","file":"/etc/nginx/nginx.conf","line":3}

{"level":"warn","ts":1676773574.7557764,"msg":"pid: unrecognized or unsupported nginx directive","adapter":"nginx","file":"/etc/nginx/nginx.conf","line":4}

{"level":"warn","ts":1676773574.7557797,"msg":"events: unrecognized or unsupported nginx directive","adapter":"nginx","file":"/etc/nginx/nginx.conf","line":6}

{"level":"warn","ts":1676773574.7557838,"msg":"ssl_session_cache: unrecognized or unsupported nginx directive","adapter":"nginx","file":"/etc/nginx/nginx.conf","line":12}

{"level":"warn","ts":1676773574.755787,"msg":"ssl_session_timeout: unrecognized or unsupported nginx directive","adapter":"nginx","file":"/etc/nginx/nginx.conf","line":13}

{"level":"warn","ts":1676773574.7557902,"msg":"ssl_certificate: unrecognized or unsupported nginx directive","adapter":"nginx","file":"/etc/nginx/nginx.conf","line":20}

{"level":"warn","ts":1676773574.7557933,"msg":"ssl_certificate_key: unrecognized or unsupported nginx directive","adapter":"nginx","file":"/etc/nginx/nginx.conf","line":21}

{"level":"warn","ts":1676773574.7557964,"msg":"ssl_protocols: unrecognized or unsupported nginx directive","adapter":"nginx","file":"/etc/nginx/nginx.conf","line":22}

{"level":"warn","ts":1676773574.7558014,"msg":"ssl_ciphers: unrecognized or unsupported nginx directive","adapter":"nginx","file":"/etc/nginx/nginx.conf","line":23}

{"level":"warn","ts":1676773574.7558048,"msg":"location: the adapter treats the ^~ location modifier as prefix match only, with no prioritization","adapter":"nginx","file":"/etc/nginx/nginx.conf","line":38}

and it returned error message,
Error: loading initial config: loading new config: http app module: start: listening on :80: listen tcp :80: bind: permission denied

6. Links to relevant resources:

[WeidiDeng]

change

reverse_proxy @rlsRegex grpc://127.0.0.1:7710 {
        transport grpc 
    }

to

reverse_proxy @rlsRegex h2c://127.0.0.1:7710

[Samir]

It didn’t work …
"error":"rpc error: code = Unknown desc = malformed header: missing HTTP content-type"

[francislavoie]

Your regexp matcher is likely incorrect.

It looks like you have nothing else in that site though. Just remove the matcher altogether. Matchers are only needed if you’re splitting the traffic in some way.

[Samir]

Thanks! Fixing the regex solved the problem.