Caddy-security gitlab oauth errors : Client authentication failed

1. Caddy version (caddy version):

caddy:2.5.1

2. How I run Caddy:

I am building caddy with the following Dockerfile

ARG image_ref
FROM ${image_ref}-builder AS caddy-build
RUN xcaddy build --with github.com/greenpau/caddy-security

FROM ${image_ref}
COPY --from=caddy-build /usr/bin/caddy /usr/bin/caddy

which is built and launched from docker-compose.yml (extract) :

  caddy:
    # cf. https://github.com/caddyserver/caddy/releases
    #image: caddy:2.5.2
    build:
      context: ./caddy/
      dockerfile: Dockerfile
      args:
        image_ref: caddy:2.5.1
    container_name: caddy
    ports:
      - "80:80"
      - "443:443"
      - "9090:9090"
        # - "2019:2019"
        # - "3000:3000"
        # - "9090:9090"
        # - "9093:9093"
        # - "9091:9091"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - ./caddy:/etc/caddy
      - caddy_data:/data
    env_file:
      ./.env
    environment:
      - ADMIN_USER=${ADMIN_USER}
      - ADMIN_PASSWORD=${ADMIN_PASSWORD}
      - ADMIN_PASSWORD_HASH=${ADMIN_PASSWORD_HASH}
      - OAUTH_CLIENT_ID=${CADDY_GITLAB_APP}
      - OAUTH_CLIENT_SECRET=${CADDY_GITLAB_SECRET}
      - OAUTH_USER_GROUP_FILTERS=${GITLAB_USER_GROUP}
    restart: always
    networks:
      - monitor-net
    labels:
      org.label-schema.group: "monitoring"

a. System environment:

docker

b. Command:

Paste command here.

c. Service/unit/compose file:

Paste full file contents here.
Make sure backticks stay on their own lines,
and the post looks nice in the preview pane.

d. My complete Caddyfile or JSON config:

{
        # email to use on Let's Encrypt
        email email@siteweb.com

        # Uncomment for debug
#       acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
        #debug

        # https://authp.github.io/docs/authenticate/oauth/backend-oauth2-0009-gitlab
        #
        http_port 80
        https_port 443
        # debug

        order authenticate before respond
        order authorize before basicauth

        security {
                oauth identity provider gitlab {
                        realm gitlab
                        driver gitlab
                        domain_name gitlab.com
                        client_id ${OAUTH_CLIENT_ID}
                        client_secret ${OAUTH_CLIENT_SECRET}
                        scopes openid email profile
                        #user_group_filters ${OAUTH_USER_GROUP_FILTERS}
                }

                authentication portal myportal {
                        crypto default token lifetime 3600
                        crypto key sign-verify {env.JWT_SHARED_KEY}
                        enable identity provider gitlab
                        cookie domain jump.siteweb.com
                        ui {
                                links {
                                        "My Identity" "/whoami" icon "las la-user"
                                }
                        }

                        #transform user {
                        #       match realm gitlab
                        #       action add role authp/user
                        #       ui link "File Server" https://assetq.jump.siteweb.com:443/ icon "las la-star"
                        #}

                        #transform user {
                        #       match realm gitlab
                        #       match email me@mail.com
                        #       action add role authp/admin
                        #}
                }

                authorization policy mypolicy {
                        set auth url https://jump.siteweb.com:443/oauth2/gitlab
                        crypto key verify {env.JWT_SHARED_KEY}
                        allow roles authp/admin authp/user
                        validate bearer header
                        inject headers with claims
                }
        }

        #oauth identity provider generic {
        #  icon "Gitlab" "gitlab" "yellow darken-4"
        #}
}

(my-auth) {
        basicauth {
                {$ADMIN_USER} {$ADMIN_PASSWORD_HASH}
        }
}

auth.jump.siteweb.com {
        authenticate with myportal
}

#assetq.jump.siteweb.com {
#       authorize with mypolicy
#       root * {env.HOME}/www
#       file_server
#}

whoami.jump.siteweb.com {
        #import my-auth
        authenticate with myportal
        reverse_proxy whoami:80
}

I created the application in the gitlab subgroup (same than the one indicated in user_group_filters) and it seems from the log to be ok :

caddy  | 2022-07-12T22:57:15.361419619Z {"level":"info","ts":1657666635.3608851,"logger":"security","msg":"successfully configured OAuth 2.0 identity provider","provider":"gitlab","client_id":"$ERROR_REPLACEMENT","server_id":"","domain_name":"gitlab.com","metadata":{"authorization_endpoint":"https://gitlab.com/oauth/authorize","claim_types_supported":["normal"],"claims_supported":["iss","sub","aud","exp","iat","sub_legacy","name","nickname","email","email_verified","website","profile","picture","groups","groups_direct","https://gitlab.org/claims/groups/owner","https://gitlab.org/claims/groups/maintainer","https://gitlab.org/claims/groups/developer"],"grant_types_supported":["authorization_code","password","client_credentials","refresh_token"],"id_token_signing_alg_values_supported":["RS256"],"introspection_endpoint":"https://gitlab.com/oauth/introspect","issuer":"https://gitlab.com","jwks_uri":"https://gitlab.com/oauth/discovery/keys","response_modes_supported":["query","fragment"],"response_types_supported":["code"],"revocation_endpoint":"https://gitlab.com/oauth/revoke","scopes_supported":["api","read_api","read_user","read_repository","write_repository","read_registry","write_registry","sudo","openid","profile","email"],"subject_types_supported":["public"],"token_endpoint":"https://gitlab.com/oauth/token","token_endpoint_auth_methods_supported":["client_secret_basic","client_secret_post"],"userinfo_endpoint":"https://gitlab.com/oauth/userinfo"},"jwks_keys":{"":{"alg":"RS256","e":"AQAB","kid":"-","kty":"RSA","n":"=","use":"sig"},"":{"alg":"RS256","e":"AQAB","kid":"","kty":"RSA","n":"=","use":"sig"}},"required_token_fields":["access_token","id_token"],"delayed_by":0,"retry_attempts":0,"retry_interval":0,"scopes":["openid","email","profile"],"login_icon":{"class_name":"lab la-gitlab la-2x","color":"white","background_color":"#fc6d26","text_color":"#37474f"}}
caddy  | 2022-07-12T22:57:15.378897204Z {"level":"info","ts":1657666635.378642,"logger":"security","msg":"provisioned app instance","app":"security"}

3. The problem I’m having:

This is the message I receive on gitlab :

An error has occurred

Client authentication failed due to unknown client, no client authentication included, or unsupported authentication method.

image
then
image

4. Error messages and/or full log output:

5. What I already tried:

Changing the callback :

6. Links to relevant resources:

I was trying to adapt : https://github.com/authp/authp.github.io/blob/main/assets/conf/oauth/pingid/Caddyfile (thanks @greenpau !) after following https://authp.github.io/docs/authenticate/oauth/backend-oauth2-0009-gitlab documentation.

Thanks in advance for your help, any clue will be welcome !

#caddy-security #oauth #gitlab

@copolycube , based on your config, the callback is https://auth.jump.siteweb.com/oauth2/gitlab/authorization-code-callback

The following makes the root as /, not /auth/.

auth.jump.siteweb.com {
        authenticate with myportal
}
4 Likes

issue 1 : env var in Caddyfile format / $ERROR_REPLACEMENT :white_check_mark:

exploration step by step for who might find the same error later...

I noticed something strange (or that I do not fully understand yet):

caddy logs:

caddy  | 2022-07-13T15:36:44.911264882Z {"level":"debug","ts":1657726604.9107363,"logger":"security","msg":"fetchMetadataURL succeeded","identity_provider_name":"gitlab","metadata":{"authorization_endpoint":"https://gitlab.com/oauth/authorize","claim_types_supported":["normal"],"claims_supported":["iss","sub","aud","exp","iat","sub_legacy","name","nickname","email","email_verified","website","profile","picture","groups","groups_direct","https://gitlab.org/claims/groups/owner","https://gitlab.org/claims/groups/maintainer","https://gitlab.org/claims/groups/developer"],"grant_types_supported":["authorization_code","password","client_credentials","refresh_token"],"id_token_signing_alg_values_supported":["RS256"],"introspection_endpoint":"https://gitlab.com/oauth/introspect","issuer":"https://gitlab.com","jwks_uri":"https://gitlab.com/oauth/discovery/keys","response_modes_supported":["query","fragment"],"response_types_supported":["code"],"revocation_endpoint":"https://gitlab.com/oauth/revoke","scopes_supported":["api","read_api","read_user","read_repository","write_repository","read_registry","write_registry","sudo","openid","profile","email"],"subject_types_supported":["public"],"token_endpoint":"https://gitlab.com/oauth/token","token_endpoint_auth_methods_supported":["client_secret_basic","client_secret_post"],"userinfo_endpoint":"https://gitlab.com/oauth/userinfo"},"userinfo_endpoint":"https://gitlab.com/oauth/userinfo"}

caddy  | 2022-07-13T15:36:45.105955001Z {"level":"info","ts":1657726605.1053586,"logger":"security","msg":"successfully configured OAuth 2.0 identity provider","provider":"gitlab","client_id":"$ERROR_REPLACEMENT","server_id":"","domain_name":"gitlab.com","metadata":{"authorization_endpoint":"https://gitlab.com/oauth/authorize","claim_types_supported":["normal"],"claims_supported":["iss","sub","aud","exp","iat","sub_legacy","name","nickname","email","email_verified","website","profile","picture","groups","groups_direct","https://gitlab.org/claims/groups/owner","https://gitlab.org/claims/groups/maintainer","https://gitlab.org/claims/groups/developer"],"grant_types_supported":["authorization_code","password","client_credentials","refresh_token"],"id_token_signing_alg_values_supported":["RS256"],"introspection_endpoint":"https://gitlab.com/oauth/introspect","issuer":"https://gitlab.com","jwks_uri":"https://gitlab.com/oauth/discovery/keys","response_modes_supported":["query","fragment"],"response_types_supported":["code"],"revocation_endpoint":"https://gitlab.com/oauth/revoke","scopes_supported":["api","read_api","read_user","read_repository","write_repository","read_registry","write_registry","sudo","openid","profile","email"],"subject_types_supported":["public"],"token_endpoint":"https://gitlab.com/oauth/token","token_endpoint_auth_methods_supported":["client_secret_basic","client_secret_post"],"userinfo_endpoint":"https://gitlab.com/oauth/userinfo"},"jwks_keys":{"<REMOVED FOR READABILITY>","use":"sig"},"<REMOVED FOR READABILITY>":{"alg":"RS256","e":"AQAB","kid":"<REMOVED FOR READABILITY>","use":"sig"}},"required_token_fields":["access_token","id_token"],"delayed_by":0,"retry_attempts":0,"retry_interval":0,"scopes":["openid","email","profile"],"login_icon":{"class_name":"lab la-gitlab la-2x","color":"white","background_color":"#fc6d26","text_color":"#37474f"}}

The first message "successfully configured OAuth 2.0 identity provider" had me think that it was correctly set up, but I also see : "client_id":"$ERROR_REPLACEMENT",

docker-compose config extract:

  caddy:
    build:
      context: /path/to/dir/caddy
      dockerfile: Dockerfile
      args:
        image_ref: caddy:2.5.1
    container_name: caddy
    environment:
      ADMIN_PASSWORD: *PASSWDREDACTED*
      ADMIN_PASSWORD_HASH: *PASSWDREDACTED_HASH*
      ADMIN_USER: user
      HETZNER_API_TOKEN_READ: *REDACTED*
      OAUTH_CLIENT_ID: *REDACTED*
      OAUTH_CLIENT_SECRET: *REDACTED*
      OAUTH_USER_GROUP_FILTERS: group/sub-group
    extra_hosts:
    - host.docker.internal:host-gateway
    labels:
      org.label-schema.group: monitoring
    networks:
      monitor-net: null
    ports:
    - mode: ingress
      target: 80
      published: "80"
      protocol: tcp
    - mode: ingress
      target: 443
      published: "443"
      protocol: tcp
    - mode: ingress
      target: 9090
      published: "9090"
      protocol: tcp
    restart: always
    volumes:
    - type: bind
      source: /path/to/dir/caddy
      target: /etc/caddy
      bind:
        create_host_path: true
    - type: volume
      source: caddy_data
      target: /data
      volume: {}
  • problem : "$ERROR_REPLACEMENT"
    it seems like the environments variables were not used by Caddy / caddy-security plugin / Caddyfile:

Caddyfile extract :

                        client_id ${OAUTH_CLIENT_ID}
                        client_secret ${OAUTH_CLIENT_SECRET}

But it’s found in the container when running the env command :

$ sudo docker exec -it caddy 'env' |grep OAUTH
OAUTH_USER_GROUP_FILTERS=group/subgroup
OAUTH_CLIENT_ID=*REDACTED*
OAUTH_CLIENT_SECRET=*REDACTED*

… leading me to the question on how to pass environment variable to caddy or caddy-security when running within docker ? (since docker detects them correctly it’s actually at caddy level → Caddyfile expects the format to be {$var} and not ${var} …

:white_check_mark: First issue: I wasn’t passing my environment variables correctly in the Caddyfile, using, the correct Caddyfile format beeing {$VAR}.

`Caddyfile

                        client_id {$OAUTH_CLIENT_ID}
                        client_secret {$OAUTH_CLIENT_SECRET}

with .env containing

OAUTH_CLIENT_ID=*redacted*
OAUTH_CLIENT_SECRET=*redacted*

issue 2 : Redirect URI in Gitlab application :white_check_mark:

issue 2, details... and exploration path

after solving the issue 1, it still fails with the following message :

and I can see the URL beeing the following :
https://gitlab.com/oauth/authorize?client_id=<correct client id>&redirect_uri=https%3A%2F%2Fwhoami.jump.siteweb.com%2Foauth2%2Fgitlab%2Fauthorization-code-callback&response_type=code&scope=openid+email+profile&state=fa3d53d2-0040-44d8-ba0f-0834679e5a8e

=> each URL that gitlab redirects to needs to be included in gitlab application “redirect URI” field.

i.e. in my case :

https://auth.jump.siteweb.com/oauth2/gitlab/authorization-code-callback
https://whoami.jump.siteweb.com/oauth2/gitlab/authorization-code-callback

doc suggestion

doc modif suggestion

So if I’m correct in the caddy-security doc & examples I think there is either a fix or a note that could be helpfully added:

(extract from : authp.github.io/Caddyfile at main · authp/authp.github.io · GitHub )

auth.myfiosgateway.com {
	import tls_config
	authenticate with myportal
}

implies that the callback is
https://auth.myfiosgateway.com/oauth2/gitlab/authorization-code-callback

and not the one found on Gitlab | Caddy Security first screenshot, right ?

I’ll create a PR on github with those details if my understanding is correct.

Thanks a lot @greenpau for leading me on this path !

1 Like

This topic was automatically closed after 30 days. New replies are no longer allowed.