Renewing certificate on-demand failed

Help
1

1. Caddy version (caddy version):

caddy:2.2.1-alpine

2. How I run Caddy:

Caddy is running inside docker and it run with caddy run command.

a. System environment:

AWS EKS

b. Command:

caddy run

c. Service/unit/compose file:

paste full file contents here

d. My complete Caddyfile or JSON config:

(common) {
    header * {
        Server "API-Gateway"
    }

    route /_health/live {
        respond "live"
    }

    route /_health/ready {
        respond "ready"
    }   

    log # Enable access logs
}



{
    admin off
    email admin@web.com
    on_demand_tls {
		ask      http://app.default.svc.cluster.local/api/v1/domains/check
		interval 1m # in a 1 minute interval, allow at most 10 requests for certifcates
		burst    10
	}
    
}

:80 {
    encode zstd gzip
    import common

    route * {
        redir https://{host}{uri} 301
    }
}

:443 {
    tls {
        on_demand
    }
    encode zstd gzip
    import common
     route * {
        reverse_proxy http://app.default.svc.cluster.local
    }
}

3. The problem I’m having:

Hey team, thanks for the amazing job on this, recently one of my user SSL certificates expired but caddy can not renew it again

4. Error messages and/or full log output:

After checking the logs from docker container the only error that i saw is saying
**renewing certificate on-demand failed** and i am extremely new to the caddy and not sure how i can fix it.
Moreover, I saw this as well
no OCSP stapling for [domain.com]: parsing OCSP response: ocsp: error from server: unauthorized

5. What I already tried:

6. Links to relevant resources:

2

Caddy’s usually a bit more explicit than that when it comes to certificate failures.

Can you paste the actual log output?

1 Like
3

Please upgrade to v2.3.0!

4

Thanks for the replies. I had updated to the v2.3.0 however yesterday for fixing the issue I used zeroSSL to issue a certificate.
This the complete log that I received yesterday and I am afraid if this happens again. any helps will appreciate.

stapling OCSP: no OCSP stapling for [domain.com]: parsing OCSP response: ocsp: error from server: unauthorized
renewing certificate on-demand failed
aborting with incomplete response
renewing certificate on-demand failed
Unexpected EOF reading from backend

5

Can we get the full log lines rather than just parts of them? You’re omitting important information.

6

@francislavoie
{
“id”: “AQAAAXg8rh-KgFWkxwAAAABBWGc4cmlkNUFBRHVCMVNWSThHR1J3QVM”,
“content”: {
“timestamp”: “2021-03-16T20:15:47.594Z”,
“tags”: [
“cluster_name:production-us-east-1”,
“container_id:8ff1dfe68326fc92d0190952fbc71e4378d0aadc1af0970731cfa2a1e53ccc0f”,
“container_name:k8s_web_api-gateway-65478f6cb6-8bmh9_default_4ec59138-e4b7-450e-b99f-e4900aa31ae4_0”,
“dirname:/var/log/pods/default_api-gateway-65478f6cb6-8bmh9_4ec59138-e4b7-450e-b99f-e4900aa31ae4/web”,
“display_container_name:web_api-gateway-65478f6cb6-8bmh9”,
“env:production-us-east-1”,
“filename:0.log”,
“kube_cluster_name:production-us-east-1”,
“kube_container_name:web”,
“kube_deployment:api-gateway”,
“kube_namespace:default”,
“kube_replica_set:api-gateway-65478f6cb6”,
“kube_service:api-gateway”,
“pod_name:api-gateway-65478f6cb6-8bmh9”,
“pod_phase:running”,
“service:api-gateway”,
“short_image:api-gateway”,
“source:custom”
],
“service”: “api-gateway”,
“attributes”: {
“msg”: “renewing certificate on-demand failed”,
“logger”: {
“name”: “tls.on_demand”
},
“not_after”: 1615781969,
“level”: “error”,
“subjects”: [
“community.dyme.app”
],
“error”: “EOF”,
“ts”: 1615925747.4728048
}
}
}

{
“id”: “AQAAAXg8rh-KgFWkxwAAAABBWGc4cmlkNUFBRHVCMVNWSThHR1J3QVM”,
“content”: {
“timestamp”: “2021-03-16T20:15:47.594Z”,
“tags”: [
“cluster_name:production-us-east-1”,
“container_id:8ff1dfe68326fc92d0190952fbc71e4378d0aadc1af0970731cfa2a1e53ccc0f”,
“container_name:k8s_web_api-gateway-65478f6cb6-8bmh9_default_4ec59138-e4b7-450e-b99f-e4900aa31ae4_0”,
“dirname:/var/log/pods/default_api-gateway-65478f6cb6-8bmh9_4ec59138-e4b7-450e-b99f-e4900aa31ae4/web”,
“display_container_name:web_api-gateway-65478f6cb6-8bmh9”,
“env:production-us-east-1”,
“filename:0.log”,
“kube_cluster_name:production-us-east-1”,
“kube_container_name:web”,
“kube_deployment:api-gateway”,
“kube_namespace:default”,
“kube_replica_set:api-gateway-65478f6cb6”,
“kube_service:api-gateway”,
“pod_name:api-gateway-65478f6cb6-8bmh9”,
“pod_phase:running”,
“service:api-gateway”,
“short_image:api-gateway”,
“source:custom”
],
“host”: “i-0d35cbf6702efcfed”,
“service”: “api-gateway”,
“attributes”: {
“msg”: “renewing certificate on-demand failed”,
“logger”: {
“name”: “tls.on_demand”
},
“not_after”: 1615781969,
“level”: “error”,
“subjects”: [
“community.dyme.app”
],
“error”: “EOF”,
“ts”: 1615925747.4728048
}
}
}
{
“id”: “AQAAAXg8tcBrKwPqKQAAAABBWGc4dGNScUFBQlZrMXQwQ29vWF9RQmY”,
“content”: {
“timestamp”: “2021-03-16T20:24:07.531Z”,
“tags”: [
“cluster_name:production-us-east-1”,
“container_id:41a26d43c823bdfeb60ae0a29d52b53a526a3de3dabecedeca51def4730a86a8”,
“container_name:k8s_web_api-gateway-65478f6cb6-w4hsx_default_5a983c1e-98ac-42ed-8d76-3138f331f106_0”,
“dirname:/var/log/pods/default_api-gateway-65478f6cb6-w4hsx_5a983c1e-98ac-42ed-8d76-3138f331f106/web”,
“display_container_name:web_api-gateway-65478f6cb6-w4hsx”,
“env:production-us-east-1”,
“filename:0.log”,
“kube_cluster_name:production-us-east-1”,
“kube_container_name:web”,
“kube_deployment:api-gateway”,
“kube_namespace:default”,
“kube_replica_set:api-gateway-65478f6cb6”,
“kube_service:api-gateway”,
“pod_name:api-gateway-65478f6cb6-w4hsx”,
“pod_phase:running”,
“service:api-gateway”,
“short_image:api-gateway”,
“source:custom”
],
“host”: “i-0ec0f07450da3948f”,
“service”: “api-gateway”,
“attributes”: {
“msg”: “maintining newly-loaded certificate”,
“logger”: {
“name”: “tls.on_demand”
},
“server_name”: “community.dyme.app”,
“level”: “error”,
“error”: “EOF”,
“ts”: 1615926247.2368112
}
}
}
{
“id”: “AQAAAXg8tcBqKwPqEQAAAABBWGc4dGNScUFBQlZrMXQwQ29vWF9RQkg”,
“content”: {
“timestamp”: “2021-03-16T20:24:07.530Z”,
“tags”: [
“cluster_name:production-us-east-1”,
“container_id:41a26d43c823bdfeb60ae0a29d52b53a526a3de3dabecedeca51def4730a86a8”,
“container_name:k8s_web_api-gateway-65478f6cb6-w4hsx_default_5a983c1e-98ac-42ed-8d76-3138f331f106_0”,
“dirname:/var/log/pods/default_api-gateway-65478f6cb6-w4hsx_5a983c1e-98ac-42ed-8d76-3138f331f106/web”,
“display_container_name:web_api-gateway-65478f6cb6-w4hsx”,
“env:production-us-east-1”,
“filename:0.log”,
“kube_cluster_name:production-us-east-1”,
“kube_container_name:web”,
“kube_deployment:api-gateway”,
“kube_namespace:default”,
“kube_replica_set:api-gateway-65478f6cb6”,
“kube_service:api-gateway”,
“pod_name:api-gateway-65478f6cb6-w4hsx”,
“pod_phase:running”,
“service:api-gateway”,
“short_image:api-gateway”,
“source:custom”
],
“host”: “i-0ec0f07450da3948f”,
“service”: “api-gateway”,
“attributes”: {
“msg”: “stapling OCSP”,
“logger”: {
“name”: “tls”
},
“level”: “warn”,
“error”: “no OCSP stapling for [community.dyme.app]: parsing OCSP response: ocsp: error from server: unauthorized”,
“ts”: 1615926247.046596
}
}
}
This is full log that I get from our logger, if this is not useful is there any better way to see logs inside docker container and I can share with u?

7

This topic was automatically closed after 30 days. New replies are no longer allowed.