How can I use Let's encrypt staging endpoint for certificates with the JSON config file?

1. Output of caddy version:

v2.6.1 h1:EDqo59TyYWhXQnfde93Mmv4FJfYe00dO60zMiEt+pzo=

2. How I run Caddy:

Docker on Linux

a. System environment:

Rocky Linux 8.6


Client: Docker Engine - Community
 Version:           20.10.18
 API version:       1.41
 Go version:        go1.18.6
 Git commit:        b40c2f6
 Built:             Thu Sep  8 23:11:56 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.18
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.6
  Git commit:       e42327a
  Built:            Thu Sep  8 23:10:04 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.8
  GitCommit:        9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

b. Command:

caddy run --config /cfg/caddy.json

c. Service/unit/compose file:

  caddy:
    image: docker.io/caddy
    container_name: caddy
    command: caddy run --config /cfg/caddy.json
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"
      - "2019:2019"
    volumes:
      - "/opt/volumes/caddy/_data:/data:Z"
      - "/opt/volumes/caddy/_config:/config:Z"
      - "/opt/volumes/caddy/config:/cfg:z"
      - "/opt/volumes/caddy/storage:/storage:z"
      - "/opt/volumes/caddy/certs:/certs:z"
    networks:
      - pod-local
      - pod-public

d. My complete Caddy config:

{
    "admin": {
        "disabled": false,
        "listen": "0.0.0.0:2019",
        "enforce_origin": false
    },
    "logging": {
        "sink": {
            "writer": {
                "output": "stdout"
            }
        }
    },
    "storage": {
            "module": "file_system",
            "root": "/storage"
    },
    "apps": {
        "tls": {
            "automation": {
                "policies": [
                    {
                        "subjects": ["*"],
                        "issuers": [
                            {
                                "module": "acme",
                                "ca": "https://acme-staging-v02.api.letsencrypt.org/directory"
                            }
                        ],
                        "storage": {
                            "module": "file_system",
                            "root": "/certs"
                        },
                        "key_type": "ed25519"
                    }
                ]
            }
        }
    }
}

3. The problem I’m having:

I’m trying to fetch certificates with the staging URL from Let’s Encrypt

4. Error messages and/or full log output:

Domain name is not relevent here.

[rocky@nabisipi caddy]$ docker run caddy caddy reverse-proxy --from https://8137860965.com --to webtest:3000
{"level":"warn","ts":1664750040.243026,"logger":"admin","msg":"admin endpoint disabled"}
{"level":"info","ts":1664750040.2431931,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"proxy","https_port":443}
{"level":"info","ts":1664750040.2432148,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"proxy"}
{"level":"info","ts":1664750040.243566,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0000f3b20"}
{"level":"info","ts":1664750040.2437458,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"info","ts":1664750040.243895,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details."}
{"level":"info","ts":1664750040.2440612,"logger":"http.log","msg":"server running","name":"proxy","protocols":["h1","h2","h3"]}
{"level":"info","ts":1664750040.2441492,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
{"level":"info","ts":1664750040.24416,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["8137860965.com"]}
{"level":"info","ts":1664750040.244514,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
{"level":"info","ts":1664750040.2445436,"logger":"tls","msg":"finished cleaning storage units"}
Caddy proxying https://8137860965.com -> webtest:3000
{"level":"info","ts":1664750040.2450132,"logger":"tls.obtain","msg":"acquiring lock","identifier":"8137860965.com"}
{"level":"info","ts":1664750040.2462323,"logger":"tls.obtain","msg":"lock acquired","identifier":"8137860965.com"}
{"level":"info","ts":1664750040.2464411,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"8137860965.com"}
{"level":"info","ts":1664750040.6320271,"logger":"http","msg":"waiting on internal rate limiter","identifiers":["8137860965.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":""}
{"level":"info","ts":1664750040.63207,"logger":"http","msg":"done waiting on internal rate limiter","identifiers":["8137860965.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":""}
{"level":"info","ts":1664750040.7878456,"logger":"http.acme_client","msg":"trying to solve challenge","identifier":"8137860965.com","challenge_type":"http-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
{"level":"error","ts":1664750041.1605318,"logger":"http.acme_client","msg":"challenge failed","identifier":"8137860965.com","challenge_type":"http-01","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"123.123.123.123: Fetching http://8137860965.com/.well-known/acme-challenge/gcPCmqNHpsBRvndGjfYhrNK4_igMGEYoRS5iiH6NuaQ: Connection refused","instance":"","subproblems":[]}}
{"level":"error","ts":1664750041.1605787,"logger":"http.acme_client","msg":"validating authorization","identifier":"8137860965.com","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"123.123.123.123: Fetching http://8137860965.com/.well-known/acme-challenge/gcPCmqNHpsBRvndGjfYhrNK4_igMGEYoRS5iiH6NuaQ: Connection refused","instance":"","subproblems":[]},"order":"https://acme-v02.api.letsencrypt.org/acme/order/757996646/130973344656","attempt":1,"max_attempts":3}
{"level":"info","ts":1664750042.312727,"logger":"http.acme_client","msg":"trying to solve challenge","identifier":"8137860965.com","challenge_type":"tls-alpn-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
{"level":"error","ts":1664750042.684173,"logger":"http.acme_client","msg":"challenge failed","identifier":"8137860965.com","challenge_type":"tls-alpn-01","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"123.123.123.123: Connection refused","instance":"","subproblems":[]}}
{"level":"error","ts":1664750042.6842291,"logger":"http.acme_client","msg":"validating authorization","identifier":"8137860965.com","problem":{"type":"urn:ietf:params:acme:error:connection","title":"","detail":"123.123.123.123: Connection refused","instance":"","subproblems":[]},"order":"https://acme-v02.api.letsencrypt.org/acme/order/757996646/130973349756","attempt":2,"max_attempts":3}
{"level":"error","ts":1664750042.684261,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"8137860965.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:connection - 123.123.123.123: Connection refused"}
{"level":"warn","ts":1664750042.6843982,"logger":"http","msg":"missing email address for ZeroSSL; it is strongly recommended to set one for next time"}
{"level":"info","ts":1664750043.1502469,"logger":"http","msg":"generated EAB credentials","key_id":"WHVEt2ON9nDeC8naDvHp3w"}
{"level":"info","ts":1664750058.0071847,"logger":"http","msg":"waiting on internal rate limiter","identifiers":["8137860965.com"],"ca":"https://acme.zerossl.com/v2/DV90","account":""}
{"level":"info","ts":1664750058.0072193,"logger":"http","msg":"done waiting on internal rate limiter","identifiers":["8137860965.com"],"ca":"https://acme.zerossl.com/v2/DV90","account":""}
{"level":"info","ts":1664750061.948888,"msg":"shutting down","signal":"SIGINT"}
{"level":"warn","ts":1664750061.949001,"msg":"exiting; byeee!! đź‘‹","signal":"SIGINT"}
{"level":"info","ts":1664750061.9494817,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0xc0000f3b20"}
{"level":"warn","ts":1664750061.9497657,"logger":"http.acme_client","msg":"HTTP request failed; retrying","url":"https://acme.zerossl.com/v2/DV90/newOrder","error":"performing request: Post \"https://acme.zerossl.com/v2/DV90/newOrder\": context canceled"}
{"level":"error","ts":1664750061.949819,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"8137860965.com","issuer":"acme.zerossl.com-v2-DV90","error":"[8137860965.com] creating new order: attempt 1: https://acme.zerossl.com/v2/DV90/newOrder: context canceled (ca=https://acme.zerossl.com/v2/DV90)"}
{"level":"info","ts":1664750061.9498565,"logger":"tls.obtain","msg":"releasing lock","identifier":"8137860965.com"}
{"level":"info","ts":1664750061.9497736,"msg":"shutdown complete","signal":"SIGINT","exit_code":0}

5. What I already tried:

I consulted the documentation. The Caddyfile global config is straightforward, but for JSON, the docs is harder to navigate for someone new - I looked for “tls” and “cert” configuration keywords but no matter the settings I put in, it’s always the main ca that is being used.

The forum showed a 2018 post I can’t seem to find again about someone asking a similar question, but the answer was incomplete.

With Caddyfile I had no issue making this works. I want all domains to use the staging server while I migrate from NGINX Proxy Manager to Caddy, get used on it’s config and decide on an approach on how to configure it.

6. Links to relevant resources:

The easiest way to figure out the right JSON config to use is to first start from a Caddyfile, then adapt it to JSON. It usually gives you a good starting point.

For example, this simple Caddyfile:

{
    acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
}

https://

Becomes this JSON:

{
	"apps": {
		"http": {
			"servers": {
				"srv0": {
					"listen": [
						":443"
					],
					"tls_connection_policies": [
						{}
					]
				}
			}
		},
		"tls": {
			"automation": {
				"policies": [
					{
						"issuers": [
							{
								"ca": "https://acme-staging-v02.api.letsencrypt.org/directory",
								"module": "acme"
							}
						]
					}
				]
			}
		}
	}
}

Is there any particular reason you’re using JSON? If you’re new to Caddy, you’ll have a much better time using the Caddyfile instead, it’s easier to use.

Also, all this said, you don’t necessarily need to configure the staging endpoint. Caddy will automatically fall back to using the staging endpoint after the first failure, until it gets a success with the staging endpoint at which point it will then retry with the production endpoint. See the docs on Automatic HTTPS:

2 Likes

Hi! Sorry for the late reply. I wanted to use JSON to add routes without having to restart caddy :confused: But I think your idea of using Caddyfile for now to get myself familiar with it is not a bad idea :joy:

I tried converting like you did, but I omitted the https: part, so the json file was empty.

You don’t need to restart Caddy, even when using the Caddyfile. You can gracefully reload Caddy with any config. We have the recommended command here in the docs for how to reload the config using Docker Compose (same idea if you’re not using Compose, though):

Please elaborate. What’s your current config? What are you seeing? We can’t do much unless you’re specific.

1 Like

This topic was automatically closed after 30 days. New replies are no longer allowed.