1. Caddy version (caddy version
): v2.1.1 h1:X9k1+ehZPYYrSqBvf/ocUgdLSRIuiNiMo7CvyGUQKeA=
2. How I run Caddy:
a. System environment:
Linux rishighan 4.15.0-117-generic #118-Ubuntu SMP Fri Sep 4 20:02:41 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
b. Command:
#!/bin/bash
# Rishi Ghan
# Deployment script for a docker-composed project
# usage: ./deploy.sh -s [service name]
# -h [hostname]
# -u [username]
# -r [repository base url to the raw text content]
# Emojis
CLIPBOARD="π"
CHECKMARK="β
"
RENAME="π·οΈ"
SCISSORS="βοΈ"
DOWNLOAD="π₯"
BROOM="π§Ή"
CONSTRUCTION="ποΈ"
START="π"
# ssh config
cat >> ~/.ssh/config << EOF
VerifyHostKeyDNS yes
StrictHostKeyChecking no
EOF
# params
service_name=''
hostname=''
username=''
while getopts 'd:s:h:u:r:x:p:b:o:' flag; do
case "${flag}" in
s) service_name="${OPTARG}" ;;
h) hostname="${OPTARG}" ;;
u) username="${OPTARG}" ;;
r) repository_base_url="${OPTARG}" ;;
x) zookeeper_client_user="${OPTARG}" ;;
p) zookeeper_client_password="${OPTARG}" ;;
b) mongodb_database="${OPTARG}" ;;
o) mongodb_root_password="${OPTARG}" ;;
*) printf "Usage..."
exit 1 ;;
esac
done
printf "$CLIPBOARD Attempting to create configuration folder...\n"
ssh "$username@$hostname" /bin/bash << EOF
if [ ! -d "$service_name" ]
then
printf "\n$CLIPBOARD Directory doesn't exist. Creating now...\n"
mkdir "$service_name"
printf "$service_name created."
else
printf "\n$RENAME $service_name already exists. Removing and recreating...\n"
rm -Rf "$service_name"
mkdir "$service_name"
printf "$CHECKMARK Done.\n"
fi
printf "\n$CLIPBOARD Changing directory to $service_name...\n"
cd "$service_name"
printf "\n$SCISSORS Pruning Docker images, networks and volumes...\n\n"
docker system prune -f
printf "\n$CONSTRUCTION Creating an external docker network...\n\n"
docker network create main-network
printf "$DOWNLOAD Downloading the docker-compose configuration for $service_name...\n\n"
printf "$repository_base_url\n\n"
curl "$repository_base_url"/docker-compose.yml --output docker-compose.yml
curl "$repository_base_url"/docker-compose.env --output docker-compose.env
curl "$repository_base_url"/Caddyfile --output Caddyfile
printf "Writing Zookeeper configuration to docker-compose.env... \n"
echo -e "\n" >> docker-compose.env
echo -e "ZOOKEEPER_CLIENT_USER=$zookeeper_client_user" >> docker-compose.env
echo -e "ZOOKEEPER_CLIENT_PASSWORD=$zookeeper_client_password" >> docker-compose.env
printf "Writing Mongo configuration to docker-compose.env... \n"
echo -e "MONGODB_DATABASE=$mongodb_database" >> docker-compose.env
echo -e "MONGODB_PASSWORD=$mongodb_root_password" >> docker-compose.env
printf "\n$BROOM Stopping and removing containers and volumes...\n\n"
docker-compose down -v
printf "\n$DOWNLOAD Pulling the relevant Docker images...\n\n"
docker-compose pull
printf "\n$CONSTRUCTION Creating containers...\n\n"
docker-compose up --no-start
printf "\n$START Starting images...\n\n"
docker-compose start
EOF
c. Service/unit/compose file:
version: "3.3"
services:
nats:
image: nats
container_name: nats
networks:
- main-network
caddy:
image: caddy
container_name: caddy
restart: always
ports:
- "443:443"
- "80:80"
networks:
- main-network
volumes:
- "./Caddyfile:/etc/caddy/Caddyfile"
- $PWD/.caddy/data:/data
- $PWD/.caddy/config:/config
mongo:
image: bitnami/mongodb
container_name: db
restart: always
env_file: docker-compose.env
ports:
- "27017:27017"
volumes:
- /usr/local/var/mongodb/:/bitnami/mongodb
networks:
- main-network
zookeeper:
image: 'bitnami/zookeeper:latest'
container_name: zookeeper
env_file: docker-compose.env
ports:
- '2181:2181'
volumes:
- ../zookeeper/data:/bitnami/zookeeper
networks:
- main-network
networks:
main-network:
external: true
d. My complete Caddyfile or JSON config:
{
# Global options block. Entirely optional, https is on by default
email boo@foo.com
acme_ca https://acme.zerossl.com/v2/DV90
acme_eab {
key_id <key_id>
hmac <hmac>
}
}
rishighan.com {
reverse_proxy rishighan:8999
}
posts.services.rishighan.com {
reverse_proxy posts-service-api:5000
}
user.services.rishighan.com {
reverse_proxy user-service-api:3456
}
assets.services.rishighan.com {
reverse_proxy assets-service-api:6000
}
analytics.services.rishighan.com {
reverse_proxy analytics-service-api:9876
}
3. The problem Iβm having:
Caddy is configured as a docker image in my docker-compose
file. I deploy to Digital Ocean, where a script spins up these images.
The caddy image works perfectly fine, generating certificates for the configured domains only with fresh EAB credentials. If I make changes to my project and re-deploy with the same EAB credentials, I get tls handshake error
and not awaiting EAB credentials
errors. (See logs below)
I use ZeroSSL as my certificate provider.
4. Error messages and/or full log output:
{"level":"info","ts":1600197993.0440996,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"info","ts":1600197993.0478976,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["127.0.0.1:2019","localhost:2019","[::1]:2019"]}
{"level":"info","ts":1600197993.0482848,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1600197993.0483599,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
2020/09/15 19:26:33 [INFO][cache:0xc000494660] Started certificate maintenance routine
{"level":"info","ts":1600197993.048977,"logger":"tls","msg":"cleaned up storage units"}
{"level":"info","ts":1600197993.0491297,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["posts.services.rishighan.com","user.services.rishighan.com","rishighan.com","analytics.services.rishighan.com","assets.services.rishighan.com"]}
2020/09/15 19:26:33 [INFO][posts.services.rishighan.com] Obtain certificate; acquiring lock...
2020/09/15 19:26:33 [INFO][posts.services.rishighan.com] Obtain: Lock acquired; proceeding...
{"level":"info","ts":1600197993.05156,"msg":"autosaved config","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1600197993.051571,"msg":"serving initial configuration"}
2020/09/15 19:26:33 [INFO][user.services.rishighan.com] Obtain certificate; acquiring lock...
2020/09/15 19:26:33 [INFO][user.services.rishighan.com] Obtain: Lock acquired; proceeding...
2020/09/15 19:26:33 [INFO][rishighan.com] Obtain certificate; acquiring lock...
2020/09/15 19:26:33 [INFO][rishighan.com] Obtain: Lock acquired; proceeding...
2020/09/15 19:26:33 [INFO][analytics.services.rishighan.com] Obtain certificate; acquiring lock...
2020/09/15 19:26:33 [INFO][analytics.services.rishighan.com] Obtain: Lock acquired; proceeding...
2020/09/15 19:26:33 [INFO][assets.services.rishighan.com] Obtain certificate; acquiring lock...
2020/09/15 19:26:33 [INFO][assets.services.rishighan.com] Obtain: Lock acquired; proceeding...
2020/09/15 19:26:33 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:26:34 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 1/2)
2020/09/15 19:26:34 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:26:35 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 1/2)
2020/09/15 19:26:35 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:26:35 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 1/2)
2020/09/15 19:26:35 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:26:36 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 1/2)
2020/09/15 19:26:36 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:26:36 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 1/2)
2020/09/15 19:26:36 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:26:37 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 2/2)
2020/09/15 19:26:37 [ERROR] attempt 1: [user.services.rishighan.com] Obtain: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: - retrying in 1m0s (4.119763151s/720h0m0s elapsed)...
2020/09/15 19:26:37 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:26:37 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 2/2)
2020/09/15 19:26:37 [ERROR] attempt 1: [posts.services.rishighan.com] Obtain: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: - retrying in 1m0s (4.619054731s/720h0m0s elapsed)...
2020/09/15 19:26:37 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:26:38 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 2/2)
2020/09/15 19:26:38 [ERROR] attempt 1: [rishighan.com] Obtain: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: - retrying in 1m0s (5.124534482s/720h0m0s elapsed)...
2020/09/15 19:26:38 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:26:38 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 2/2)
2020/09/15 19:26:38 [ERROR] attempt 1: [analytics.services.rishighan.com] Obtain: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: - retrying in 1m0s (5.6264752s/720h0m0s elapsed)...
2020/09/15 19:26:38 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:26:39 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 2/2)
2020/09/15 19:26:39 [ERROR] attempt 1: [assets.services.rishighan.com] Obtain: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: - retrying in 1m0s (6.128027819s/720h0m0s elapsed)...
2020/09/15 19:27:37 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:27:37 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 1/2)
2020/09/15 19:27:37 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:27:38 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 1/2)
2020/09/15 19:27:38 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:27:38 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 1/2)
2020/09/15 19:27:38 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:27:39 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 1/2)
2020/09/15 19:27:39 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:27:39 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 1/2)
2020/09/15 19:27:39 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:27:40 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 2/2)
2020/09/15 19:27:40 [ERROR] attempt 2: [user.services.rishighan.com] Obtain: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: - retrying in 2m0s (1m7.444232566s/720h0m0s elapsed)...
2020/09/15 19:27:40 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:27:41 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 2/2)
2020/09/15 19:27:41 [ERROR] attempt 2: [posts.services.rishighan.com] Obtain: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: - retrying in 2m0s (1m7.957014777s/720h0m0s elapsed)...
2020/09/15 19:27:41 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:27:41 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 2/2)
2020/09/15 19:27:41 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:27:41 [ERROR] attempt 2: [rishighan.com] Obtain: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: - retrying in 2m0s (1m8.453652409s/720h0m0s elapsed)...
2020/09/15 19:27:42 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 2/2)
2020/09/15 19:27:42 [ERROR] attempt 2: [analytics.services.rishighan.com] Obtain: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: - retrying in 2m0s (1m8.978335448s/720h0m0s elapsed)...
2020/09/15 19:27:42 [INFO] acme: Registering account for frishi@me.com
2020/09/15 19:27:42 [ERROR] Making new ACME client: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: (attempt 2/2)
2020/09/15 19:27:42 [ERROR] attempt 2: [assets.services.rishighan.com] Obtain: acme: error: 400 :: POST :: https://acme.zerossl.com/v2/DV90/newAccount :: urn:ietf:params:acme:error:malformed :: [External Account Binding] The account is not awaiting external account binding, url: - retrying in 2m0s (1m9.464931251s/720h0m0s elapsed)...
2020/09/15 19:28:07 http: TLS handshake error from 17.149.234.204:59779: no certificate available for 'rishighan.com'
5. What I already tried:
I have mapped caddy config and data folders in an effort to persist the certificates, but upon checking the contents of the folder inside the docker container, I found that both the config
and data
folders are empty.
The only solution that works for me is generating fresh EAB credentials in the ZeroSSL dashboard and re-deploying.