How to make idempotent config changes via API?

[coolaj86 (AJ ONeal)]

1. The problem I’m having:

I want idempotent config changes - meaning that I can “upsert” config options, probably by @id, especially from a config script.

@ids don’t exist before they’re created and most things in the configs are arrays and the index also can’t be known ahead of time (or afterwards).

It seems that if the item has an @id, or the data is a primitive that’s exactly the same, it should respect that there was no change, or update the thing by that @id in the array.

Examples:

Adding a primitive value to an array which is semantically a set:

fn_add_tls_automation() { (
    #my_lxc_id="${1:-}"
    my_lxc_domain="${2:-}"

    curl -fsS --proto '=https' --tlsv1.2 \
        -u "$CADDY_USER:$CADDY_PASS" \
        -X POST \
        "${CADDY_HOST}/config/apps/tls/certificates/automate/..." \
        -H "Content-Type: application/json" \
        --data-binary '["'"$my_lxc_domain"'"]'
); }

Adding to the array, but if the item already exists, it should just be replaced:

fn_add_tls_policy() { (
    my_lxc_id="${1:-}"
    my_lxc_domain="${2:-}"

    curl -fsS --proto '=https' --tlsv1.2 \
        -u "$CADDY_USER:$CADDY_PASS" \
        -X POST \
        "${CADDY_HOST}/config/apps/tls/automation/policies/..." \
        -H "Content-Type: application/json" \
        --data-binary '
          [{
            "@id": "'"$my_lxc_id"'_tls_policy",
            "subjects": [ "'"$my_lxc_domain"'" ]
          }]
        '
); }

It would be very complex to have to try to traverse the config for several different arrays (TLS, TLS Policy, L4 Routing, HTTP server)

I think another solution would be to use @id in place of ... to target a specific item to append or update:

    my_id="${my_lxc_id}_tls_policy"

    curl -fsS --proto '=https' --tlsv1.2 \
        -u "$CADDY_USER:$CADDY_PASS" \
        -X POST \
        "${CADDY_HOST}/config/apps/tls/automation/policies/@${my_id}" \
        -H "Content-Type: application/json" \
        --data-binary '
          {
            "@id": "'"$my_id"'",
            "subjects": [ "'"$my_lxc_domain"'" ]
          }
        '

2. Error messages and/or full log output:

< HTTP/1.1 500 Internal Server Error
< Content-Length: 262
< Content-Type: application/json
< Date: Sat, 01 Apr 2023 06:10:41 GMT
< Server: Caddy
* The requested URL returned error: 500
* Closing connection 0
curl: (22) The requested URL returned error: 500

Apr 01 06:12:05 caddy caddy[3312]: {"level":"error","ts":1680329525.418667,"logger":"admin.api","msg":"request error","error":"loading new config: loading http app module: provision http: getting tls app: loading tls app module: tls: invalid configuration: automation policy 6: cannot apply more than one automation policy to host: lxc104.duckdns.org (first match in policy 3)","status_code":500}

3. Caddy version:

v2.6.4 h1:2hwYqiRwk1tf3VruhMpLcYTg+11fCdr8S3jhNAdnPy8=

4. How I installed and ran Caddy:

a. System environment:

Ubuntu Cloud Server

b. Command:

caddy run --envfile ~/srv/caddy/.env --resume

c. Service/unit/compose file:

N/A

5. Links to relevant resources:

therootcompany/proxmox-sh/blob/main/caddy-add-lxc