Running caddy under docker with commercial certs where ports 80 and 443 are NOT available to caddy

Help
1

1. The problem I’m having:

I am running Apache on EC2 where ports 80 and 443 are taken. Caddy cannot use them. I am trying to [1] Use my official signed certs and [2] launch Caddy via docker compose on the same machine running Apache. Caddy launches, I see listeners on ports 7080- and 1443 but nothing responds on

CONTAINER ID   NAMES                          PORTS
89b7ef24c5de   invenio-rdm-starter-proxy-1    443/udp, 2019/tcp, 0.0.0.0:7080->80/tcp, [::]:7080->80/tcp, 0.0.0.0:1443->443/tcp, [::]:1443->443/tcp
e972ab0efb13   invenio-rdm-starter-web-1      5000/tcp
fba21a7f2b38   invenio-rdm-starter-worker-1   5000/tcp
b0b3d771f603   invenio-rdm-starter-db-1       0.0.0.0:5432->5432/tcp, :::5432->5432/tcp
7b7fcb450aca   invenio-rdm-starter-cache-1    0.0.0.0:6379->6379/tcp, :::6379->6379/tcp
207ab0890c82   invenio-rdm-starter-search-1   0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 9300/tcp, 0.0.0.0:9600->9600/tcp, :::9600->9600/tcp, 9650/tcp
-->

2. Error messages and/or full log output:

<!--
 docker logs 89b7ef24c5de
{"level":"info","ts":1731548413.814979,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
{"level":"info","ts":1731548413.8200822,"msg":"adapted config to JSON","adapter":"caddyfile"}
{"level":"warn","ts":1731548413.8212967,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
{"level":"info","ts":1731548413.8244894,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1731548413.8253832,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x4000714980"}
{"level":"info","ts":1731548414.1799903,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
{"level":"info","ts":1731548414.1800575,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv1"}
{"level":"info","ts":1731548414.1814866,"logger":"http","msg":"enabling HTTP/3 listener","addr":":1443"}
{"level":"info","ts":1731548414.1816676,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details."}
{"level":"info","ts":1731548414.1818423,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1731548414.181947,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"info","ts":1731548414.1821587,"logger":"http.log","msg":"server running","name":"srv1","protocols":["h1","h2","h3"]}
{"level":"info","ts":1731548414.1825533,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1731548414.1825926,"msg":"serving initial configuration"}
{"level":"info","ts":1731548414.1925423,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"62804943-2f3d-4ee8-8fd5-77ad8e1e1906","try_again":1731634814.1924639,"try_again_in":86399.99999916}
{"level":"info","ts":1731548414.1927273,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1731549023.6854749,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
{"level":"warn","ts":1731549023.6855683,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
{"level":"info","ts":1731549023.6856172,"logger":"http","msg":"servers shutting down with eternal grace period"}
{"level":"info","ts":1731549023.6861906,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
{"level":"info","ts":1731549023.6862235,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
{"level":"info","ts":1731549037.7405841,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
{"level":"info","ts":1731549037.7535276,"msg":"adapted config to JSON","adapter":"caddyfile"}
{"level":"warn","ts":1731549037.7536101,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
{"level":"info","ts":1731549037.758018,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1731549037.7675722,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x400047af80"}
{"level":"info","ts":1731549037.769165,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
{"level":"info","ts":1731549037.7692447,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv1"}
{"level":"info","ts":1731549037.7750409,"logger":"http","msg":"enabling HTTP/3 listener","addr":":1443"}
{"level":"info","ts":1731549037.7773721,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details."}
{"level":"info","ts":1731549037.7777345,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1731549037.7778757,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"info","ts":1731549037.7788632,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"62804943-2f3d-4ee8-8fd5-77ad8e1e1906","try_again":1731635437.7788603,"try_again_in":86399.999999172}
{"level":"info","ts":1731549037.7790785,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1731549037.77978,"logger":"http.log","msg":"server running","name":"srv1","protocols":["h1","h2","h3"]}
{"level":"info","ts":1731549037.7800925,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1731549037.7801151,"msg":"serving initial configuration"}
{"level":"info","ts":1731549043.752711,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
{"level":"warn","ts":1731549043.752748,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
{"level":"info","ts":1731549043.7528157,"logger":"http","msg":"servers shutting down with eternal grace period"}
{"level":"info","ts":1731549043.7532938,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
{"level":"info","ts":1731549043.7533178,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
{"level":"info","ts":1731549061.0222008,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
{"level":"info","ts":1731549061.025246,"msg":"adapted config to JSON","adapter":"caddyfile"}
{"level":"warn","ts":1731549061.0258512,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
{"level":"info","ts":1731549061.0307374,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1731549061.0327919,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x400054a280"}
{"level":"info","ts":1731549061.0358374,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
{"level":"info","ts":1731549061.0368738,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv1"}
{"level":"info","ts":1731549061.0377018,"logger":"http","msg":"enabling HTTP/3 listener","addr":":1443"}
{"level":"info","ts":1731549061.0391169,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details."}
{"level":"info","ts":1731549061.0411649,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1731549061.0414407,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"info","ts":1731549061.0418816,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"62804943-2f3d-4ee8-8fd5-77ad8e1e1906","try_again":1731635461.0418785,"try_again_in":86399.999999112}
{"level":"info","ts":1731549061.0420423,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1731549061.043643,"logger":"http.log","msg":"server running","name":"srv1","protocols":["h1","h2","h3"]}
{"level":"info","ts":1731549061.0448604,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1731549061.0448844,"msg":"serving initial configuration"}
{"level":"info","ts":1731549443.1985216,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
{"level":"warn","ts":1731549443.1985748,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
{"level":"info","ts":1731549443.1986146,"logger":"http","msg":"servers shutting down with eternal grace period"}
{"level":"info","ts":1731549443.4658566,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
{"level":"info","ts":1731549443.4659243,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
{"level":"info","ts":1731549476.4998822,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Error: adapting config using caddyfile: server block without any key is global configuration, and if used, it must be first
{"level":"info","ts":1731549478.3067036,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Error: adapting config using caddyfile: server block without any key is global configuration, and if used, it must be first
{"level":"info","ts":1731549479.9966707,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Error: adapting config using caddyfile: server block without any key is global configuration, and if used, it must be first
{"level":"info","ts":1731549481.839985,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Error: adapting config using caddyfile: server block without any key is global configuration, and if used, it must be first
{"level":"info","ts":1731549484.0899343,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Error: adapting config using caddyfile: server block without any key is global configuration, and if used, it must be first
{"level":"info","ts":1731549487.1564856,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Error: adapting config using caddyfile: server block without any key is global configuration, and if used, it must be first
{"level":"info","ts":1731549491.5785766,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Error: adapting config using caddyfile: server block without any key is global configuration, and if used, it must be first
{"level":"info","ts":1731549499.36514,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Error: adapting config using caddyfile: server block without any key is global configuration, and if used, it must be first
{"level":"info","ts":1731549513.335468,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Error: adapting config using caddyfile: server block without any key is global configuration, and if used, it must be first
{"level":"info","ts":1731549539.5761023,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Error: adapting config using caddyfile: server block without any key is global configuration, and if used, it must be first
{"level":"info","ts":1731549573.0143125,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
{"level":"info","ts":1731549573.0209274,"msg":"adapted config to JSON","adapter":"caddyfile"}
{"level":"warn","ts":1731549573.020969,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
{"level":"info","ts":1731549573.060774,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]}
{"level":"info","ts":1731549573.0615127,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x4000526500"}
{"level":"info","ts":1731549573.070833,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
{"level":"info","ts":1731549573.071009,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv1"}
{"level":"info","ts":1731549573.0715191,"logger":"http","msg":"enabling HTTP/3 listener","addr":":1443"}
{"level":"info","ts":1731549573.0739725,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details."}
{"level":"info","ts":1731549573.074645,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1731549573.0775244,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"info","ts":1731549573.0777855,"logger":"http.log","msg":"server running","name":"srv1","protocols":["h1","h2","h3"]}
{"level":"info","ts":1731549573.0794752,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1731549573.0796957,"msg":"serving initial configuration"}
{"level":"info","ts":1731549573.0800788,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"62804943-2f3d-4ee8-8fd5-77ad8e1e1906","try_again":1731635973.080076,"try_again_in":86399.999999088}
{"level":"info","ts":1731549573.0807061,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1731549665.4211237,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
{"level":"warn","ts":1731549665.4211621,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
{"level":"info","ts":1731549665.4212484,"logger":"http","msg":"servers shutting down with eternal grace period"}
{"level":"info","ts":1731549665.4216416,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
{"level":"info","ts":1731549665.4216619,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
{"level":"info","ts":1731549683.469618,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
{"level":"info","ts":1731549683.472135,"msg":"adapted config to JSON","adapter":"caddyfile"}
{"level":"warn","ts":1731549683.4721746,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
{"level":"info","ts":1731549683.4792776,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1731549683.4815862,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x40003ef300"}
{"level":"info","ts":1731549683.483854,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
{"level":"info","ts":1731549683.4841745,"logger":"http","msg":"enabling HTTP/3 listener","addr":":1443"}
{"level":"info","ts":1731549683.4843063,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details."}
{"level":"info","ts":1731549683.4853306,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1731549683.4856603,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1731549683.4856808,"msg":"serving initial configuration"}
{"level":"info","ts":1731549683.4893281,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"62804943-2f3d-4ee8-8fd5-77ad8e1e1906","try_again":1731636083.489325,"try_again_in":86399.999998836}
{"level":"info","ts":1731549683.4896302,"logger":"tls","msg":"finished cleaning storage units"}

-->

3. Caddy version:

caddy:2.8.1

4. How I installed and ran Caddy:

It is pulled as part of a docker compose run. See below for docker-compose.yml

a. System environment:

EC2 with Amazon Linux.
The security groups are OPEN (for now) from 0.0.0.0/0 → 7080 and from 0.0.0.0/0->1443

Linux 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 17:10:38 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

Docker version 27.3.1, build ce12230

hostname command A record on internet resolves to 3.67.17.69

b. Command:

Run as part of overall bring up via compose.
docker compose up -d

c. Service/unit/compose file:

# InvenioDRM Starter Docker Compose
#
# Usage::
#
#   $ docker compose up -d
#
# The following services are included:
# - Reverse proxy: Caddy (exposed ports: 80, 443)
# - Web application: Gunicorn (exposed port: none)
# - DB: (PostgresSQL) (exposed port: 5432)
# - Cache and message queue: Redis (exposed port: 6379)
# - Search platform: (OpenSearch) (exposed ports: 9200, 9300)

services:
  proxy:
    image: caddy:2.8.1
    restart: "unless-stopped"
    ports:
      - "7080:80"
      - "1443:443"
    volumes:
      - /etc/ssl/2024-Oct:/etc/caddy/certs
      - ./Caddyfile:/etc/caddy/Caddyfile
      - caddy_data:/data
      - caddy_config:/config
  web:
    image: ghcr.io/front-matter/invenio-rdm-starter:latest
    pull_policy: if_not_present
    volumes:
      - uploaded_data:/opt/invenio/var/instance/data
      - archived_data:/opt/invenio/var/instance/archive
    environment:
      # Flask
      - INVENIO_APP_ALLOWED_HOSTS=${INVENIO_APP_ALLOWED_HOSTS:-['0.0.0.0', 'localhost', '127.0.0.1']}

      # Flask-SQLAlchemy
      - INVENIO_SQLALCHEMY_DATABASE_URI=${INVENIO_SQLALCHEMY_DATABASE_URI:-postgresql+psycopg2://inveniordm:inveniordm@db/inveniordm}

      # Flask-Babel
      - INVENIO_BABEL_DEFAULT_LOCALE=${INVENIO_BABEL_DEFAULT_LOCALE:-en}
      - INVENIO_BABEL_DEFAULT_TIMEZONE=${INVENIO_BABEL_DEFAULT_TIMEZONE:-UTC}

      # Invenio-App
      - INVENIO_CACHE_TYPE=${INVENIO_CACHE_TYPE:-redis}
      - INVENIO_CACHE_REDIS_URL=${INVENIO_CACHE_REDIS_URL:-redis://cache:6379/0}
      - INVENIO_ACCOUNTS_SESSION_REDIS_URL=${INVENIO_ACCOUNTS_SESSION_REDIS_URL:-redis://cache:6379/1}
      - INVENIO_CELERY_RESULT_BACKEND=${INVENIO_CELERY_RESULT_BACKEND:-redis://cache:6379/2}
      - INVENIO_RATELIMIT_STORAGE_URL=${INVENIO_RATELIMIT_STORAGE_URL:-redis://cache:6379/3}
      - INVENIO_COMMUNITIES_IDENTITIES_CACHE_REDIS_URL=${INVENIO_COMMUNITIES_IDENTITIES_CACHE_REDIS_URL:-redis://cache:6379/4}
      - INVENIO_BROKER_URL=${INVENIO_BROKER_URL:-redis://cache:6379/5}
      - INVENIO_CELERY_BROKER_URL=${INVENIO_BROKER_URL:-redis://cache:6379/5}

      - INVENIO_WSGI_PROXIES=4
      - INVENIO_SECRET_KEY=${INVENIO_SECRET_KEY:-changeme}

      # Invenio-I18N
      # TODO: fix tuple parsing
      # - INVENIO_I18N_LANGUAGES=${INVENIO_I18N_LANGUAGES:-'[("de", _("German"))]'}

      # Invenio-Theme
      - INVENIO_THEME_LOGO=${INVENIO_THEME_LOGO:-images/invenio-rdm-white.svg}
      - INVENIO_THEME_SITENAME=${INVENIO_THEME_SITENAME:-InvenioRDM Starter}
      - INVENIO_THEME_FRONTPAGE_TITLE=${INVENIO_THEME_FRONTPAGE_TITLE:-InvenioRDM Starter}
      - INVENIO_THEME_FRONTPAGE_SUBTITLE=${INVENIO_THEME_FRONTPAGE_SUBTITLE:-A starter project for the turn-key research data management repository.}
      - INVENIO_THEME_SHOW_FRONTPAGE_INTRO_SECTION=False

      # Invenio-Records-Resources
      - INVENIO_SITE_UI_URL=${INVENIO_SITE_UI_URL:-https://localhost}
      - INVENIO_SITE_API_URL=${INVENIO_SITE_API_URL:-https://localhost/api}

      # Invenio-RDM-Records
      - INVENIO_DATACITE_ENABLED=${INVENIO_DATACITE_ENABLED:-False}
      - INVENIO_DATACITE_USERNAME=${INVENIO_DATACITE_USERNAME:-}
      - INVENIO_DATACITE_PASSWORD=${INVENIO_DATACITE_PASSWORD:-}
      - INVENIO_DATACITE_PREFIX=${INVENIO_DATACITE_PREFIX:-}
      - INVENIO_DATACITE_TEST_MODE=${INVENIO_DATACITE_TEST_MODE:-True}
      - INVENIO_DATACITE_DATACENTER_SYMBOL=${INVENIO_DATACITE_DATACENTER_SYMBOL:-}

      - INVENIO_RDM_ALLOW_METADATA_ONLY_RECORDS=${INVENIO_RDM_ALLOW_METADATA_ONLY_RECORDS:-True}
      - INVENIO_RDM_ALLOW_RESTRICTED_RECORDS=${INVENIO_RDM_ALLOW_RESTRICTED_RECORDS:-True}
      - INVENIO_RDM_ALLOW_EXTERNAL_DOI_VERSIONING=${INVENIO_RDM_ALLOW_EXTERNAL_DOI_VERSIONING:-True}

      - INVENIO_RDM_CITATION_STYLES_DEFAULT=${INVENIO_RDM_CITATION_STYLES_DEFAULT:-apa}
      - INVENIO_SECURITY_EMAIL_SENDER=${INVENIO_SECURITY_EMAIL_SENDER:-}
      # TODO: fix tuple parsing
      # - INVENIO_RDM_CITATION_STYLES=${INVENIO_RDM_CITATION_STYLES:-"[('apa', 'APA'), ('chicago-note-bibliography', 'Chicago'), ('harvard-cite-them-right', 'Harvard')]"}

      # Invenio-Accounts
      - INVENIO_ACCOUNTS_LOCAL_LOGIN_ENABLED=${INVENIO_ACCOUNTS_LOCAL_LOGIN_ENABLED:-True}

      # OAI-PMH
      - INVENIO_OAISERVER_ID_PREFIX=${OAISERVER_ID_PREFIX:-invenio-rdm}

      # Invenio-Files-REST
      - INVENIO_FILES_REST_STORAGE_FACTORY=invenio_s3.s3fs_storage_factory

      # Invenio-S3
      - INVENIO_S3_ENDPOINT_URL=${INVENIO_S3_ENDPOINT_URL:-}
      - INVENIO_S3_ACCESS_KEY_ID=${INVENIO_S3_ACCESS_KEY_ID:-}
      - INVENIO_S3_SECRET_ACCESS_KEY=${INVENIO_S3_SECRET_ACCESS_KEY:-}
      - INVENIO_S3_BUCKET_NAME=${INVENIO_S3_BUCKET_NAME:-}

      # Invenio-Search
      - INVENIO_SEARCH_HOSTS=${INVENIO_SEARCH_HOSTS:-['search:9200']}
      - INVENIO_SEARCH_INDEX_PREFIX=${INVENIO_SEARCH_INDEX_PREFIX:-invenio-rdm-}

      # Logging
      - INVENIO_LOGGING_CONSOLE_LEVEL=${INVENIO_LOGGING_CONSOLE_LEVEL:-WARNING}
    depends_on:
      search:
        condition: service_started
      cache:
        condition: service_started
      db:
        condition: service_started
  worker:
    command: "celery -A invenio_app.celery worker --beat --events --loglevel=WARNING"
    image: ghcr.io/front-matter/invenio-rdm-starter:latest
    pull_policy: if_not_present
    volumes:
      - uploaded_data:/opt/invenio/var/instance/data
    environment:
      - INVENIO_SQLALCHEMY_DATABASE_URI=${INVENIO_SQLALCHEMY_DATABASE_URI:-postgresql+psycopg2://inveniordm:inveniordm@db/inveniordm}
      - INVENIO_SEARCH_HOSTS=${INVENIO_SEARCH_HOSTS:-['search:9200']}
      - INVENIO_SEARCH_INDEX_PREFIX=${INVENIO_SEARCH_INDEX_PREFIX:-invenio-rdm-}

      - INVENIO_CACHE_TYPE=${INVENIO_CACHE_TYPE:-redis}
      - INVENIO_CACHE_REDIS_URL=${INVENIO_CACHE_REDIS_URL:-redis://cache:6379/0}
      - INVENIO_ACCOUNTS_SESSION_REDIS_URL=${INVENIO_ACCOUNTS_SESSION_REDIS_URL:-redis://cache:6379/1}
      - INVENIO_CELERY_RESULT_BACKEND=${INVENIO_CELERY_RESULT_BACKEND:-redis://cache:6379/2}
      - INVENIO_RATELIMIT_STORAGE_URL=${INVENIO_RATELIMIT_STORAGE_URL:-redis://cache:6379/3}
      - INVENIO_COMMUNITIES_IDENTITIES_CACHE_REDIS_URL=${INVENIO_COMMUNITIES_IDENTITIES_CACHE_REDIS_URL:-redis://cache:6379/4}
      - INVENIO_BROKER_URL=${INVENIO_BROKER_URL:-redis://cache:6379/5}
      - INVENIO_CELERY_BROKER_URL=${INVENIO_BROKER_URL:-redis://cache:6379/5}

      - INVENIO_MAIL_SUPPRESS_SEND=${INVENIO_MAIL_SUPPRESS_SEND:-True}
      - INVENIO_MAIL_SERVER=${INVENIO_MAIL_SERVER:-smtp.gmail.com}
      - INVENIO_MAIL_PORT=${INVENIO_MAIL_PORT:-465}
      - INVENIO_MAIL_USERNAME=${INVENIO_MAIL_USERNAME:-info}
      - INVENIO_MAIL_PASSWORD=${INVENIO_MAIL_PASSWORD:-changeme}
      - INVENIO_MAIL_USE_SSL=${INVENIO_MAIL_USE_SSL:-True}

      # Invenio-Files-REST
      - INVENIO_FILES_REST_STORAGE_FACTORY=${INVENIO_FILES_REST_STORAGE_FACTORY:-invenio_s3.s3fs_storage_factory}

      # Invenio-S3
      - INVENIO_S3_ENDPOINT_URL=${INVENIO_S3_ENDPOINT_URL:-}
      - INVENIO_S3_ACCESS_KEY_ID=${INVENIO_S3_ACCESS_KEY_ID:-}
      - INVENIO_S3_SECRET_ACCESS_KEY=${INVENIO_S3_SECRET_ACCESS_KEY:-}
      - INVENIO_S3_BUCKET_NAME=${INVENIO_S3_BUCKET_NAME:-}
    depends_on:
      search:
        condition: service_started
      cache:
        condition: service_started
      db:
        condition: service_started
  cache:
    image: valkey/valkey:7.2.5-bookworm
    restart: "unless-stopped"
    ports:
      - "6379:6379"
  db:
    image: postgres:16.4-bookworm
    restart: "unless-stopped"
    environment:
      - POSTGRES_USER=${POSTGRES_USER:-inveniordm}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-inveniordm}
      - POSTGRES_DB=${POSTGRES_DB:-inveniordm}
    ports:
      - 5432:5432
  search:
    image: opensearchproject/opensearch:2.9.0
    restart: "unless-stopped"
    environment:
      - bootstrap.memory_lock=true
      - OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m
      - DISABLE_INSTALL_DEMO_CONFIG=true
      - DISABLE_SECURITY_PLUGIN=true
      - discovery.type=single-node
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    mem_limit: 2g
    ports:
      - "9200:9200"
      - "9600:9600"
volumes:
  app_data:
  uploaded_data:
  archived_data:
  caddy_data:
  caddy_config:

d. My complete Caddy config:

{
  auto_https off
  servers {
    timeouts {
                        read_body   60s
                        read_header 60s
                }
  }
  http_port 7080
  https_port 1443
}


liberation.scot :1443 {
        #reverse_proxy / web:7080
        tls /etc/caddy/certs/liberation.crt /etc/caddy/certs/liberation.key
}

#localhost :443 {
#       root /static/* /opt/invenio/var/instance/static
#       try_files {path} {path}/ /index.html
#       reverse_proxy web:5000
#       file_server
#       encode gzip
#}

5. Links to relevant resources:

https://www.ssllabs.com/ssltest/analyze.html?d=liberation.scot&latest
A+ cert with chain

2

I think I’ll make this a simpler ask.

GOAL
Caddy needs to act as a reverse proxy for service web on port 5000

I cannot use ports 80 or 443
I want to run on ports 7080(http) and 1443 (https)

I have to use commercial certs.
I cannot use autoconfigure or “download certs”
Static viable PEM certs are already on the host /etc/ssl/2024-oct

My original working Caddyfile (on a server with NO port 80 or 443 is in use looks like)

localhost :443 {
root /static/* /opt/invenio/var/instance/static
try_files {path} {path}/ /index.html
reverse_proxy web:5000
file_server
encode gzip
}

My docker-composer Caddy section likes like

services:
proxy:
image: caddy:2.8.1
restart: “unless-stopped”
ports:
- “7080:80”
- “1443:443”
volumes:
- /etc/ssl/2024-Oct:/etc/caddy/certs
- ./Caddyfile:/etc/caddy/Caddyfile
- caddy_data:/data
- caddy_config:/config
web:
image: Package invenio-rdm-starter · GitHub

Anyone care to take a stab at helping me make this change?
Thanks

3

The logs show repetitive reloading of Caddy while it boots up, and I see this log line:

Error: adapting config using caddyfile: server block without any key is global configuration, and if used, it must be first

This indicates the shared Caddyfile isn’t the same supplied to Caddy. Generally, your Caddyfile would look like this:

{
	http_port 7080
	https_port 1443
	# ... other global options
}
example.com {
	tls file.crt file.key
	# ... other directives
}
1 Like
4

Hi. This is pretty much what I have in “d” in my first ask in the forum above.

Thanks for replying.

So, Caddy is getting the WRONG config file?

The docker-compose contains

volumes:
- /etc/ssl/2024-Oct:/etc/caddy/certs
- ./Caddyfile:/etc/caddy/Caddyfile
- caddy_data:/data
- caddy_config:/config

What should the Line be to read Caddyfile from my local disk and let the Caddy container consume it?

Assume my Caddyfile is in /data/TOCADDY/Caddyfile

What should the docker compose contain to let the caddy service read from /data/TOCADDY/Caddyfile

Do I specify and absolute path (host is Linux)
- /data/TOCADDY/Caddyfile:/etc/caddy/Caddyfile
Thanks!

5

Oops, sorry I missed it while skimming the thread.

The structure in compose files is - local:container. To avoid confusion, you can use the source and destination.

6

thanks again. I’ll specify and absolute path on the LHS and see what happens.

7

I used docker exec to “ash” into the caddy container and I see /etc/caddy/Caddyfile is from the host system (good)

But STILL no web pages.

I have run tcpdump port 1443 and I see a handshake from
browser<->HOST:1443

PORTS

 docker container ls --format "table {{.ID}}\t{{.Names}}\t{{.Ports}}" -a
CONTAINER ID   NAMES                          PORTS
52ba1b8c7e07   invenio-rdm-starter-web-1      5000/tcp
6fe684a98abd   invenio-rdm-starter-worker-1   5000/tcp
50a8adff3316   invenio-rdm-starter-db-1       0.0.0.0:5432->5432/tcp, :::5432->5432/tcp
26c453e60b09   invenio-rdm-starter-cache-1    0.0.0.0:6379->6379/tcp, :::6379->6379/tcp
33b86b9230cb   invenio-rdm-starter-proxy-1    443/udp, 2019/tcp, 0.0.0.0:7080->80/tcp, [::]:7080->80/tcp, 0.0.0.0:1443->443/tcp, [::]:1443->443/tcp
cf12d045b388   invenio-rdm-starter-search-1   0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 9300/tcp, 0.0.0.0:9600->9600/tcp, :::9600->9600/tcp, 9650/tcp

NEW CADDY FILE

{
  http_port 7080
  https_port 1443
  auto_https off
  servers {
    timeouts {
                        read_body   60s
                        read_header 60s
                }
  }
}

liberation.scot:1443 {
        tls /etc/caddy/certs/liberation.crt /etc/caddy/certs/liberation.key
        reverse_proxy web:5000
        root /static/* /opt/invenio/var/instance/static
        try_files {path} {path}/ /index.html
        file_server
        encode gzip
}
8

Can you show curl -v and Caddy logs as evidence?

9

My OPENSSL host library is here

 openssl                              3.0.2-0ubuntu1.18                            arm64        Secure Sockets Layer toolkit - cryptographic utility

But OpenSSL doesnt support QUIC till >=3.2
OpenSSL QUIC and HTTP3 clients

Do I need a more recent openSSL on my host???

ALSO
The listener for caddy is

proxy-1   | {"level":"info","ts":1731970359.0172782,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
proxy-1   | {"level":"info","ts":1731970359.0272598,"msg":"adapted config to JSON","adapter":"caddyfile"}
proxy-1   | {"level":"warn","ts":1731970359.0273218,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
proxy-1   | {"level":"info","ts":1731970359.0339358,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
proxy-1   | {"level":"info","ts":1731970359.0355625,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x400077a200"}
proxy-1   | {"level":"info","ts":1731970359.0398388,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
proxy-1   | {"level":"info","ts":1731970359.040976,"logger":"http","msg":"enabling HTTP/3 listener","addr":":1443"}
proxy-1   | {"level":"info","ts":1731970359.0412495,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
proxy-1   | {"level":"info","ts":1731970359.0455549,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
proxy-1   | {"level":"info","ts":1731970359.0455923,"msg":"serving initial configuration"}
proxy-1   | {"level":"info","ts":1731970359.0519857,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"62804943-2f3d-4ee8-8fd5-77ad8e1e1906","try_again":1732056759.0519822,"try_again_in":86399.999998728}
proxy-1   | {"level":"info","ts":1731970359.0522,"logger":"tls","msg":"finished cleaning storage units"}

{“level”:“info”,“ts”:1731970359.040976,“logger”:“http”,“msg”:“enabling HTTP/3 listener”,“addr”:“:1443”} ?

Do I need an http3 enabled browser to connect to Caddy?

10

Yes, thanks

 docker logs ce89d9cca150
{"level":"info","ts":1731969312.70626,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
{"level":"info","ts":1731969312.7110066,"msg":"adapted config to JSON","adapter":"caddyfile"}
{"level":"warn","ts":1731969312.7110543,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
{"level":"info","ts":1731969312.7129905,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1731969312.7135913,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x40004f7080"}
{"level":"info","ts":1731969312.7147992,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
{"level":"info","ts":1731969312.7169771,"logger":"http","msg":"enabling HTTP/3 listener","addr":":1443"}
{"level":"info","ts":1731969312.7184522,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1731969312.7189543,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1731969312.718983,"msg":"serving initial configuration"}
{"level":"info","ts":1731969312.7219563,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"62804943-2f3d-4ee8-8fd5-77ad8e1e1906","try_again":1732055712.7219527,"try_again_in":86399.999999136}
{"level":"info","ts":1731969312.7221177,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1731969580.637468,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
{"level":"warn","ts":1731969580.6375291,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
{"level":"info","ts":1731969580.637573,"logger":"http","msg":"servers shutting down with eternal grace period"}
{"level":"info","ts":1731969580.6379297,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
{"level":"info","ts":1731969580.6379497,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
{"level":"info","ts":1731970359.0172782,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
{"level":"info","ts":1731970359.0272598,"msg":"adapted config to JSON","adapter":"caddyfile"}
{"level":"warn","ts":1731970359.0273218,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
{"level":"info","ts":1731970359.0339358,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1731970359.0355625,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x400077a200"}
{"level":"info","ts":1731970359.0398388,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
{"level":"info","ts":1731970359.040976,"logger":"http","msg":"enabling HTTP/3 listener","addr":":1443"}
{"level":"info","ts":1731970359.0412495,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1731970359.0455549,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1731970359.0455923,"msg":"serving initial configuration"}
{"level":"info","ts":1731970359.0519857,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"62804943-2f3d-4ee8-8fd5-77ad8e1e1906","try_again":1732056759.0519822,"try_again_in":86399.999998728}

type or paste code here
11 
 curl -v https://localhost:1443
*   Trying 127.0.0.1:1443...
* Connected to localhost (127.0.0.1) port 1443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.0 (OUT), TLS header, Unknown (21):
* TLSv1.3 (OUT), TLS alert, decode error (562):
* error:0A000126:SSL routines::unexpected eof while reading
* Closing connection 0
curl: (35) error:0A000126:SSL routines::unexpected eof while reading

Why is ALPN running if I said NO auto https??

12

You made a request to Caddy with a domain that it isn’t configured for. Caddy was not configured for localhost. It doesn’t know how to respond to that except with empty 200.

But you DID say HTTPS. Your command was curl -v https://localhost:1443, which contains HTTPS and port 1443, which you configured Caddy to use it as the HTTPS port.

No. It’ll work with all browsers.

Caddy doesn’t use openssl.

13

OK, here’s the Crux. My servers A record is configured on the internet as liberation.scot

How does Caddy resolve its hostnames that appear in Caddyfile?

This is an AWS EC2 host so it gets packets from its external facing (NATd) A record liberation.scot to a LOCAL (non routable IP address).

What records need to be set for Caddy ?

What goes into /etc/hosts? Whats is ‘hostname’ meant to be set to? Where does Caddy look via its resolver to “see” liberation.scot:1443 (which is the internet facing side).

Route53 is not is use here.

Thanks

14

It uses the local resolver when it needs to make a lookup. In a Docker container, that’s handled by Docker’s internal DNS resolver, which provides Compose network service/container resolution and then leans on the host operating system’s resolver for external hosts. That said, Caddy typically doesn’t need to make lookups for the websites it is configured to serve.

Strictly speaking, the only hosts you’d need to ensure are resolveable are probably your ACME providers as well as your backend (web is resolved by Docker, LetsEncrypt/ZeroSSL resolved by the operating system).

Usually nothing specific to Caddy.

Usually not important; set it to whatever you’re comfortable with.

Caddy doesn’t typically look up its own external address unless it needs to proxy to itself on that external address for some reason. I don’t see that you’ve configured that anywhere, so I can’t say it’s relevant. But, nonetheless, it looks ultimately to the operating system resolver to answer that question.

All the DNS resolution magic that turns traffic into packets destined for Caddy essentially takes place on the side of the client, not the server. The client needs to lookup liberation.scot and translate that to an IP address that routes packets to Caddy. Caddy just sits there and listens for those requests to come in.

I’m assuming you’re having issues getting curl to send traffic for liberation.scot to Caddy? If so, this might help: https://acquia.my.site.com/s/article/360005257154-Use-cURL-s-resolve-option-to-pin-a-request-to-an-IP-address

2 Likes
15

OK, I have tried
curl -v https://liberation.scot:1443 --resolve liberation.scot:1443:127.0.0.1

* Added liberation.scot:1443:127.0.0.1 to DNS cache
* Hostname liberation.scot was found in DNS cache
*   Trying 127.0.0.1:1443...
* Connected to liberation.scot (127.0.0.1) port 1443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.0 (OUT), TLS header, Unknown (21):
* TLSv1.3 (OUT), TLS alert, decode error (562):
* error:0A000126:SSL routines::unexpected eof while reading
* Closing connection 0
curl: (35) error:0A000126:SSL routines::unexpected eof while reading

Why do I see

> *  CAfile: /etc/ssl/certs/ca-certificates.crt
> *  CApath: /etc/ssl/certs

If I open “ash” on the docker container I see MY commercial certs in /etc/caddy/certs/liberation.crt /etc/caddy/certs/liberation.key

Why then is Caddy STILL behaving like it wants to serve localhost and stuff from /etc/ssl/certs ??

Caddyfile.

{
  http_port 7080
  https_port 1443
  auto_https off
  servers {
    timeouts {
                        read_body   60s
                        read_header 60s
                }
  }
}

:1443 {
        tls /etc/caddy/certs/liberation.crt /etc/caddy/certs/liberation.key
        reverse_proxy web:5000
        root /static/* /opt/invenio/var/instance/static
        try_files {path} {path}/ /index.html
        file_server
        encode gzip
}

type or paste code here

docker container ls --format “table {{.ID}}\t{{.Names}}\t{{.Ports}}” -a

CONTAINER ID   NAMES                          PORTS
e2cffd90ae24   invenio-rdm-starter-worker-1   5000/tcp
ede86121ef46   invenio-rdm-starter-web-1      5000/tcp
e0a5bc48ac74   invenio-rdm-starter-cache-1    0.0.0.0:6379->6379/tcp, :::6379->6379/tcp
c6cb140ea9a5   invenio-rdm-starter-search-1   0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 9300/tcp, 0.0.0.0:9600->9600/tcp, :::9600->9600/tcp, 9650/tcp
016467ada2e0   invenio-rdm-starter-proxy-1    443/udp, 2019/tcp, 0.0.0.0:7080->80/tcp, [::]:7080->80/tcp, 0.0.0.0:1443->443/tcp, [::]:1443->443/tcp
79d127c0e01e   invenio-rdm-starter-db-1       0.0.0.0:5432->5432/tcp, :::5432->5432/tcp
16

It’s not, and you are misinterpreting this information. It makes no sense for Caddy to serve that certificate because that’s a root CA cert bundle. It also makes no sense for curl to be able to tell you what path on disk Caddy is serving certs from; curl has no way to get that information from Caddy.

That is the local trust store for certificates that the client is using. curl is telling you what certificate authority bundle it is referencing when it tries to validate whatever certificate Caddy is providing.

It did this because you changed your Caddyfile.

Previously you had liberation.scot:1443 as your site address. Now you have :1443 and Caddy doesn’t know you want to serve liberation.scot anymore. Why would you change that? Are you just making random changes like this to see what works or did you have a reason for removing the site address?

Enabling debug in your global options just while you’re troubleshooting this might be a good idea as Caddy will output logs to tell you what’s going on. In this case with debug you would’ve seen http: TLS handshake error from 127.0.0.1:60742: no certificate available for 'liberation.scot' (or similar).

1 Like
17

The drop of the site name was accidental actually. Thanks for spotting. I didn’t realize there was global debug :frowning:

Enabling now. Thanks

18

The full log with debug in global looks like

docker logs 610189d4db8e
{"level":"info","ts":1732166454.7348437,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
{"level":"info","ts":1732166454.7401466,"msg":"adapted config to JSON","adapter":"caddyfile"}
{"level":"warn","ts":1732166454.7402098,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
{"level":"info","ts":1732166454.7428176,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1732166454.7433677,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x400040fa80"}
{"level":"debug","ts":1732166454.7446072,"logger":"events","msg":"event","name":"cached_unmanaged_cert","id":"ef6219f0-b44b-49fe-a6ef-474393dbe632","origin":"tls","data":{"sans":["*.liberation.scot","liberation.scot"]}}
{"level":"debug","ts":1732166454.7447143,"logger":"tls.cache","msg":"added certificate to cache","subjects":["*.liberation.scot","liberation.scot"],"expiration":1763389967,"managed":false,"issuer_key":"","hash":"11814135d9b36b3b021ce09dc68455486c1ba9b9e30e83998a7e69271d2fdb41","cache_size":1,"cache_capacity":10000}
{"level":"info","ts":1732166454.7448063,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
{"level":"debug","ts":1732166454.7448807,"logger":"http.auto_https","msg":"adjusted config","tls":{"automation":{"policies":[{}]}},"http":{"http_port":7080,"https_port":1443,"servers":{"srv0":{"listen":[":1443"],"read_timeout":60000000000,"read_header_timeout":60000000000,"routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"handler":"vars","root":"/opt/invenio/var/instance/static"}],"match":[{"path":["/static/*"]}]},{"handle":[{"handler":"rewrite","uri":"{http.matchers.file.relative}"}],"match":[{"file":{"try_files":["{http.request.uri.path}","{http.request.uri.path}/","/index.html"]}}]},{"handle":[{"encodings":{"gzip":{}},"handler":"encode","prefer":["gzip"]},{"handler":"reverse_proxy","upstreams":[{"dial":"web:5000"}]},{"handler":"file_server","hide":["/etc/caddy/Caddyfile"]}]}]}],"terminal":true}],"tls_connection_policies":[{"match":{"sni":["liberation.scot"]},"certificate_selection":{"any_tag":["cert0"]}},{}],"automatic_https":{"disable":true}}}}}
{"level":"info","ts":1732166454.7479804,"logger":"http","msg":"enabling HTTP/3 listener","addr":":1443"}
{"level":"debug","ts":1732166454.7497249,"logger":"http","msg":"starting server loop","address":"[::]:1443","tls":true,"http3":true}
{"level":"info","ts":1732166454.7498393,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"info","ts":1732166454.7506151,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1732166454.7506409,"msg":"serving initial configuration"}
{"level":"info","ts":1732166454.7516096,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"62804943-2f3d-4ee8-8fd5-77ad8e1e1906","try_again":1732252854.7516065,"try_again_in":86399.999999136}
{"level":"info","ts":1732166454.7517664,"logger":"tls","msg":"finished cleaning storage units"}
type or paste code here
19

I don’t see any errors in the logs.

20

Any other feedback is welcome here.