Remote build request doesn't work for FreeBSD amd64

1. Output of caddy version:

v2.6.2

2. How I run Caddy:

a. System environment:

Bare-metal
FreeBSD 13.1-RELEASE-p2 FreeBSD 13.1-RELEASE-p2 GENERIC amd64

b. Command:

Any command that requires triggering build request on caddy website (including download caddy with plugins):

caddy upgrade
caddy add-package
...

c. Service/unit/compose file:

#!/bin/sh

# PROVIDE: caddy
# REQUIRE: LOGIN DAEMON NETWORKING
# KEYWORD: shutdown

# To enable caddy, add 'caddy_enable="YES"' to /etc/rc.conf or
# /etc/rc.conf.local

# Optional settings:
# caddy_command (string):     Full path to the caddy binary
# caddy_config (string):      Full path to caddy config file
#                             (/usr/local/etc/caddy/Caddyfile)
# caddy_adapter (string):     Config adapter type (caddyfile)
# caddy_directory (string):   Root for caddy storage (ACME certs, etc.)
#                             (/var/db/caddy)
# caddy_extra_flags (string): Extra flags passed to caddy start
# caddy_logdir (string):      Where caddy logs are stored
#                             (/var/log/caddy)
# caddy_logfile (string):     Location of process log (${caddy_logdir}/caddy.log)
#                             This is for startup/shutdown/error messages.
#                             To create an access log, see:
#                             https://caddyserver.com/docs/caddyfile/directives/log
# caddy_user (user):          User to run caddy (root)
# caddy_group (group):        Group to run caddy (wheel)
#
# This script will honor XDG_CONFIG_HOME/XDG_DATA_HOME. Caddy will create a
# .../caddy subdir in each of those. By default, they are subdirs of /var/db/caddy.
# See https://caddyserver.com/docs/conventions#data-directory

. /etc/rc.subr

name=caddy
rcvar=caddy_enable
desc="Powerful, enterprise-ready, open source web server with automatic HTTPS written in Go"

load_rc_config $name

# Defaults
: ${caddy_enable:=NO}
: ${caddy_adapter:=caddyfile}
: ${caddy_config:="/usr/local/etc/caddy/Caddyfile"}
: ${caddy_command:="/usr/local/bin/${name}"}
: ${caddy_directory:=/var/db/caddy}
: ${caddy_extra_flags:=""}
: ${caddy_logdir:="/var/log/${name}"}
: ${caddy_logfile:="${caddy_logdir}/${name}.log"}
: ${caddy_user:="root"}
: ${caddy_group:="wheel"}

# Config and base directories
: ${XDG_CONFIG_HOME:="${caddy_directory}/config"}
: ${XDG_DATA_HOME:="${caddy_directory}/data"}
export XDG_CONFIG_HOME XDG_DATA_HOME

command="${caddy_command}"
caddy_flags="--config ${caddy_config} --adapter ${caddy_adapter}"
pidfile="/var/run/${name}/${name}.pid"

required_files="${caddy_config} ${caddy_command}"

start_precmd="caddy_precmd"
start_cmd="caddy_start"
stop_precmd="caddy_prestop"

# Extra Commands
extra_commands="configtest reload"
configtest_cmd="caddy_execute validate ${caddy_flags}"
reload_cmd="caddy_execute reload ${caddy_flags}"

caddy_execute()
{
    /usr/bin/su -m "${caddy_user}" -c "${caddy_command} $*"
}

caddy_precmd()
{
    # Create required directories and set permissions
    /usr/bin/install -d -m 755 -o "${caddy_user}" -g "${caddy_group}" ${caddy_directory}
    /usr/bin/install -d -m 700 -o "${caddy_user}" -g "${caddy_group}" ${caddy_directory}/config
    /usr/bin/install -d -m 700 -o "${caddy_user}" -g "${caddy_group}" ${caddy_directory}/data
    /usr/bin/install -d -m 755 -o "${caddy_user}" -g "${caddy_group}" ${caddy_logdir}
    /usr/bin/install -d -m 700 -o "${caddy_user}" -g "${caddy_group}" /var/run/caddy
    if [ -e ${caddy_logfile} ]; then
        /bin/chmod 644 ${caddy_logfile}
        /usr/sbin/chown root:wheel ${caddy_logfile}
    else
        /usr/bin/install -m 644 -o root -g wheel /dev/null ${caddy_logfile}
    fi
}

caddy_start()
{
    echo -n "Starting caddy... "
    /usr/bin/su -m ${caddy_user} -c "${caddy_command} start ${caddy_flags} \
        ${caddy_extra_flags} --pidfile ${pidfile}" >> ${caddy_logfile} 2>&1
    if [ $? -eq 0 ] && ps -ax -o pid | grep -q "$(cat ${pidfile})"; then
        echo "done"
        echo "Log: ${caddy_logfile}"
    else
        echo "Error: Caddy failed to start"
        echo "Check the caddy log: ${caddy_logfile}"
    fi
}

caddy_prestop()
{
    local result

    echo -n "Stopping caddy... "

    result="$(caddy_execute stop 2>&1)"
    if [ ${?} -eq 0 ]; then
        echo "done"
        exit 0
    else
        if echo "${result}" | grep -q "connection refused"; then
            echo "admin interface unavailable; using pidfile"
            return 0
        else
            echo "Error: Unable to stop caddy"
            echo "Check the caddy log: ${caddy_logfile}"
            return 1
        fi
    fi
}

run_rc_command "$1"

d. My complete Caddy config:

# The Caddyfile is an easy way to configure your Caddy web server.
#
# To use your own domain name (with automatic HTTPS), first make
# sure your domain's A/AAAA DNS records are properly pointed to
# this machine's public IP, then replace the line below with your
# domain name.

# Unless the file starts with a global options block, the first
# uncommented line is always the address of your site.
#

m.ddns.net {
                # Set up a reverse proxy:
                reverse_proxy 10.0.1.1:8080

                # Enable logging:
                log {
                                output file /var/log/caddy/m.ddns.net.access.log
                                # Caddy's structured log format:
                                format json
                                # Or, for Common Log Format:
                                # format single_field common_log
                }
                tls {
                        key_type p384
                        curves secp521r1 secp384r1 x25519
                }
                header / Strict-Transport-Security "max-age=31536000;"
}

4m.ddns.net {
                # Set up a reverse proxy:
                reverse_proxy 10.0.1.1:8080

                # Enable logging:
                log {
                                output file /var/log/caddy/4m.ddns.net.access.log
                                # Caddy's structured log format:
                                format json
                                # Or, for Common Log Format:
                                # format single_field common_log
                }
                header / Strict-Transport-Security "max-age=31536000;"
}

torrentm.ddns.net {
                # Set up a reverse proxy:
                reverse_proxy 127.0.0.1:8080

                # Enable logging:
                log {
                                output file /var/log/caddy/torrentm.ddns.net.access.log
                                # Caddy's structured log format:
                                format json
                                # Or, for Common Log Format:
                                # format single_field common_log
                }
                header / Strict-Transport-Security "max-age=31536000;"
}

3. The problem I’m having:

4. Error messages and/or full log output:

./caddy upgrade
2023/01/11 17:05:21.393 INFO    this executable will be replaced        {"path": "/usr/home/thomas/caddy"}
2023/01/11 17:05:21.393 INFO    requesting build        {"os": "freebsd", "arch": "amd64", "packages": ["github.com/corazawaf/coraza-caddy"]}
Error: download failed: download failed: HTTP 400: unable to fulfill download request (id=d92afa90-d6cd-459a-a54b-29323a984574)

The program hangs and the build is never fetched, so I guess the server isn’t able to build caddy for FreeBSD amd64 with some plugins.
I have the same result when I trigger caddy add-package on the same OS.

5. What I already tried:

I tried to trigger the build for other environments (Windows, Linux) and it worked perfectly.
I also tried to download from the official website Download Caddy , it worked everytime excepted when I tried to download for FreeBSD with plugins.

The xcaddy command worked fine so I guess the error is on the server side.

6. Links to relevant resources:

There is a bug in the go get command that causes it to hang sometimes. It was running for 23h, it looks like.

I’ll reboot the build server and it should start working again for a while.

We used to set a timeout on the go get command when building, but due to legitimately slow networks, had to remove it. Maybe there’s a higher timeout that would work, but I’m not sure what it should be.

Some builds work for you probably because they already were built and were thus cached. Or maybe a particular package server is causing go get to hang, so builds with that plugin are slow. I don’t know…

Ok nice, thank you very much!