Download with plugins building for ever

axi92 · February 21, 2023, 4:01pm

I tried this download:

I tried it 6 hours ago and now again and still 30min “building” and I get no download.
I tried to build it with xcaddy now and still stuck.
Can somebody download or build me that caddy file?

$ xcaddy build v2.6.4 --with github.com/caddy-dns/digitalocean --with github.com/mholt/caddy-events-exec
2023/02/21 16:53:07 [INFO] Temporary folder: /tmp/buildenv_2023-02-21-1653.579170422
2023/02/21 16:53:07 [INFO] Writing main module: /tmp/buildenv_2023-02-21-1653.579170422/main.go
package main

import (
	caddycmd "github.com/caddyserver/caddy/v2/cmd"

	// plug in Caddy modules here
	_ "github.com/caddyserver/caddy/v2/modules/standard"
	_ "github.com/caddy-dns/digitalocean"
	_ "github.com/mholt/caddy-events-exec"
)

func main() {
	caddycmd.Main()
}
2023/02/21 16:53:07 [INFO] Initializing Go module
2023/02/21 16:53:07 [INFO] exec (timeout=10s): /usr/local/go/bin/go mod init caddy 
go: creating new go.mod: module caddy
go: to add module requirements and sums:
	go mod tidy
2023/02/21 16:53:07 [INFO] Pinning versions
2023/02/21 16:53:07 [INFO] exec (timeout=0s): /usr/local/go/bin/go get -d -v github.com/caddyserver/caddy/v2@v2.6.4 
go: added github.com/beorn7/perks v1.0.1
go: added github.com/caddyserver/caddy/v2 v2.6.4
go: added github.com/caddyserver/certmagic v0.17.2
go: added github.com/cespare/xxhash/v2 v2.1.2
go: added github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0
go: added github.com/golang/mock v1.6.0
go: added github.com/golang/protobuf v1.5.2
go: added github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38
go: added github.com/google/uuid v1.3.0
go: added github.com/klauspost/cpuid/v2 v2.2.3
go: added github.com/libdns/libdns v0.2.1
go: added github.com/matttproud/golang_protobuf_extensions v1.0.1
go: added github.com/mholt/acmez v1.1.0
go: added github.com/miekg/dns v1.1.50
go: added github.com/onsi/ginkgo/v2 v2.2.0
go: added github.com/prometheus/client_golang v1.14.0
go: added github.com/prometheus/client_model v0.3.0
go: added github.com/prometheus/common v0.37.0
go: added github.com/prometheus/procfs v0.8.0
go: added github.com/quic-go/qpack v0.4.0
go: added github.com/quic-go/qtls-go1-18 v0.2.0
go: added github.com/quic-go/qtls-go1-19 v0.2.0
go: added github.com/quic-go/qtls-go1-20 v0.1.0
go: added github.com/quic-go/quic-go v0.32.0
go: added go.uber.org/atomic v1.9.0
go: added go.uber.org/multierr v1.6.0
go: added go.uber.org/zap v1.24.0
go: added golang.org/x/crypto v0.5.0
go: added golang.org/x/exp v0.0.0-20221205204356-47842c84f3db
go: added golang.org/x/mod v0.6.0
go: added golang.org/x/net v0.7.0
go: added golang.org/x/sys v0.5.0
go: added golang.org/x/term v0.5.0
go: added golang.org/x/text v0.7.0
go: added golang.org/x/tools v0.2.0
go: added google.golang.org/protobuf v1.28.1
2023/02/21 16:53:08 [INFO] exec (timeout=0s): /usr/local/go/bin/go get -d -v github.com/caddy-dns/digitalocean github.com/caddyserver/caddy/v2@v2.6.4 
go: added github.com/caddy-dns/digitalocean v0.0.0-20220527005842-9c71e343246b
go: added github.com/digitalocean/godo v1.41.0
go: added github.com/google/go-querystring v1.0.0
go: added github.com/libdns/digitalocean v0.0.0-20220518195853-a541bc8aa80f
2023/02/21 16:53:09 [INFO] exec (timeout=0s): /usr/local/go/bin/go get -d -v github.com/mholt/caddy-events-exec github.com/caddyserver/caddy/v2@v2.6.4 
go: added github.com/mholt/caddy-events-exec v0.0.0-20221013172934-88290c6b74c4
2023/02/21 16:53:09 [INFO] exec (timeout=0s): /usr/local/go/bin/go get -d -v

matt · February 21, 2023, 5:24pm

This is due to this bug in the go command:

github.com/golang/go

cmd/go: get hangs without error output from unknown revisions

opened 06:46PM - 15 Jul 22 UTC

Justin-W

NeedsInvestigation GoCommand modules

… ### What version of Go are you using (`go version`)? <pre> $ go version go version go1.18.4 darwin/arm64 $ git version git version 2.37.1 </pre> ### Does this issue reproduce with the latest release? Yes. ### What operating system and processor architecture are you using (`go env`)? <details><summary><code>go env</code> Output</summary><br><pre> $ go env % go env GO111MODULE="auto" GOARCH="arm64" GOBIN="" GOCACHE="REDACTED" GOENV="REDACTED" GOEXE="" GOEXPERIMENT="" GOFLAGS="" GOHOSTARCH="arm64" GOHOSTOS="darwin" GOINSECURE="" GOMODCACHE="REDACTED" GONOPROXY="REDACTED" GONOSUMDB="REDACTED" GOOS="darwin" GOPATH="REDACTED" GOPRIVATE="REDACTED" GOPROXY="https://proxy.golang.org,direct" GOROOT="/usr/local/go" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/darwin_arm64" GOVCS="" GOVERSION="go1.18.4" GCCGO="gccgo" AR="ar" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="" GOWORK="" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/vy/ryp4d95n75q5j6kf58hc551m0000gn/T/go-build559884269=/tmp/go-build -gno-record-gcc-switches -fno-common" </pre></details> ### What did you do?  1. Create/identify 3 go modules/repos: `A` depended on by `B` depended on by `C` (so C _indirectly_ depends on A) 2. make B's `go.mod` reference to A invalid (for us, A's git history was rewritten; but you could probably manually edit B's go.mod with a invalid pseudo-version to get the same effect) 3. run `go get -u ./...` in either B or C (or any module that depends on either of them) 4. wait for the `go` process to hang with high CPU, then terminate it 5. run `go list -m -u -json all | jq --raw-output 'select((.Indirect or .Main)|not) | .Path' | xargs -t -I % go get -u -t %@latest`, to find any "error:" or "panic:" output 6. Revert any changes to `go.*` files resulting from step 5. ### What did you expect to see? Either some output indicating the dependency reference problem(s), or else normal operation of the `go get ...` command in step 3. ### What did you see instead? Step 3 starts running normally (there may or may not be some output, downloads, etc., with typical levels of CPU by the process), but at some point all output ceases and CPU spikes to 350%-450% and stays basically flat at that level until the `go` process is terminated. Step 5 did not hang, and output contained several sections with errors like these: ``` go: downloading github.com/private-org/module-two v0.0.0-20220713230938-ae346380ff64 go: github.com/private-org/module-one@v0.0.0-20220203083605-df6cd891c11a: invalid version: unknown revision df6cd891c11a go get -u -t github.com/private-org/module-three@latest go: github.com/private-org/module-one@v0.0.0-20220203083605-df6cd891c11a: invalid version: unknown revision df6cd891c11a ``` ### Workaround 7. Add a `replace` directive for _each_ of the "invalid version"s identified in step 5 to the `go.mod` files of _each_ dependent repo (including indirectly-dependent ones). 8. Repeat steps 3 through 7 until step 3 finishes normally (i.e. without hanging) ### Notes: - The hanging behavior happened consistently (100% repeatable; always hanged) on only some of the M1 macs tested. Others with identical hardware and similar go, git, etc. versions, even when running exactly the same commands on the same repos. Not sure why the issue only affected some envs and not others. - The root cause for us was that 3 repos had at least 1 pseudo-version reference (by other repos) to a commit which no longer existed in the referenced repo. Once the workaround above was completed for _each_ missing revision, the unexpected `go get` behavior ceased. * I only identified a current `go.mod` reference to 1 of the 4 "missing revision"s which were in the error output. The other 3 _seemed_ to be referenced exclusively from historical versions of `go.mod` and `go.sum` files (e.g. the git history of no-longer-referenced versions of certain modules), but not by any versions which were still directly referenced. IDK how un-referenced versions' `go.*` contents could still be affecting `go get` behavior, but it _seemed_ to be the case. Very weird. If true, more details would be appreciated, since that might indicate that the `replace` directives need to remain in place forever (which would be unfortunate). - All repos involved are private repos, but IDK whether that matters or not. I suspect not. (All of the usual "private repo"-related config issues are not an issue for us, since all commands work fine on some machines, and work fine for repos with un-corrupted `go.*` files even on affected envs.) - The hanging happened with latest git and go versions (installed yesterday morning), and also with older versions of go (v1.18) and git. (IDK whether the git version matters or not, since upgrading seemed to have no effect.) - All of the repos involved had `go 1.17` or `go 1.18` directives in their `go.mod` files. We weren't attempting to preserve pre-v1.17 compatibility. - Adding git tags with names equal to the missing pseudoversions (e.g. `v0.0.0-20220203083605-df6cd891c11a`) did not work as a workaround. * It would be nice to have a tag-based way to retroactively "fix" `go.mod` refs which get broken as a result of commits which get irrevocably deleted. That would be much preferred vs the `replace` directives workaround. - Adding `retract` directives to the repos which were the source of the "missing" versions did not resolve the problem. (Would've been _much_ nicer if it had, so that the same `replace` directives didn't need to be replicated in dozens of dependent repos.) * I know that `retract` isn't intended to prevent a dependent repo from referencing a retracted version once it already is, but since I was actually trying to upgrade all the deps, it would've been nice if the `go get -u ./...` command simply updated the "bad" refs with good ones (and didn't hang), which would have allowed me to iteratively upgrade all dependencies to stop using the bad revisions (without the need to manually identify and `replace` the "missing" revisions).

(We think.)

Upgrading a plugin’s go.mod file to use the latest version of Caddy seems to fix it, but we’re not sure why. And neither is the Go team as of yet.

Sorry for the trouble. I’ve updated that plugin’s go.mod for now, which apparently works around the problem.

axi92 · February 22, 2023, 10:56am

I tried it with the online download but it did not work

I try later to build it myself.

axi92 · February 22, 2023, 1:02pm

Ok on my local machine I am able to build it.

devil1591 · February 26, 2023, 10:49am

Hi, can’t build on the caddy’s website!

francislavoie · February 26, 2023, 12:53pm

@devil1591 it builds just fine for me with xcaddy:

$ xcaddy build --with github.com/caddy-dns/cloudflare --with github.com/abiosoft/caddy-exec

devil1591 · February 27, 2023, 9:54am

Hi Francis, I didn’t try on my machine, but it is broken on the website…

axi92 · March 1, 2023, 3:04pm

I think this is also broken
https://caddyserver.com/api/download?os=darwin&arch=arm64&p=github.com%2Fcaddy-dns%2Fdigitalocean&idempotency=97639560471998

system · May 22, 2023, 4:02pm

This topic was automatically closed after 90 days. New replies are no longer allowed.