Xcaddy can't build linux arm

I need to use xcaddy to build arm64 builds with the route53 plugin

I have had no problem using xcaddy to build linux amd64 binaries but I can’t get an arm64 nor armv7 build successfully on my amd64 dev machine. I’ve opened an issue on the xcaddy repo https://github.com/caddyserver/xcaddy/issues/39 but I thought I would ask here for maybe better attention or someone who has solved this.

Repeating what is at the issue.

Seems there is some issue with this module in arm builds
# vendor/golang.org/x/crypto/poly1305 /usr/local/go/src/vendor/golang.org/x/crypto/poly1305/sum_noasm.go:10:7: undefined: newMAC

I’d guess it’s some arm cross complie setup issue as of course there are arm release builds and I assume mholt/etal uses xcaddy to cross compile those. Can anyone help track down what I am overlooking?

This is ouput of the build script which is below.

scripts/build
Using Plugin: github.com/caddy-dns/route53
deleting any old executable linux-arm64
Building binary /opt/caddy/bin/linux-arm64
GO environment before running xcaddy
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/home/david/.cache/go-build"
GOENV="/home/david/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/opt/caddy/build/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/opt/caddy/build"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build745626225=/tmp/go-build -gno-record-gcc-switches"
building with
/opt/caddy/build/bin/xcaddy build --output /opt/caddy/bin/linux-arm64 --with github.com/caddy-dns/route53
2020/10/26 15:25:06 [INFO] Temporary folder: /tmp/buildenv_2020-10-26-1525.467728616
2020/10/26 15:25:06 [INFO] Writing main module: /tmp/buildenv_2020-10-26-1525.467728616/main.go
2020/10/26 15:25:06 [INFO] Initializing Go module
2020/10/26 15:25:06 [INFO] exec (timeout=10s): /usr/local/go/bin/go mod init caddy 
go: creating new go.mod: module caddy
2020/10/26 15:25:06 [INFO] Pinning versions
2020/10/26 15:25:06 [INFO] exec (timeout=0s): /usr/local/go/bin/go get -d -v github.com/caddyserver/caddy/v2 
go: github.com/caddyserver/caddy/v2 upgrade => v2.2.1
2020/10/26 15:25:07 [INFO] exec (timeout=0s): /usr/local/go/bin/go get -d -v github.com/caddy-dns/route53 
go: github.com/caddy-dns/route53 upgrade => v1.0.2
2020/10/26 15:25:09 [INFO] Build environment ready
2020/10/26 15:25:09 [INFO] Building Caddy
2020/10/26 15:25:09 [INFO] exec (timeout=0s): /usr/local/go/bin/go build -o /opt/caddy/bin/linux-arm64 -ldflags -w -s -trimpath 
# vendor/golang.org/x/crypto/poly1305
/usr/local/go/src/vendor/golang.org/x/crypto/poly1305/sum_noasm.go:10:7: undefined: newMAC
# runtime/cgo
gcc_arm64.S: Assembler messages:
gcc_arm64.S:28: Error: no such instruction: `stp x29,x30,[sp,'
gcc_arm64.S:32: Error: too many memory references for `mov'
gcc_arm64.S:34: Error: no such instruction: `stp x19,x20,[sp,'
gcc_arm64.S:37: Error: no such instruction: `stp x21,x22,[sp,'
gcc_arm64.S:40: Error: no such instruction: `stp x23,x24,[sp,'
gcc_arm64.S:43: Error: no such instruction: `stp x25,x26,[sp,'
gcc_arm64.S:46: Error: no such instruction: `stp x27,x28,[sp,'
gcc_arm64.S:50: Error: too many memory references for `mov'
gcc_arm64.S:51: Error: too many memory references for `mov'
gcc_arm64.S:52: Error: too many memory references for `mov'
gcc_arm64.S:54: Error: no such instruction: `blr x20'
gcc_arm64.S:55: Error: no such instruction: `blr x19'
gcc_arm64.S:57: Error: no such instruction: `ldp x27,x28,[sp,'
gcc_arm64.S:60: Error: no such instruction: `ldp x25,x26,[sp,'
gcc_arm64.S:63: Error: no such instruction: `ldp x23,x24,[sp,'
gcc_arm64.S:66: Error: no such instruction: `ldp x21,x22,[sp,'
gcc_arm64.S:69: Error: no such instruction: `ldp x19,x20,[sp,'
gcc_arm64.S:72: Error: no such instruction: `ldp x29,x30,[sp],'
2020/10/26 15:25:13 [INFO] Cleaning up temporary folder: /tmp/buildenv_2020-10-26-1525.467728616
2020/10/26 15:25:13 [FATAL] exit status 2

...done building

The script is

#!/bin/bash
DIR=$(dirname "$(dirname "$(readlink -f "$0")")") || exit
BUILD_DIR=$DIR/build
export GOPATH=$BUILD_DIR
# set a specific release
REL=$1
PLUGINS=''
while IFS= read -r line; do
    [ $line ] && echo Using Plugin: $line
    PLUGINS=`[ $line ] && echo $PLUGINS' --with '$line`
done < $BUILD_DIR/plugins.txt
# reading builds file
cat $BUILD_DIR/builds.txt | while read GOOS GOARCH GOARM;
do
GOOS=${GOOS:-linux}
GOARCH=${GOARCH:-amd64}
BIN_NAME=$GOOS-$GOARCH`[ $GOARM ] && echo -$GOARM`
BIN_PATH=$DIR/bin/$BIN_NAME
echo deleting any old executable $BIN_NAME
[ -f $BIN_PATH ] && rm $BIN_PATH
echo Building binary $BIN_PATH
export GOOS=$GOOS
export GOARCH=$GOARCH
export CGO_ENABLED=1
[ $GOARM ] && export GOARM=$GOARM
echo GO environment before running xcaddy
go env
echo building with
CMD="
   ${BUILD_DIR}/bin/xcaddy
   build
   ${REL}
   --output ${BIN_PATH}
   ${PLUGINS}
   "
echo ${CMD}
echo "$(eval ${CMD})"
echo ...done building
if [ -f $BIN_PATH ]; then
echo build successful at $BIN_PATH
echo run install script with -l option to use this binary
fi
done

Got it working

Installed fresh golang using https://github.com/canha/golang-tools-install-script
Installed latest gcc/g++ using https://linuxize.com/post/how-to-install-gcc-compiler-on-ubuntu-18-04/

then per @mholt set CGO_ENABLED=0

and now it cross compiles without error

2 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.