V2 modules and cross compile

[dkebler (David Kebler)]

Ok. So I am finally ready to jump into V2 but I have two questions related to plugins

1. What V1 plugins/modules are available for V2 (is there a list like on the v1 website)? In particular I need the route53 dns plugin.

2. I run caddy mostly on armv8 machines. At the V1 caddy web it would compile an executable with the plugins I chose for the architecture I chose and then I could download it. Looks like in V2 I have to compile it myself? I have a setup of go with modules for my dev x86 for doing basic compiles of go source but since I am not a go programmer I have no idea how to cross compile for armv8. I don’t really want to compile on my armv8 machines directly they are for production and also at least with kernal compiles, slow. What is going to be the easy way to create custom binaries (i.e. with plugins) for armv8?

[matt (Matt Holt)]

Hi David!

@dkebler

What V1 plugins/modules are available for V2 (is there a list like on the v1 website)? In particular I need the route53 dns plugin.

There is no central registry yet. Like with v1: first I need to get the software released, and then I can spend time working on the website.

v2 doesn’t yet have full-fledged DNS provider support, since I need to do some major refactoring of a fork of an upstream project first. But if you don’t mind using a PR that someone made, you can at least use Route53 sooner than later with v2: https://github.com/caddyserver/tls.dns/pull/2

@dkebler

Looks like in V2 I have to compile it myself?

We have compiled binaries for various platforms at our Download link (expand the “Assets” link – I wish GitHub would make those more prominent but whatever).

Of course, these don’t have extra plugins installed.

@dkebler

What is going to be the easy way to create custom binaries (i.e. with plugins) for armv8?

Use xcaddy - you can cross-compile just like you would with the regular go command: GOOS=linux GOARCH=arm64 xcaddy build ...

[dkebler (David Kebler)]

Thx for info. I wrote a bash script to make this easier but it throws an error for that PR. Maybe I just don’t have the right module url?

#!/bin/bash
DIR="$(dirname "$(readlink -f "$0")")"
ARCH=amd64
echo Caddy Executable will be in $DIR/$ARCH
OS=linux
REL=v2.0.0-rc.3
echo deleting any old executable
[ -f $DIR/$ARCH/caddy ] && rm $DIR/$ARCH/caddy
echo ...building
GOOS=$OS GOARCH=$ARCH xcaddy build $REL \
   --output $DIR/$ARCH \
   --with github.com/danlsgiga/tls.dns/tree/route53-provider
echo ...done building
[ -f $DIR/$ARCH/caddy ] && $DIR/$ARCH/caddy version

2020/04/15 12:14:21 [INFO] Temporary folder: /tmp/buildenv_2020-04-15-1214.243984076
2020/04/15 12:14:21 [INFO] Writing main module: /tmp/buildenv_2020-04-15-1214.243984076/main.go
2020/04/15 12:14:21 [INFO] Initializing Go module
2020/04/15 12:14:21 [INFO] exec (timeout=10s): /usr/local/go/bin/go mod init caddy 
go: creating new go.mod: module caddy
2020/04/15 12:14:21 [INFO] Pinning versions
2020/04/15 12:14:21 [INFO] exec (timeout=5m0s): /usr/local/go/bin/go get -d -v github.com/caddyserver/caddy/v2@v2.0.0-rc.3 
2020/04/15 12:14:21 [INFO] exec (timeout=5m0s): /usr/local/go/bin/go get -d -v github.com/danlsgiga/tls.dns/tree/route53-provider 
go get github.com/danlsgiga/tls.dns/tree/route53-provider: module github.com/danlsgiga/tls.dns@upgrade found (v0.0.0-20200209203058-83da3c680522), but does not contain package github.com/danlsgiga/tls.dns/tree/route53-provider
2020/04/15 12:14:21 [FATAL] exit status 1

[dkebler (David Kebler)]

Another issue is that if the build completes (i.e. I leave out the module for now) xcaddy tries to run the executable (why I asked for build) but it fails for some permission error (why?). Owner of folder is same as user who launched the script. . Not a big deal my script runs it for version afterwards anyway

/opt/caddy2/amd64 version
2020/04/15 12:36:10 [FATAL] fork/exec /opt/caddy2/amd64: permission denied

[matt (Matt Holt)]

github.com/danlsgiga/tls.dns/tree/route53-provider

^ I think that is a GitHub URL, not a Go module path. I have never done this for a fork before, so I don’t know much here.

As for the permission error, I have absolutely no clue…

[francislavoie]

Could you try:

github.com/danlsgiga/tls.dns@02133ff71f3ff43dfad9cb8072d2d8a0de07f7ba

[dkebler (David Kebler)]

@francislavoie

github.com/danlsgiga/tls.dns@02133ff71f3ff43dfad9cb8072d2d8a0de07f7ba

that got me a bit further. some issue with path now. Maybe @danlsgiga who submitted the PR knows how to get this module incorporated correctly?

2020/04/15 18:10:18 [INFO] Temporary folder: /tmp/buildenv_2020-04-15-1810.634908405
2020/04/15 18:10:18 [INFO] Writing main module: /tmp/buildenv_2020-04-15-1810.634908405/main.go
2020/04/15 18:10:18 [INFO] Initializing Go module
2020/04/15 18:10:18 [INFO] exec (timeout=10s): /usr/local/go/bin/go mod init caddy 
go: creating new go.mod: module caddy
2020/04/15 18:10:19 [INFO] Pinning versions
2020/04/15 18:10:19 [INFO] exec (timeout=5m0s): /usr/local/go/bin/go get -d -v github.com/caddyserver/caddy/v2@v2.0.0-rc.3 
2020/04/15 18:10:22 [INFO] exec (timeout=5m0s): /usr/local/go/bin/go get -d -v github.com/danlsgiga/tls.dns@02133ff71f3ff43dfad9cb8072d2d8a0de07f7ba 
go: downloading github.com/danlsgiga/tls.dns v0.0.0-20200324153516-02133ff71f3f
go: github.com/danlsgiga/tls.dns 02133ff71f3ff43dfad9cb8072d2d8a0de07f7ba => v0.0.0-20200324153516-02133ff71f3f
go get: github.com/danlsgiga/tls.dns@v0.0.0-20200324153516-02133ff71f3f: parsing go.mod:
	module declares its path as: github.com/caddyserver/tls.dns
	        but was required as: github.com/danlsgiga/tls.dns
2020/04/15 18:10:30 [FATAL] exit status 1

[matt (Matt Holt)]

Oh, yeah, this is because it’s a fork… hmm… needs a replace statement. I’m not sure xcaddy supports that yet… I suppose I need to implement something like:

--with github.com/caddyserver/tls.dns=>github.com/danlsgiga/tls.dns

or something…

[francislavoie]

Uh… for now - make your own fork of danlsgiga’s and replace the package name in it

[danlsgiga (Daniel Santos)]

Exactly! I had to build it with a go mod edit -replace github.com/caddyserver/tls.dns=github.com/danlsgiga/tls.dns for it to work!

[dkebler (David Kebler)]

@danlsgiga Sorry as I mentioned I’m a go coder noob. Guess I’m gonna more explicit directions

In this case seems I can’t use xcaddy (doesn’t support replace) but must compile from the source?
ok so I cloned caddy2 source

So then I run mod edit -replace github.com/caddyserver/tls.dns=github.com/danlsgiga/tls.dns on the caddy 2 source right?

whoops another error

go mod: -replace=github.com/caddyserver/tls.dns=github.com/danlsgiga/tls.dns: unversioned new path must be local directory

I assume that github.com/caddyserver/tls.dns is already a module that caddy2 source pulls? cause I have no idea how to add modules with go build like xcaddy does --with.

hmm… if I look at go.mod github.com/caddyserver/tls.dns is not even listed` so maybe the error

Anyway noob confusion some steps appreciated.

[francislavoie]

I think the easiest solution is to make a fork of @danlsgiga’s repo, make a commit to change the go.mod file module github.com/caddyserver/tls.dns to module github.com/<your-username>/tls.dns then use github.com/<your-username>/tls.dns@<commit-hash> with xcaddy.

Not a permanent solution, but it should work for now.

[dkebler (David Kebler)]

ok that worked.

For others who might need more explicit directions

1. In your github account fork https://github.com/danlsgiga/tls.dns
2. Rather than pull down change/commit and push you can edit the file in place in your browser
   https://github.com/<username>/tls.dns/blob/master/go.mod
3. Edit the first line replacing caddyserver with your github username
4. At the bottom set a commit message and save
5. Go to repo main page and click on commits. Find your last commit and click on the copy the commit hash stamp.
6. then with xcaddy use --with github.com/<your-username>/tls.dns@<commit-hash> pasting in the commit-hash from step 5

I’m guessing this should work for anyone wanting to try a plugin/module that has yet to be merged into the caddy source.

[dkebler (David Kebler)]

now I guess my last question @danlsgiga

Is the route53 tls dns setup/directive changed from V1? or can I just refer to the V1 docs?

[danlsgiga (Daniel Santos)]

@dkebler, you can use the json config I’ve referenced here

[danlsgiga (Daniel Santos)]

Basically:

{
  "apps": {
    "tls": {
      "automation": {
        "policies": [{
          "issuer": {
            "module": "acme",
            "email": "my@email.com",
            "challenges": {
              "dns": {
                "provider": "route53"
              }
            }
          }
        }]
      }
    }
  }
}