Caddy's download api server is not responding

[BilalBudhani]

1. Caddy version (caddy version):

NA

2. How I run Caddy:

NA

a. System environment:

Ubuntu 22.04 VPS from DigitalOcean

b. Command:

https://caddyserver.com/api/download?os=linux&arch=amd64&p=github.com%2Fcaddy-dns%2Falidns&p=github.com%2Fcaddy-dns%2Fazure&p=github.com%2Fcaddy-dns%2Fcloudflare&p=github.com%2Fcaddy-dns%2Fdigitalocean&p=github.com%2Fcaddy-dns%2Fdnspod&p=github.com%2Fcaddy-dns%2Fduckdns&p=github.com%2Fcaddy-dns%2Fgandi&p=github.com%2Fcaddy-dns%2Fhetzner&p=github.com%2Fcaddy-dns%2Flego-deprecated&p=github.com%2Fcaddy-dns%2Froute53&p=github.com%2Fcaddy-dns%2Fvultr&idempotency=9360055365624

c. Service/unit/compose file:

NA

d. My complete Caddyfile or JSON config:

NA

3. The problem I’m having:

I’m trying to download Caddy from the above mentioned URL which was working earlier but since yesterday it is not working

4. Error messages and/or full log output:

I don’t get any error message, It just doesn’t respond to the request

5. What I already tried:

I tried from multiple different servers and my local machine to download but nothing works

6. Links to relevant resources:

NA

[matt (Matt Holt)]

I swear, DigitalOcean has the most obscure networking issues. The build server has apparently been dropping out of connectivity intermittently for days. Looks like it’s getting worse lately. I can’t even SSH into it.

I’ve had this issue with other droplets before, so it’s not just the build server. Sigh.

Forced a power cycle, hopefully that will fix it.

[BilalBudhani]

It doesn’t seem to fix the issue.

[vincent (Vincent Ritter)]

I can confirm this issue here also. However, I don’t think this is a DNS issue as such.

The issue is the multiple uses of the p keys in the request params. If you only use one it will trigger a download, for example:
https://caddyserver.com/api/download?os=linux&arch=amd64&p=github.com/caddy-dns/cloudflare

I’ve tried this with multiple different p param values and they all seem to work if you only use one.

[Edit] - To follow up on this a little, a request with multiple p params returns a 400 HTTP status code.

[matt (Matt Holt)]

Gah. It was working, but DigitalOcean is doing it again. Here’s a couple snapshots from their own metrics utility:

You can see how it intermittently cuts out. I can’t even SSH into the instance when this happens. It’s like they drop connectivity to the thing entirely and randomly. And I’ve seen this happen with multiple droplets over the last several months. Their support hasn’t been helpful in diagnosing the problem.

I’ll power cycle again. I’m not sure what else to do.

(@vincent I’m also pretty sure it’s not a DNS issue.)

[vincent (Vincent Ritter)]

Yeah, this is weird for sure.

I resolved the issue by using the API endpoint without the p params.

Using the original URL quoted in this thread always fails… no matter what I do.

Shame DO is having issues like that.

[matt (Matt Holt)]

Some builds will succeed if they’ve already been built and are thus cached. Unique builds will fail when the build server is unreachable.

[BilalBudhani]

You can see how it intermittently cuts out. I can’t even SSH into the instance when this happens. It’s like they drop connectivity to the thing entirely and randomly. And I’ve seen this happen with multiple droplets over the last several months. Their support hasn’t been helpful in diagnosing the problem.

As a long time DigitalOcean enthusiast, this issue is so unfortunate to witness. Their service has been not so great lately.

We (hatchbox.io) help customers in provisioning servers for various cloud providers, our Caddy setup script is failing because it can’t move past the “downloading caddy” step.

thinking out loud here
Is it possible to open-source the build and download code which would allow others to set up mirrors of the service?

[francislavoie]

@BilalBudhani

We (hatchbox.io) help customers in provisioning servers for various cloud providers, our Caddy setup script is failing because it can’t move past the “downloading caddy” step.

You can build yourself with xcaddy. The website is just a wrapper over xcaddy.

GitHub - caddyserver/xcaddy: Build Caddy with plugins

[matt (Matt Holt)]

Yeah, definitely don’t script against the download page, especially don’t rely on it – maybe this is something I can spend time and money improving if the sponsorships are there for it.

But like Francis said, use xcaddy instead. The build server is just running xcaddy.

[BilalBudhani]

Thank you for all the help, as suggested here, we wrote our own wrapper on xcaddy and hosting the binary at our server to make it work