Rclone plugin for caddy

Plugins
1

I’ve been wondering about making an rclone plugin for caddy.

rclone has interfaces to >20 cloud providers which it would be nice for caddy to be able to serve or use as input to any of the other plugins.

rclone can provide an http.FileSystem which caddy could use as it’s root file system.

Being #lazydev I tried to find another plugin which does this but I failed, which made me wonder whether this is possible or not? I didn’t see this type of plugin listed on the plugins page.

rclone needs a bit of internal re-organisation to make this possible (so it is more usable as a library) but making a plugin for Caddy would be good motivation for me to get this work done.

So, is this possible? And apologies in advance if I failed to find the documentation.

3 Likes
2

Hi Nick! Great to have you here. (No worries, we don’t have great docs yet. It’ll happen someday!)

Yep, this is possible, and sounds like a neat idea. Let me know if this sounds right: So you could use, say, an rclone directive to “proxy” requests directly to cloud storage (or any of rclone’s supported providers). In other words: upload your site to your Dropbox or S3 and then serve those files via Caddy, instead of the files being local to the machine Caddy is on.

The browse directive also makes a staticfiles.FileServer, so you could even say browse in your Caddyfile and have it serve its listings through rclone.

I think that’s really cool.

As for providing an http.FileSystem, I am not 100% sure off the top of my head if this is replaceable by plugins. It’s probably not. The HTTP server sets this up here: https://github.com/mholt/caddy/blob/2922d09bef3c504dde66bc12f7441668fcef6a20/caddyhttp/httpserver/server.go#L145

I could look into making this configurable by plugins, but it’d need some discussion on the best design. (Also see how the browse plugin uses it: https://github.com/mholt/caddy/blob/2922d09bef3c504dde66bc12f7441668fcef6a20/caddyhttp/browse/setup.go#L80-L84)

So, yes, it’s possible… with some minor changes to Caddy’s HTTP server. I’m happy to help make this happen.

1 Like
3

Yes that is the basic idea.

This is possible already in several ways

  • use rclone mount to mount the remote filesystem and serve it with caddy - this is how I serve beta.rclone.org
  • use rclone serve http or rclone serve webdav and proxy those via caddy (I haven’t actually tried this but it should work!)

However all of those suffer from their being unnecessary set of requests going on, either filesystem requests via FUSE through the kernel back and forth to rclone or HTTP requests.

I see! Yes an existing browse-like plugin would fit into the caddy plugin scheme as it stands…

Ah ha! I tried to find that bit of code but I didn’t succeed :wink:

What the plugin needs to do is provide a replacement for http.Dir(site.Root) in that line.

What I might try is a hideous bodge to see if I can get something working then we can discuss a better way of doing it! Unless you have an idea of how to start out initially?

Since I’m some way from re-architecting rclone so it would actually work as a library I might just do a test file system as a proof of concept.

How does that sound as a first attempt?

2 Likes
4

I don’t have a great design idea at the moment, but I will say that it’s not uncommon for plugins to import each other (as long as there’s no cycles of course!) – the staticfiles plugin stands pretty much on its own, so it can be imported. Your plugin could potentially import the browse plugin and register itself as a plugin to the plugin, as a backend for browse.

I hope that made sense.

Anyway, yeah, feel free to go for it! I will be happy to take a look at the PR. No hurry of course. It’s neat you’re thinking about it.

5

Sounds like a great idea. I wonder if it could also serve as a TLS asset storage plugin?

6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.