PUT a file to filesystem or forward request to a service on different port

1. Caddy version (caddy version):

v2.4.3

2. How I run Caddy:

  • Command I run Caddy with: caddy start (or caddy reload afterwards)
  • I use Caddy as a reverse_proxy for OshiUpload file upload service. It works from web browser UI (file uploading, file downloading, file removing) and also works with HTTPS; no issues. However, main idea behind Oshi is to use it from CLI with curl, like: curl -T yourfile.txt https://domain.com and then you get a download link in return.
    I’ve made this Caddyfile:
aim.us.to {
        reverse_proxy 127.0.0.1:4019
}

cl.aim.us.to {
        reverse_proxy 127.0.0.1:4020
}

First one is the “Web-UI” and it works OK. Second one is the “curl -T” interface (which does CURL PUT request). The request always fails with 502:

< HTTP/2 502

So, can I tell Caddy to forward files that was PUT into it without changing the PUT request to localhost:4020? How could I achieve this?

a. System environment:

  • Bedrock Linux 0.7.23 Poki on top of Debian 9 with systemd

b. Command:

caddy start

c. Service/unit/compose file:

This is oshi config file, the part with ports in use

# HTTP main app & POST uploads listen address and port, * for listening on all interfaces
HTTP_APP_ADDRESS = *
HTTP_APP_PORT = 4019

# HTTP PUT uploads listen address and port
HTTP_PUT_ADDRESS = ::
HTTP_PUT_PORT = 4020

d. My complete Caddyfile or JSON config:

aim.us.to {
        reverse_proxy 127.0.0.1:4019
}

cl.aim.us.to {
        reverse_proxy 127.0.0.1:4020
}

3. The problem I’m having:

I use Caddy as a reverse_proxy for OshiUpload file upload service. It works from web browser UI (file uploading, file downloading, file removing) and also works with HTTPS; no issues. However, main idea behind Oshi is to use it from CLI with curl, like: curl -T yourfile.txt https://domain.com and then you get a download link in return.

4. Error messages and/or full log output:

< HTTP/2 502

5. What I already tried:

  • Changing from HTTPS to HTTP with http:// and :80 adding into Caddyfile
  • Searching on how to enable PUT for reverse proxy

6. Links to relevant resources:

N/A

What’s in Caddy’s logs?

Try running with caddy run instead of caddy start, so that you can see the logs.

Running with caddy start is not recommended for a permanent server, because it won’t restart automatically when the machine boots. Please run Caddy as a system service instead.

Here is the log @francislavoie :

2021/08/28 03:26:13.594	INFO	using adjacent Caddyfile
2021/08/28 03:26:13.612	INFO	admin	admin endpoint started	{"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127.0.0.1:2019"]}
2021/08/28 03:26:13.613	INFO	http	server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS	{"server_name": "srv0", "https_port": 443}
2021/08/28 03:26:13.613	INFO	tls.cache.maintenance	started background certificate maintenance	{"cache": "0xc000426af0"}
2021/08/28 03:26:13.613	INFO	http	enabling automatic HTTP->HTTPS redirects	{"server_name": "srv0"}
2021/08/28 03:26:13.614	INFO	http	enabling automatic TLS certificate management	{"domains": ["cl.aim.us.to", "aim.us.to"]}
2021/08/28 03:26:13.614	INFO	tls	cleaning storage unit	{"description": "FileStorage:/root/.local/share/caddy"}
2021/08/28 03:26:13.619	INFO	tls	finished cleaning storage units
2021/08/28 03:26:13.641	INFO	autosaved config (load with --resume flag)	{"file": "/root/.local/share/caddy/autosave.json"}
2021/08/28 03:26:13.642	INFO	serving initial configuration

About running; I will move Caddy to systemctl service after I understand if it can be used for curl PUT transfers. Thanks for advices.

That’s only Caddy’s startup. Make a request, then check the logs again.

Also, you can add this at the top of your Caddyfile to get additional information in the logs:

{
	debug
}

Ah, sure. Here is the error:

2021/08/28 21:50:28.675	ERROR	http.log.error	dial tcp 127.0.0.1:4020: connect: connection refused	{"request": {"remote_addr": "84.209.96.137:45612", "proto": "HTTP/2.0", "method": "PUT", "host": "cl.aim.us.to", "uri": "/testfile", "headers": {"User-Agent": ["curl/7.76.1"], "Accept": ["*/*"], "Content-Length": ["11"]}, "tls": {"resumed": false, "version": 772, "cipher_suite": 4865, "proto": "h2", "proto_mutual": true, "server_name": "cl.aim.us.to"}}, "duration": 0.000551022, "status": 502, "err_id": "zzzfzkrcu", "err_trace": "reverseproxy.statusError (reverseproxy.go:857)"}

Interestingly, changed the oshi app config to:

# HTTP PUT uploads listen address and port
#HTTP_PUT_ADDRESS = :: - this one I commented out and put this instead:
HTTP_PUT_ADDRESS = *
HTTP_PUT_PORT = 4020

Now, when I do:

(base) [sm@fedora-mbp test]$ curl -T testfile -vvvv https://cl.aim.us.to
*   Trying 162.248.160.11:443...
* connect to 162.248.160.11 port 443 failed: Connection refused
(base) [sm@fedora-mbp test]$ curl -T testfile -vvvv cl.aim.us.to
* connect to 162.248.160.11 port 80 failed: Connection refused

Finally, if I do it with a port:

curl -T testfile -vvvv cl.aim.us.to:4020

It works OK. The thing is, I was thing to bind cl.* subdomain to port 4020 via Caddy.

Seems like I fixed it myself. The issues was:

  • Wrong config for OshiUpload service (for the PUT that I showed above)
  • Wrong systemd unit (had to add WorkingDirectory argument to systemd unit; then it worked)

Now everything works.
Thanks for reaching up for help! :slight_smile:

1 Like

Glad you figured it out! :tada:

This topic was automatically closed after 30 days. New replies are no longer allowed.