Webdav reverse proxy (can't upload file)

giopas · October 4, 2019, 2:35pm

1. My Caddy version (`caddy -version`):

“unknown” (but it should be 1.03)

2. How I run Caddy:

Please provide all of the relevant information and DO NOT REDACT anything except passwords/keys! Thank you!

/etc/init.d/Qthttpd stop; caddy -conf /share/Public/caddy/caddy.conf -agree -email sub@domain.com -root /home/Qhttpd -log $QPKG_ROOT/var/logs/caddy.log -pidfile /var/run/caddy.pid &

a. System environment:

OS, relevant versions, systemd? docker? etc.

QNAP NAS (QTS)

b. Command:

none

c. Service/unit/compose file:

none

d. My complete Caddyfile:

(gzipconf) {
	gzip {
		ext *
		level 7
		min_length 1
	}
}

(addheader) {
	header / {
		Strict-Transport-Security "max-age=31536000;"
		X-XSS-Protection "1; mode=block"
		X-Content-Type-Options "nosniff"
		X-Frame-Options "DENY"
		Referrer-Policy "strict-origin-when-cross-origin"
		-Server
	}
}

webdav.domain.com {
	import gzipconf
	import addheader
	proxy / http://192.168.152.10:5000 {
		keepalive 32
		transparent
		websocket
	}
}

3. The problem I’m having:

Please describe the issue thoroughly enough so that anyone can reproduce the exact behavior you’re seeing. Be as specific as possible.

This is a webdav repository service through a QNAP NAS server (HTTP port 5000). Therefore I am not looking to use Caddy for webdav functionalities (via the plugin).

If I try to access to this webdav repository via Caddy (in HTTPS), I can read, create and delete files, but I cannot save any change, otherwise I receive a writing error and the file I tried to changed is renamed as *.0.tmp.

This in practice corrupts the file.

4. Error messages and/or full log output:

Please DO NOT REDACT any information except passwords/keys!

On the webdav client: “Failed to upload”

5. What I already tried:

Accessing the webdav repository via Caddy in HTTP works fine.

http://webdav.domain.com {
	import gzipconf
	import addheader
	proxy / http://192.168.152.10:5000 {
		keepalive 32
		transparent
		websocket
	}
}

6. Links to relevant resources:

none

giopas · October 5, 2019, 6:43am

I did another type of test: I tried to connect with a client to the webdav server through the LAN, using the address http://192.168.152.10:5000 and I could create, edit and save files.

Any idea where could be the problem with Caddy in HTTPS?

Thanks

giopas · October 6, 2019, 7:17am

After a restart of the webdav server, it seems that the error is no longer there (at least within the LAN in https).

However I cannot confirm yet via caddy as I get problems with the certificates: “Too many certificates already issued for domain”, probably because after each Caddy restart the certificates are deleted and not saved them. By the way, how can I fix this? Can I point in the launch command to a specific folder I know it is not deleted?

Whitestrake · October 6, 2019, 12:02pm

Yes, you can. By default, Caddy saves certificates in $HOME/.caddy. You can override this behaviour by setting the CADDYPATH env var.

https://caddyserver.com/docs/cli#caddypath

giopas · October 6, 2019, 1:42pm

Thank you!

How can I use this? In the config file or in the launch command?

Can I ask you a simple example?

Indeed, as I am making quite a few tests, I would like to avoid to be blocked by letsencrypt

giopas · October 7, 2019, 5:33am

Here is the Caddy.sh script, made by the QNAP Caddy packager, that I have slightly customized with email and conf location. Where and how should I put the CADDYPATH variable?

#!/bin/sh
CONF=/etc/config/qpkg.conf
QPKG_NAME="Caddy"
QPKG_ROOT=`/sbin/getcfg $QPKG_NAME Install_Path -f ${CONF}`
APACHE_ROOT=`/sbin/getcfg SHARE_DEF defWeb -d Qweb -f /etc/config/def_share.info`
export QNAP_QPKG=$QPKG_NAME

export QPKG_ROOT
export QPKG_NAME
export APACHE_ROOT

export SHELL=/bin/sh
export LC_ALL=en_US.UTF-8
export USER=admin
export LANG=en_US.UTF-8
export LC_CTYPE=en_US.UTF-8

ulimit -n 8192

export PIDF=/var/run/caddy.pid

case "$1" in
  start)
    ENABLED=$(/sbin/getcfg $QPKG_NAME Enable -u -d FALSE -f $CONF)
    if [ "$ENABLED" != "TRUE" ]; then
        echo "$QPKG_NAME is disabled."
        exit 1
    fi
/bin/ln -sf $QPKG_ROOT /opt/$QPKG_NAME
/bin/ln -sf $QPKG_ROOT/caddy /usr/bin/caddy

/etc/init.d/Qthttpd stop ; caddy -conf /share/Public/caddy/caddy4.conf -agree -email mail@domain.com -root /home/Qhttpd -log /share/Public/caddy/caddy.log -pidfile /var/run/caddy.pid &

    ;;

  stop)

ID=$(more /var/run/caddy.pid)

        if [ -e $PIDF ]; then
            kill -9 $ID
            rm -f $PIDF
        fi


rm -rf /opt/$QPKG_NAME
rm -rf /usr/bin/caddy

    ;;

  restart)
    $0 stop
    $0 start
    ;;

  *)
    echo "Usage: $0 {start|stop|restart}"
    exit 1
esac

exit 0

Thank you!

EDIT: ok, I have understood: I just need to add to the Caddy.sh file the following line:

export CADDYPATH=/share/Public/Caddy

Now I can try to test the original problem of webdav again.

giopas · October 7, 2019, 1:57pm

Nope, I still receive the same upload error, even after a fresh restart.

what else could I look at?

Whitestrake · October 8, 2019, 1:14am

Hmm.

Some general things to test:

Does it do this with any file or just specific ones? Large size, small size, long filename, short filename?
Does the error occur in all browsers? (Safari/Edge, Chrome, Firefox?)

giopas · October 8, 2019, 5:18am

Thank for your answer.

I use webdav for KeePass (a password manager), which uses a small file (few Kb) as a database which can be loaded directly through the various KeePass clients. But the error also occurs on simple txt files that I can create for testing purposes with a simple webdav client (CX File Explorer, on Android).

I can reconfirm that with Caddy on HTTP the error does not occur.

Whitestrake · October 8, 2019, 5:21am

I’m wondering if the issue is with the clients trying to make connections over HTTP without regard to the client’s scheme.

Can you keep the browser tools open on the Network tab while reproducing this issue and see if there are any anomalies?

giopas · October 9, 2019, 5:45am

I tried several times, both in http and https and both within the LAN (private IP address) and from outside of the LAN (using Caddy) and I always receive access denied on the browser.

Forbidden

You don’t have permission to access /folder/ on this server.

It seems [1] there is a setting on the server which block the use of webdav through a browser as with a webdav client I can access the repository but I cannot edit files with Caddy in https.

Is there another way to find where is the error on Caddy?

[1] Keine Zugriff via WebDAV - Netzwerkeinstellungen - NAS Hilfe und Support Forum

Whitestrake · October 9, 2019, 5:54am

Ahh, unfortunately I can’t read German.

I’m not convinced there is an error in Caddy.

Conceptually, Caddy’s reverse proxy is very simple. You make request to Caddy. Caddy notes the particulars of the request. Then it makes a new request to the upstream server. Then it takes the response from the upstream server, and it returns that response to you.

If the site works in a browser when accessed directly, it should work in a browser when proxied through Caddy. Again, the reverse proxy is not very complex conceptually, and neither is a HTTP request. If it doesn’t work on a browser directly, it won’t work in a browser through Caddy.

giopas · October 9, 2019, 6:07am

No, the site (webdav repository) does not work at all in a browser. Only with webdav enabled clients (but with errors in https via Caddy).

I will open a ticket with Qnap, but I cannot exclude it is Caddy related…

Whitestrake · October 9, 2019, 6:21am

It might be Caddy related, I just hesitate to say it’s an error in Caddy.

Remember that Caddy’s proxy directive proxies standard HTTP, and that WebDAV is an extension to HTTP (i.e. nonstandard). This might just be a case of the client/server using WebDAV functions that Caddy isn’t handling, I’m not sure - but notably Caddy is similarly unable to fully proxy Remote Desktop Gateway (insofar it uses some kind of MSRPC over HTTP, establishing a tunnel of sorts for RDP communication).

Edit: Looking into this further, I don’t have QNAP specifically, but I do reverse proxy my own Nextcloud instance, which is accessible via WebDAV. Previously I’ve used the Nextcloud client for syncing, but I just tested adding the WebDAV endpoint via Finder on MacOS. Everything’s working fine in that regard, so whether Nextcloud/MacOS Finder are using different methods than QNAP server/QNAP client, I’m not sure… But I was able to add, delete, rename etc. files at random.

giopas · October 9, 2019, 7:01am

Thanks for having tested it. Does it also work in https?

I will try to escalate to the Qnap support. Thank you very much!

Whitestrake · October 9, 2019, 7:04am

Yes, my Nextcloud instance itself is inaccessible except via Caddy (the Docker network is the only network connection it has, with Caddy as the ingress point).

The URL I use in MacOS Finder’s “Connect to Server” window is https://nextcloud.whitestrake.net/remote.php/dav/files/Whitestrake/.

Attempts to connect via HTTP would be issued a redirect to HTTPS.

Another edit: Reviewing my server logs, the vast majority of access to Nextcloud is at this endpoint - implying that the Nextcloud client I’ve been running all this time also uses WebDAV (or at least the WebDAV endpoint) too, and it’s functioning perfectly.

system · January 7, 2020, 7:04am

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.

Webdav reverse proxy (can't upload file)

1. My Caddy version (caddy -version):

2. How I run Caddy:

a. System environment:

b. Command:

c. Service/unit/compose file:

d. My complete Caddyfile:

3. The problem I’m having:

4. Error messages and/or full log output:

5. What I already tried:

6. Links to relevant resources:

Forbidden

1. My Caddy version (`caddy -version`):