Web server - cgi and webdav

Help
1

1. Caddy version:

v2.6.2

2. How I installed, and run Caddy:

systemctl reload caddy

a. System environment:

Ubuntu 22.04.1

b. Command:

systemctl reload caddy

d. My complete Caddy config:

{
	order webdav last
}
example.com {
	root * /home/www

	@notget not method GET

	route @notget {
	    basicauth {
			user1 $2a$14$YHmpP.202ARX0meMIEGlfeKt8MlMxqDxm8otEzL8zk0OXl4TPkMSS	        
	    }
	    webdav
	}
	file_server {
	}
}

3. The problem I’m having:

When using the caddy file above the web server and webdav work fine but
when we add the CGI section down below (to make the cgi scripts work)

example.com {
	root * /home/www

	@cgi path *.cgi
	try_files {path} /index.cgi
	reverse_proxy @cgi unix//var/run/fcgiwrap.socket {
		transport fastcgi {
			split .cgi
		}
	}

	@notget not method GET

	route @notget {
	    basicauth {
			user1 $2a$14$YHmpP.202ARX0meMIEGlfeKt8MlMxqDxm8otEzL8zk0OXl4TPkMSS	        
	    }
	    webdav
	}
	file_server {
	}
}

we get a user/pwd request in the browser when trying to access the web site
or the CGI scripts work fine but the webdav server stops working

if we create a subdomain and split the tasks WEB/CGI & DAV everything works fine
like in the caddy file below

{
	order webdav last
}
example.com {
	root * /local/www
	@cgi path *.cgi
	try_files {path} /index.cgi
	reverse_proxy @cgi unix//var/run/fcgiwrap.socket {
		transport fastcgi {
			split .cgi
		}
	}
	file_server {
	}
}

dav.example.com {
	root * /local/www

	@notget not method GET

	route @notget {
	    basicauth {
			user1 $2a$14$YHmpP.202ARX0meMIEGlfeKt8MlMxqDxm8otEzL8zk0OXl4TPkMSS
	    }
	    webdav
	}
}

but how do we keep all of this working without adding a subdomain ?

2

I’m not sure I understand the question.

Please describe in detail the various usecases you have, and for each, what you see happening now and what you expect to happen.

Enable the debug global option. Show your logs. Make example requests with curl -v to show the behaviour you’re seeing.

Please don’t remove any parts of the help topic template. It’s required that you properly fill it out, as per the forum rules.

3

the goal is to have a web server that can run scripts and webdav

the caddy file below works perfectly

{
	order webdav last
}
example.com {
	root * /local/www
	@cgi path *.cgi
	try_files {path} /index.cgi
	reverse_proxy @cgi unix//var/run/fcgiwrap.socket {
		transport fastcgi {
			split .cgi
		}
	}
	file_server {
	}
}

dav.example.com {
	root * /local/www

	@notget not method GET

	route @notget {
	    basicauth {
			user1 $2a$14$YHmpP.202ARX0meMIEGlfeKt8MlMxqDxm8otEzL8zk0OXl4TPkMSS
	    }
	    webdav
	}
}

but instead of having example.com for Web server with CGI and (a subdomain) dev.example.com for Webdav, we’d like to have Web, CGI and DAV configured into the same domain (example.com)

I’ll try to debug and post log files

4 
Feb 04 22:45:14 testing caddy[545]: {"level":"debug","ts":1675550714.7096848,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_ip":"69.162.81.155","remote_port":"63719","proto":"HTTP/3.0","method":"GET","host":"100e149.com","uri":"/css/camphor.woff2","headers":{"Referer":["https://100e149.com/test3.cgi"],"Accept-Encoding":["gzip, deflate, br"],"Accept":["*/*"],"Priority":["u=3, i"],"User-Agent":["Mozilla/5.0 (Ubuntu Custom AP395) Amiga/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15"],"Accept-Language":["en-US,en;q=0.9"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","server_name":""}},"method":"GET","uri":"/test.cgi"}
Feb 04 22:45:14 testing caddy[545]: {"level":"debug","ts":1675550714.7097487,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"/var/run/fcgiwrap.socket","total_upstreams":1}
Feb 04 22:45:14 testing caddy[545]: {"level":"debug","ts":1675550714.7098856,"logger":"http.reverse_proxy.transport.fastcgi","msg":"roundtrip","request":{"remote_ip":"69.162.81.155","remote_port":"63719","proto":"HTTP/3.0","method":"GET","host":"100e149.com","uri":"/test.cgi","headers":{"Referer":["https://100e149.com/test3.cgi"],"Accept":["*/*"],"Priority":["u=3, i"],"X-Forwarded-Proto":["https"],"X-Forwarded-Host":["100e149.com"],"Accept-Language":["en-US,en;q=0.9"],"Accept-Encoding":["gzip, deflate, br"],"User-Agent":["Mozilla/5.0 (Ubuntu Custom AP395) Amiga/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15"],"X-Forwarded-For":["69.162.81.155"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","server_name":""}},"env":{"AUTH_TYPE":"","CONTENT_TYPE":"","PATH_INFO":"","HTTP_PRIORITY":"u=3, i","HTTP_USER_AGENT":"Mozilla/5.0 (Ubuntu Custom AP395) Amiga/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15","HTTP_X_FORWARDED_FOR":"69.162.81.155","SERVER_NAME":"100e149.com","SERVER_PORT":"443","HTTP_X_FORWARDED_PROTO":"https","HTTP_X_FORWARDED_HOST":"100e149.com","REMOTE_IDENT":"","CONTENT_LENGTH":"","REMOTE_USER":"","SERVER_PROTOCOL":"HTTP/3.0","SCRIPT_FILENAME":"/var/www/test.cgi","REMOTE_ADDR":"69.162.81.155","HTTPS":"on","HTTP_ACCEPT_ENCODING":"gzip, deflate, br","GATEWAY_INTERFACE":"CGI/1.1","REMOTE_HOST":"69.162.81.155","DOCUMENT_ROOT":"/var/www","SCRIPT_NAME":"/test.cgi","HTTP_ACCEPT":"*/*","QUERY_STRING":"","REMOTE_PORT":"63719","REQUEST_SCHEME":"https","DOCUMENT_URI":"/test.cgi","REQUEST_URI":"/css/camphor.woff2","REQUEST_METHOD":"GET","SERVER_SOFTWARE":"Caddy/v2.6.2","HTTP_HOST":"100e149.com","HTTP_REFERER":"https://100e149.com/test3.cgi","HTTP_ACCEPT_LANGUAGE":"en-US,en;q=0.9"},"dial":"/var/run/fcgiwrap.socket","env":{"REQUEST_METHOD":"GET","SERVER_SOFTWARE":"Caddy/v2.6.2","HTTP_HOST":"100e149.com","HTTP_REFERER":"https://100e149.com/test3.cgi","HTTP_ACCEPT_LANGUAGE":"en-US,en;q=0.9","AUTH_TYPE":"","CONTENT_TYPE":"","PATH_INFO":"","HTTP_PRIORITY":"u=3, i","HTTP_USER_AGENT":"Mozilla/5.0 (Ubuntu Custom AP395) Amiga/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15","HTTP_X_FORWARDED_FOR":"69.162.81.155","SERVER_NAME":"100e149.com","SERVER_PORT":"443","HTTP_X_FORWARDED_PROTO":"https","HTTP_X_FORWARDED_HOST":"100e149.com","REMOTE_IDENT":"","CONTENT_LENGTH":"","REMOTE_USER":"","SERVER_PROTOCOL":"HTTP/3.0","SCRIPT_FILENAME":"/var/www/test.cgi","REMOTE_ADDR":"69.162.81.155","HTTPS":"on","HTTP_ACCEPT_ENCODING":"gzip, deflate, br","GATEWAY_INTERFACE":"CGI/1.1","REMOTE_HOST":"69.162.81.155","DOCUMENT_ROOT":"/var/www","SCRIPT_NAME":"/test.cgi","HTTP_ACCEPT":"*/*","QUERY_STRING":"","REMOTE_PORT":"63719","REQUEST_SCHEME":"https","DOCUMENT_URI":"/test.cgi","REQUEST_URI":"/css/camphor.woff2"},"request":{"remote_ip":"69.162.81.155","remote_port":"63719","proto":"HTTP/3.0","method":"GET","host":"100e149.com","uri":"/test.cgi","headers":{"X-Forwarded-Host":["100e149.com"],"Referer":["https://100e149.com/test3.cgi"],"Accept":["*/*"],"Priority":["u=3, i"],"X-Forwarded-Proto":["https"],"Accept-Language":["en-US,en;q=0.9"],"Accept-Encoding":["gzip, deflate, br"],"User-Agent":["Mozilla/5.0 (Ubuntu Custom AP395) Amiga/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15"],"X-Forwarded-For":["69.162.81.155"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","server_name":""}}}
Feb 04 22:45:14 testing caddy[545]: {"level":"debug","ts":1675550714.7114663,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"unix//var/run/fcgiwrap.socket","duration":0.001656379,"request":{"remote_ip":"69.162.81.155","remote_port":"63719","proto":"HTTP/3.0","method":"GET","host":"100e149.com","uri":"/test.cgi","headers":{"Referer":["https://100e149.com/test3.cgi"],"Accept":["*/*"],"Priority":["u=3, i"],"X-Forwarded-Proto":["https"],"X-Forwarded-Host":["100e149.com"],"Accept-Language":["en-US,en;q=0.9"],"Accept-Encoding":["gzip, deflate, br"],"User-Agent":["Mozilla/5.0 (Ubuntu Custom AP395) Amiga/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15"],"X-Forwarded-For":["69.162.81.155"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","server_name":""}},"headers":{"Content-Type":["text/plain"],"Status":["403 Forbidden"]},"status":403}
Feb 04 22:45:17 testing caddy[545]: {"level":"debug","ts":1675550717.4203448,"logger":"http.log.error.log0","msg":"not authenticated","request":{"remote_ip":"69.162.81.155","remote_port":"63719","proto":"HTTP/3.0","method":"POST","host":"100e149.com","uri":"/test4.cgi","headers":{"User-Agent":["Mozilla/5.0 (Ubuntu Custom AP395) Amiga/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15"],"Referer":["https://100e149.com/test3.cgi"],"Accept-Language":["en-US,en;q=0.9"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"],"Content-Type":["application/x-www-form-urlencoded"],"Origin":["https://100e149.com"],"Accept-Encoding":["gzip, deflate, br"],"Priority":["u=0, i"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","server_name":""}},"duration":0.000225538,"status":401,"err_id":"bwnhqqtes","err_trace":"caddyauth.Authentication.ServeHTTP (caddyauth.go:88)"}
Feb 04 22:45:20 testing caddy[545]: {"level":"debug","ts":1675550720.2893493,"logger":"http.log.error.log0","msg":"not authenticated","request":{"remote_ip":"69.162.81.155","remote_port":"63719","proto":"HTTP/3.0","method":"POST","host":"100e149.com","uri":"/test4.cgi","headers":{"Origin":["https://100e149.com"],"Accept-Encoding":["gzip, deflate, br"],"Priority":["u=0, i"],"User-Agent":["Mozilla/5.0 (Ubuntu Custom AP395) Amiga/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15"],"Referer":["https://100e149.com/test3.cgi"],"Accept-Language":["en-US,en;q=0.9"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"],"Content-Type":["application/x-www-form-urlencoded"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","server_name":""}},"duration":0.000370908,"status":401,"err_id":"t0wpi196d","err_trace":"caddyauth.Authentication.ServeHTTP (caddyauth.go:88)"}
(END)
5 
{
	debug
	order webdav last
}
example.com {
    root * /local/www

	@cgi path *.cgi
	try_files {path} /index.cgi
	reverse_proxy @cgi unix//var/run/fcgiwrap.socket {
		transport fastcgi {
			split .cgi
		}
	}

	file_server {
	}

	@notget not method GET

	route @notget {
	    basicauth {
	        user1 $2a$14$u4SEN81Xb8s5J30.pJSvn.rSvTo5ZfQ7naHFBlSOTEsp6Yb8i/Dzu
	    }
	    webdav
	}

}

we are using this caddy file now

it works fine, but (only sometimes) when we try to access a web page on the server using a local CGI script, we get a log-in window asking user name and password

like instead of showing a web page built by the script, it prompts a webdav log-in

I think somewhere in this caddy file the priority of the CGI over wevdav needs to be tweaked

does this caddy file look properly formatted to you ?

6

Directives are sorted according to a predetermined directive order:

If you mean that you only want basicauth to happen with webdav (which is not clear to me from your explanation – it would help if you laid out exactly what you want to happen, with explicit examples, as I asked) then you should probably wrap each part handle blocks to make them mutually exclusive.

7

I apologize for not being clear…my brain is melting down

yes we only want basicauth to happen with webdav and I don’t know how to wrap each block separately and correctly…

8

Maybe something like this:

{
	debug
	order webdav last
}

example.com {
	root * /local/www

	try_files {path} /index.cgi

	@cgi path *.cgi
	handle @cgi {
		reverse_proxy unix//var/run/fcgiwrap.socket {
			transport fastcgi {
				split .cgi
			}
		}
	}

	@notget not method GET
	handle @notget {
		basicauth {
			user1 $2a$14$u4SEN81Xb8s5J30.pJSvn.rSvTo5ZfQ7naHFBlSOTEsp6Yb8i/Dzu
		}
		webdav
	}

	handle {
		file_server
	}
}

In order:

  • If the request path doesn’t map to file that exists on disk, then rewrite to /index.cgi
  • If the request is *.cgi (e.g. was rewritten) then proxy to CGI
  • If the request is not GET, basicauth + webdav
  • Else server the file with file_server

The handle directive ensures those blocks are mutually exclusive, so only the first matching one will run (and the last one with no matcher is a fallback).

9

thank you!!

this last caddy file makes the CGI and Web server work but not the webdav

I have to remove the line below

try_files {path} /index.cgi

in order for webdav to work

10

:man_shrugging:

Like I said you haven’t shown specific examples of requests that should work, so I’ve had to make assumptions.

Please be specific. What requests (i.e. paths, headers, HTTP method) should do what? To craft request matchers to do what you need, you need to figure out what those requests look like.

11

we use caddy to host web sites and webdav with standard FTP/webdav client software in order to access and modify files

the feature of being able to read a CGI script by default instead of an index.html is not necessary at the moment so we can live without it

but thank you so much for the last file, we are now able to serve scripts and the webdav is working properly!!

12

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.