V2 simple root/file_server returns nothing

1. Caddy version (caddy version):

caddy2 2.1.1-4 from aur

2. How I run Caddy:

a. System environment:

arch linux (5.7.10), systemd

b. Command:

sudo systemctl start caddy

c. Service/unit/compose file:

This is the service provided by the aur package

[Unit]
Description=Caddy webserver
Documentation=https://caddyserver.com/docs/
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service
StartLimitIntervalSec=14400
StartLimitBurst=10

[Service]
User=caddy
Group=caddy
Environment=XDG_DATA_HOME=/var/lib
Environment=XDG_CONFIG_HOME=/etc
ExecStart=/usr/bin/caddy run --adapter caddyfile --environ --config /etc/caddy/caddy.conf
ExecReload=/usr/bin/caddy reload --adapter caddyfile --config /etc/caddy/caddy.conf

# Do not allow the process to be restarted in a tight loop. If the
# process fails to start, something critical needs to be fixed.
Restart=on-abnormal

# Use graceful shutdown with a reasonable timeout
KillMode=mixed
KillSignal=SIGQUIT
TimeoutStopSec=5s

LimitNOFILE=1048576
LimitNPROC=512

# Hardening options
PrivateTmp=true
PrivateDevices=true
ProtectHome=true
ProtectSystem=strict
ReadWritePaths=/var/lib/caddy /var/log/caddy
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
AmbientCapabilities=CAP_NET_BIND_SERVICE
NoNewPrivileges=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
LockPersonality=true

[Install]
WantedBy=multi-user.target

d. My complete Caddyfile or JSON config:

csos95.com, www.csos95.com, blog.csos95.com {
	root * /var/www/blog/public
	file_server
}

ssilva.work, www.ssilva.work {
	root * /var/www/ssilva
	file_server
}

csilva.dev {
	root * /var/www/csilva
	file_server
}

rss.csos95.com {
	reverse_proxy localhost:2000
}

derpsin.space {
	root * /home/csos95/dockerfiles/mastodon/public
	file_server
	encode gzip

	header Strict-Transport-Security "max-age=31536000;"
	header /emoji Cache-Control "public, max-age=31536000, immutable"
	header /packs Cache-Control "public, max-age=31536000, immutable"
	header /system/accounts/avatars Cache-Control "public, max-age=31536000, immutable"
	header /system/media_attachments/files Cache-Control "public, max-age=31536000, immutable"

	handle_errors {
		rewrite * /500.html
		file_server
	}

	@rewrite1 {
		path /
	}
	rewrite @rewrite1 /proxy{path}

	@rewrite2 {
		not path /api/v1/streaming
	}
	rewrite @rewrite2 /proxy{path}

	@websockets {
		header Connection *Upgrade*
		header Upgrade websocket
	}

	reverse_proxy @websockets /proxy localhost:3000
}

wiki.csos95.com {
	reverse_proxy localhost:4080
}

wallabag.csos95.com {
	reverse_proxy localhost:5000
}

fimr.csos95.com {
	reverse_proxy localhost:6000
}

git.csos95.com, gitlab.csos95.com {
	reverse_proxy localhost:6080
}

write.csos95.com {
	reverse_proxy localhost:8181
}

plex.csos95.com {
	reverse_proxy localhost:32400
}

3. The problem I’m having:

The first three blocks with just root/file_server and the derpsin.space blocks do not work.
When I try to go to/curl those urls they return nothing.

4. Error messages and/or full log output:

There are no errors showing in sudo systemctl status caddy and the /var/log/caddy folder is empty.
If there’s a option I need to add to enable logging please let me know.

5. What I already tried:

I went through the docs and the migration guide and the blocks that are just reverse_proxy are working, but I can’t get any output at all when calling the others.

Could you make a request with curl -v and show us the output?

Also could you check that the caddy user has permission to read files from those directories?

> curl -v "https://csos95.com"
*   Trying 68.201.21.63:443...
* Connected to csos95.com (68.201.21.63) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=csos95.com
*  start date: Jul 26 06:20:50 2020 GMT
*  expire date: Oct 24 06:20:50 2020 GMT
*  subjectAltName: host "csos95.com" matched cert's "csos95.com"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x559ce1f389f0)
> GET / HTTP/2
> Host: csos95.com
> user-agent: curl/7.71.1
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 403 
< server: Caddy
< content-length: 0
< date: Sun, 26 Jul 2020 19:32:41 GMT
< 
* Connection #0 to host csos95.com left intact

The directory and all files inside /var/www/blog/public were owned by csos95:users and had permissions 744 so the caddy user should have had permission to read them.
Just to be safe I set the files to 777 and that didn’t work so I also set the owner to caddy:caddy and that didn’t work either.

Is it possible that you have some SELinux policies involved that might block access to those files? That 403 response definitely is from Caddy getting a permission error from the system call.

There’s only two places in the code that could originate from, and both of them are from OS permission errors:

1 Like

I don’t think I’ve ever messed with SELinux stuff. Do you know how I would check that?

I went and looked back at my old systemd service and I think have I had this issue before.
The old service set the user/group as www-data, but I had that commented out and I’m remembering it wasn’t working and I couldn’t figure it out, but once I removed that it worked.

I commented out the user/group and followed the errors until it worked then tried various combinations of the changes.

The current changes are:

  • commented out user and group
  • set HOME to the caddy users home
  • set ProtectHome=false (forgot to do that at the start for derpsin.space)
  • commented out CapabilityBoundingSet

I’d like to figure out why the caddy user doesn’t have permissions, but at least it’s working right now.

So now the first three site blocks are working.
derpsin.space isn’t working, but I think it’s because I translated the config incorrectly.

Here is the old site block

derpsin.space {
    root /home/csos95/dockerfiles/mastodon/public
    gzip

    header / {
        Strict-Transport-Security "max-age=31536000;"
    }

    header /emoji Cache-Control "public, max-age=31536000, immutable"
    header /packs Cache-Control "public, max-age=31536000, immutable"
    header /system/accounts/avatars Cache-Control "public, max-age=31536000, immutable"
    header /system/media_attachments/files Cache-Control "public, max-age=31536000, immutable"

    errors {
        * 500.html
    }

    rewrite {
    if {path} is /
    to /proxy{path}
    }

    rewrite {
        if {path} not_has /api/v1/streaming
        to {path} /proxy{path}
    }

    proxy /proxy 127.0.0.1:3000 {
        without /proxy

        transparent
        websocket
    }

    proxy /api/v1/streaming 127.0.0.1:4000 {
        transparent
        websocket
    }
}

Here is the new site block

derpsin.space {
	root * /home/csos95/dockerfiles/mastodon/public
	file_server
	encode gzip

	header Strict-Transport-Security "max-age=31536000;"
	header /emoji Cache-Control "public, max-age=31536000, immutable"
	header /packs Cache-Control "public, max-age=31536000, immutable"
	header /system/accounts/avatars Cache-Control "public, max-age=31536000, immutable"
	header /system/media_attachments/files Cache-Control "public, max-age=31536000, immutable"

	handle_errors {
		rewrite * /500.html
		file_server
	}

	@rewrite1 {
		path /
	}
	rewrite @rewrite1 /proxy{path}

	@rewrite2 {
		not path /api/v1/streaming
	}
	rewrite @rewrite2 /proxy{path}

	@websockets {
		header Connection *Upgrade*
		header Upgrade websocket
	}

	reverse_proxy @websockets /proxy localhost:3000
}

EDIT:

Ok I found this thread and was able to fix my mastodon block.

Here’s the current one that works.

derpsin.space {
	@try_masto {
		file
	}

	@websockets {
		header Connection *Upgrade*
		header Upgrade websocket
	}

	root * /home/csos95/dockerfiles/mastodon/public

	handle @try_masto {
		file_server *
	}

	handle /api/v1/streaming* {
		reverse_proxy @websockets localhost:4000
	}

	handle {
		reverse_proxy localhost:3000
	}

	encode gzip

	header {
		Strict-Transport-Security "max-age=31536000;"
	}

	header /emoji* Cache-Control "public, max-age=31536000, immutable"
	header /packs* Cache-Control "public, max-age=31536000, immutable"
	header /system/accounts/avatars* Cache-Control "public, max-age=31536000, immutable"
	header /system/media_attachments/files* Cache-Control "public, max-age=31536000, immutable"

	handle_errors {
		root * /home/csos95/dockerfiles/mastodon/public
		rewrite 500.html
		file_server
	}
}
1 Like

Here’s a bit of simplification of the Caddyfile:

derpsin.space {
	root * /home/csos95/dockerfiles/mastodon/public

	@try_masto file
	handle @try_masto {
		file_server
	}

	handle /api/v1/streaming* {
		reverse_proxy localhost:4000
	}

	handle {
		reverse_proxy localhost:3000
	}

	encode gzip

	header Strict-Transport-Security "max-age=31536000;"
	header /emoji* Cache-Control "public, max-age=31536000, immutable"
	header /packs* Cache-Control "public, max-age=31536000, immutable"
	header /system/accounts/avatars* Cache-Control "public, max-age=31536000, immutable"
	header /system/media_attachments/files* Cache-Control "public, max-age=31536000, immutable"

	handle_errors {
		root * /home/csos95/dockerfiles/mastodon/public
		rewrite 500.html
		file_server
	}
}
3 Likes

This topic was automatically closed after 30 days. New replies are no longer allowed.