1. The problem I’m having:
I am unable to use an NFS share as the root for sites. I can use the exact same configuration if I move the files over to the local machine. This is on a local dev environment, so not web accessible.
[sergio@samara:/var/www]$ curl -vL https://nerdpress.localhost
* Host nerdpress.localhost:443 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
* Trying [::1]:443...
* Connected to nerdpress.localhost (::1) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS alert, unknown CA (560):
* OpenSSL/3.0.13: error:16000069:STORE routines::unregistered scheme
* Closing connection
curl: (35) OpenSSL/3.0.13: error:16000069:STORE routines::unregistered scheme
I suspected it was something to do with file permissions, since I also couldn’t get this to work from my ~/www directory. I’ve since mounted the NFS share with the GID of the local caddy user (239) and chowned the /var/www/nerdpress directory to sergio:239. This changed the error message from a 403 to a 500… Getting a stale NFS file handle from the logs (shown on step 2 below) which I tried to address, but does not seem to be the problem? I reset the server/guest and unmounted/mounted the NFS share with -l option…
This system has been installed through a Nix file on NixOS. Specifics in the NixOS forum post I created:
Setting up a WordPress dev environment. AKA... Trying to run before I can walk - #21 by Seedsca - Help - NixOS Discourse
Output of file permissions showing caddy group:
[sergio@samara:/var/www]$ ls -l nerdpress/
total 9520
-rw-rw-r-- 1 sergio caddy 2574 Sep 1 2021 db_check
-rw-rw-r-- 1 sergio caddy 900 Jun 25 23:51 devenv.nix
-rw-r--r-- 1 sergio caddy 0 Jul 3 13:49 favicon.ico
-rw-r--r-- 1 sergio caddy 534 Jun 21 14:01 flake.lock
-rw-r--r-- 1 sergio caddy 3652 Jun 22 15:20 flake.nix
-rw-r--r-- 1 sergio caddy 2832 Jun 21 13:44 flake.nix.pause
-rw-rw-r-- 1 sergio caddy 405 Jun 4 2020 index.php
-rw-rw-r-- 1 sergio caddy 1545 Sep 5 2021 integrity.php
-rw-rw-r-- 1 sergio caddy 19915 Sep 28 2023 license.txt
-rw-rw-r-- 1 sergio caddy 201 Dec 23 2021 nano.save
-rw-rw-r-- 1 sergio caddy 2376922 Dec 15 2021 nerdpress-2021-12-15-efd1283.sql
-rw-rw-r-- 1 sergio caddy 3407305 Sep 28 2023 nerdpress-2023-09-28-b1c9be3.sql
-rw-rw-r-- 1 sergio caddy 3702713 Mar 11 09:33 nerdpress-2024-03-11-a9e003e.sql
-rw-rw-r-- 1 sergio caddy 73 Apr 28 2023 php.ini
-rw-rw-r-- 1 sergio caddy 7399 Sep 28 2023 readme.html
-rw-r--r-- 1 sergio caddy 150 Jun 20 14:37 shell.nix
-rw-rw-r-- 1 sergio caddy 207 Jun 30 14:22 test.php
-rw-r--r-- 1 sergio caddy 0 Jun 14 23:26 wordpress.db
-rw-rw-r-- 1 sergio caddy 7211 Sep 28 2023 wp-activate.php
drwxr-xr-x 1 sergio caddy 2804 Sep 28 2023 wp-admin
-rw-rw-r-- 1 sergio caddy 351 Jun 4 2020 wp-blog-header.php
-rw-rw-r-- 1 sergio caddy 2323 Sep 28 2023 wp-comments-post.php
-rw-rw-r-- 1 sergio caddy 3535 Apr 28 2023 wp-config.php
-rw-rw-r-- 1 sergio caddy 3013 Sep 28 2023 wp-config-sample.php
drwxr-xr-x 1 sergio caddy 286 Jun 21 14:36 wp-content
-rw-rw-r-- 1 sergio caddy 5638 Sep 28 2023 wp-cron.php
drwxr-xr-x 1 sergio caddy 9744 Sep 28 2023 wp-includes
-rw-rw-r-- 1 sergio caddy 2502 Sep 28 2023 wp-links-opml.php
-rw-rw-r-- 1 sergio caddy 3927 Sep 28 2023 wp-load.php
-rw-rw-r-- 1 sergio caddy 49441 Sep 28 2023 wp-login.php
-rw-rw-r-- 1 sergio caddy 8537 Sep 28 2023 wp-mail.php
-rw-rw-r-- 1 sergio caddy 25602 Sep 28 2023 wp-settings.php
-rw-rw-r-- 1 sergio caddy 34385 Sep 28 2023 wp-signup.php
-rw-rw-r-- 1 sergio caddy 4885 Sep 28 2023 wp-trackback.php
-rw-rw-r-- 1 sergio caddy 3236 Sep 28 2023 xmlrpc.php
NFS exportfs -v output from server:
/mnt/btr-vault/vault/www 192.168.0.0/16(sync,wdelay,hide,no_subtree_check,anonuid=1000,anongid=239,sec=sys,rw,secure,root_squash,all_squash)
mount output from local:
192.168.1.10:/mnt/btr-vault/vault/www on /var/www type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.1.128,local_lock=none,addr=192.168.1.10,x-systemd.automount,x-systemd.after=network-online.target,x-systemd.mount-timeout=90)
I think the server can read all the directories fine:
<?php
$dirs = [
"/var",
"/var/www",
"/var/www/nerdpress",
];
foreach ( $dirs as $dir ) {
if ( is_array( scandir($dir ) ) ){
echo "<p> $dir is readable</p>";
}
}
phpinfo();
2. Error messages and/or full log output:
[sergio@samara:~/nixos-config]$ sudo journalctl -f -u caddy.service -u mysql.service -u phpfpm-nerdpress.localhost.service
Jul 03 16:53:13 samara caddy[1087]: {"level":"error","ts":1720050793.9880104,"logger":"http.log.error.log0","msg":"stat /var/www/nerdpress: stale NFS file handle","request":{"remote_ip":"127.0.0.1","remote_port":"42830","client_ip":"127.0.0.1","proto":"HTTP/2.0","method":"GET","host":"nerdpress.localhost","uri":"/","headers":{"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-User":["?1"],"Pragma":["no-cache"],"Accept-Encoding":["gzip, deflate, br, zstd"],"Sec-Fetch-Mode":["navigate"],"Dnt":["1"],"Priority":["u=1"],"Cache-Control":["no-cache"],"Te":["trailers"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"],"Sec-Fetch-Site":["none"],"Sec-Gpc":["1"],"Cookie":[],"Accept-Language":["en-US,en;q=0.5"],"Sec-Fetch-Dest":["document"],"User-Agent":["Mozilla/5.0 (X11; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"nerdpress.localhost"}},"duration":0.067520311,"status":500,"err_id":"ybc9vpats","err_trace":"fileserver.(*FileServer).ServeHTTP (staticfiles.go:284)"}
3. Caddy version:
2.7.6
● caddy.service - Caddy
Loaded: loaded (/etc/systemd/system/caddy.service; enabled; preset: enabled)
Drop-In: /nix/store/k6m7mfl2aqs9qlx7xx7sziqclkn4c6f2-system-units/caddy.service.d
└─overrides.conf
Active: active (running) since Wed 2024-07-03 16:26:24 PDT; 1h 50min ago
Docs: https://caddyserver.com/docs/
Main PID: 1087 (caddy)
IP: 13.7K in, 12.1K out
IO: 35.2M read, 52.0K written
Tasks: 14 (limit: 38313)
Memory: 45.6M (peak: 48.2M)
CPU: 718ms
CGroup: /system.slice/caddy.service
└─1087 /nix/store/c496kw0yni8wjg8636dwvnjb4hcn21c7-caddy-2.7.6/bin/caddy run --config /etc/caddy/caddy_config --adapter caddyfile
4. How I installed and ran Caddy:
a. System environment:
NixOS 24.05, x86, systemd
b. Command:
Systemd service is used to run Caddy. The command to run Caddy is:
/nix/store/c496kw0yni8wjg8636dwvnjb4hcn21c7-caddy-2.7.6/bin/caddy run --config /etc/caddy/caddy_config --adapter caddyfile
c. Service/unit/compose file:
[sergio@samara:/var/www]$ systemctl cat caddy.service
# /etc/systemd/system/caddy.service
# caddy.service
#
# For using Caddy with a config file.
#
# Make sure the ExecStart and ExecReload commands are correct
# for your installation.
#
# See https://caddyserver.com/docs/install for instructions.
#
# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.
[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target
[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/nix/store/c496kw0yni8wjg8636dwvnjb4hcn21c7-caddy-2.7.6/bin/caddy run --environ --config /etc/caddy/Caddyfi>
ExecReload=/nix/store/c496kw0yni8wjg8636dwvnjb4hcn21c7-caddy-2.7.6/bin/caddy reload --config /etc/caddy/Caddyfile --f>
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE
[Install]
WantedBy=multi-user.target
# /nix/store/k6m7mfl2aqs9qlx7xx7sziqclkn4c6f2-system-units/caddy.service.d/overrides.conf
[Unit]
StartLimitBurst=10
StartLimitIntervalSec=14400
X-Reload-Triggers=/nix/store/18l4b9c4fx49xvqivgv8wl2r00dmjkm4-X-Reload-Triggers-caddy
[Service]
Environment="LOCALE_ARCHIVE=/nix/store/naa6s54b4hc55740lblbs9phv7xk317h-glibc-locales-2.39-52/lib/locale/locale-archi>
Environment="PATH=/nix/store/php4qidg2bxzmm79vpri025bqi0fa889-coreutils-9.5/bin:/nix/store/jjcsr5gs4qanf7ln5c6wgcq4sn>
Environment="TZDIR=/nix/store/y6hmqbmbwq0rmx1fzix5c5jszla2pzmp-tzdata-2024a/share/zoneinfo"
ExecReload=
ExecReload=/nix/store/c496kw0yni8wjg8636dwvnjb4hcn21c7-caddy-2.7.6/bin/caddy reload --config /etc/caddy/caddy_config >
ExecStart=
ExecStart=/nix/store/c496kw0yni8wjg8636dwvnjb4hcn21c7-caddy-2.7.6/bin/caddy run --config /etc/caddy/caddy_config --ad>
Group=caddy
LogsDirectory=caddy
NoNewPrivileges=true
PrivateDevices=false
PrivateTmp=false
ProtectHome=false
ProtectSystem=off
ReadWritePaths=/var/lib/caddy
Restart=on-failure
RestartPreventExitStatus=1
RestartSec=5s
StateDirectory=caddy
User=caddy
[Install]
WantedBy=multi-user.target
d. My complete Caddy config:
[sergio@samara:/var/www]$ cat /etc/caddy/caddy_config
{
log {
level ERROR
}
}
https://nerdpress.localhost:443 {
log {
output file /var/log/caddy/access-https://nerdpress.localhost:443.log
}
root * /var/www/nerdpress
php_fastcgi unix//run/phpfpm/nerdpress.localhost.sock
file_server
}
Created by this lamp.nix file:
[sergio@samara:~/nixos-config]$ cat modules/lamp.nix
{ config, pkgs, lib, ... }:
let
php' = pkgs.php83.buildEnv {
extensions = ({ enabled, all }: enabled ++ (with all; [
xdebug
imagick
]));
# any customizations to your `php.ini` go here
extraConfig = ''
memory_limit = 1024M
xdebug.mode = debug
xdebug.start_with_request = yes
xdebug.idekey = gdbp
'';
};
webPath = "/var/www";
sites = [
"nerdpress.localhost"
];
in
{
networking.hosts = {
# convenient if you're going to work on multiple sites
"127.0.0.1" = [
"nerdpress.localhost"
];
};
services.mysql.enable = true;
services.mysql.package = pkgs.mariadb;
services.mysql.ensureDatabases = [
# list a database for every site you want and they will be automatically created
"nerdpress"
];
services.mysql.ensureUsers = [
# NOTE: it is important that `name` matches your `$USER` name, this allows us to avoid password authentication
{ name = "sergio";
ensurePermissions = {
"*.*" = "ALL PRIVILEGES";
};
}
];
services.phpfpm.pools."nerdpress.localhost" = {
user = "sergio";
group = "users";
phpPackage = php';
settings = {
"listen.owner" = config.services.caddy.user;
"listen.group" = config.services.caddy.group;
"pm" = "dynamic";
"pm.max_children" = 5;
"pm.start_servers" = 2;
"pm.min_spare_servers" = 1;
"pm.max_spare_servers" = 5;
"php_admin_value[error_log]" = "stderr";
"php_admin_flag[log_errors]" = true;
"catch_workers_output" = true;
};
};
services.caddy.enable = true;
services.caddy.virtualHosts."https://nerdpress.localhost:443".extraConfig = ''
root * ${webPath}/nerdpress
php_fastcgi unix/${config.services.phpfpm.pools."nerdpress.localhost".socket}
file_server
'';
# automatically create a directory for each site you will work on with appropriate ownership+permissions
systemd.tmpfiles.rules = [
"d ${webPath}/nerdpress 0755 sergio users"
];
systemd.services."phpfpm-nerdpress.localhost".serviceConfig = {
PrivateDevices = lib.mkForce false;
PrivateTmp = lib.mkForce false;
ProtectSystem = lib.mkForce "off";
ProtectHome = lib.mkForce false;
};
systemd.services.mysql.serviceConfig = {
PrivateDevices = lib.mkForce false;
PrivateTmp = lib.mkForce false;
ProtectSystem = lib.mkForce "off";
ProtectHome = lib.mkForce false;
};
systemd.services.caddy.serviceConfig = {
PrivateDevices = lib.mkForce false;
PrivateTmp = lib.mkForce false;
ProtectSystem = lib.mkForce "off";
ProtectHome = lib.mkForce false;
};
}
5. Links to relevant resources:
Where I started to create this setup and some relevant information:
