Help with proxying to php in a docker container

fyrfli · December 15, 2022, 4:07am

1. Output of `caddy version`:

v2.6.2 h1:wKoFIxpmOJLGl3QXoo6PNbYvGW4xLEgo32GPBEjWL8o=

2. How I run Caddy:

a. System environment:

Debian Bullsys
systemd

b. Command:

sudo systemctl start caddy

c. Service/unit/compose file:

[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target

[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

d. My complete Caddy config:

{
	email my.email
	log {
		output file /var/log/caddy/caddy.log
		level info
	}
}

...

https://fyrfli.org {
	log {
		output file /var/log/fyrfli-org.log
	}
	root * /srv/public
	encode gzip
	php_fastcgi localhost:9000 {
		root /srv/public
	}
	file_server
}

...

Every other host in the Caddyfile works just fine so snipped out those lines for clarity (It’s a long file)

3. The problem I’m having:

When I load up https://fyrfli.org, I get a blank page and the following in the caddy.log file (prettified for readability):

{
    "level": "error",
    "ts": 1671076650.9727628,
    "logger": "http.log.error.log7",
    "msg": "dialing backend: dial tcp 127.0.0.1:9000: connect: connection refused",
    "request": {
        "remote_ip": "69.10.106.221",
        "remote_port": "51881",
        "proto": "HTTP/2.0",
        "method": "GET",
        "host": "fyrfli.org",
        "uri": "/",
        "headers": {
            "Accept-Encoding": [
                "gzip, deflate, br"
            ],
            "Dnt": [
                "1"
            ],
            "Sec-Fetch-User": [
                "?1"
            ],
            "Sec-Gpc": [
                "1"
            ],
            "Te": [
                "trailers"
            ],
            "Accept": [
                "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"
            ],
            "Accept-Language": [
                "en,en-US;q=0.5"
            ],
            "Sec-Fetch-Dest": [
                "document"
            ],
            "Sec-Fetch-Mode": [
                "navigate"
            ],
            "Sec-Fetch-Site": [
                "none"
            ],
            "User-Agent": [
                "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:107.0) Gecko/20100101 Firefox/107.0"
            ],
            "Upgrade-Insecure-Requests": [
                "1"
            ]
        },
        "tls": {
            "resumed": false,
            "version": 772,
            "cipher_suite": 4865,
            "proto": "h2",
            "server_name": "fyrfli.org"
        }
    },
    "duration": 0.000822463,
    "status": 502,
    "err_id": "givurjirb",
    "err_trace": "reverseproxy.statusError (reverseproxy.go:1272)"
}

4. Error messages and/or full log output:

As stated above, error in the caddy log is:

{
    "level": "error",
    "ts": 1671076650.9727628,
    "logger": "http.log.error.log7",
    "msg": "dialing backend: dial tcp 127.0.0.1:9000: connect: connection refused",
    "request": {
        "remote_ip": "my-ip-address",
        "remote_port": "51881",
        "proto": "HTTP/2.0",
        "method": "GET",
        "host": "fyrfli.org",
        "uri": "/",
        "headers": {
            "Accept-Encoding": [
                "gzip, deflate, br"
            ],
            "Dnt": [
                "1"
            ],
            "Sec-Fetch-User": [
                "?1"
            ],
            "Sec-Gpc": [
                "1"
            ],
            "Te": [
                "trailers"
            ],
            "Accept": [
                "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"
            ],
            "Accept-Language": [
                "en,en-US;q=0.5"
            ],
            "Sec-Fetch-Dest": [
                "document"
            ],
            "Sec-Fetch-Mode": [
                "navigate"
            ],
            "Sec-Fetch-Site": [
                "none"
            ],
            "User-Agent": [
                "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:107.0) Gecko/20100101 Firefox/107.0"
            ],
            "Upgrade-Insecure-Requests": [
                "1"
            ]
        },
        "tls": {
            "resumed": false,
            "version": 772,
            "cipher_suite": 4865,
            "proto": "h2",
            "server_name": "fyrfli.org"
        }
    },
    "duration": 0.000822463,
    "status": 502,
    "err_id": "givurjirb",
    "err_trace": "reverseproxy.statusError (reverseproxy.go:1272)"
}

5. What I already tried:

I’ve tried using the reverse_proxy directive to the :80 port in the docker container.
I’ve tried adding try_files {path} {path}/index.php index.php to the Caddyfile section
I’ve tried adding the container name instead of localhost or 127.0.0.1 or nothing (i.e. php_fastcgi :9000

I either get the same blank page or a “File not found”. However, I can browse the files in the public directory (I was able to load up https://fyrfli.org/robots.txt and see the contents - which is a whole other issue I am going to need to solve later)

6. Links to relevant resources:

I am at my wits end. This is easy with nginx and php-fpm on the host, i.e.:

location ~ \.php$ {
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    if (!-f $document_root$fastcgi_script_name) {
        return 404;
    }

    fastcgi_pass unix:/run/php-fpm/php-fpm.sock;
    fastcgi_read_timeout 30m;
    fastcgi_param PATH_INFO $fastcgi_path_info;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_index index.php;
    include /etc/nginx/fastcgi_params;
}

But I cannot seem to translate this to work with caddy → php-fpm in a docker container.

Help?!

Whitestrake · December 15, 2022, 5:55am

Howdy @fyrfli, welcome to the Caddy community.

How do you run your PHP container?

The configuration of your Docker containers is 100% critical to determining how to access, and reverse proxy to, a containerized PHP-FPM.

This would work equally well for Caddy if you still had PHP on the host, too, with just:
php_fastcgi unix//run/php-fpm/php-fpm.sock

fyrfli · December 15, 2022, 5:59am

My bad. It’s an invoice ninja container. Here is the docker run command I use:

docker run -d \
  --name invoiceninja \
  --restart=unless-stopped \
  -v /srv/public:/var/www/app/public \
  -v /srv/storage:/var/www/app/storage \
  -e APP_ENV='production' \
  -e APP_DEBUG=0 \
  -e APP_URL='http://fyrfli.org' \
  -e APP_KEY='app_key' \
  -e APP_CIPHER='AES-256-CBC' \
  -e DB_TYPE='mysql' \
  -e DB_STRICT='false' \
  -e DB_HOST='172.17.0.1' \
  -e DB_DATABASE='db' \
  -e DB_USERNAME='dbuser' \
  -e DB_PASSWORD='dbpassword' \
  -p '8010:80' \
  invoiceninja/invoiceninja

Whitestrake · December 15, 2022, 6:14am

That container does not expose port 9000 on the host.

No web server running on the host will be able to proxy PHP to it on port 9000 whatsoever.

The presence of an exposed port 80 here implies that there is a web server running inside this container.

The Docker hub page for this container also references Caddy, as well as the fact that it’s based on PHP-FPM:

This image is based on php:7.2-fpm official version.

Features

Automatic HTTPS ( Caddy) Fully production-ready through docker-compose Adjustable to your needs via environment variable

—https://hub.docker.com/r/invoiceninja/invoiceninja

But I can’t find a Dockerfile, nor can I find anything explicit explaining the makeup of their app container. I went digging through their Github, but it was not very elucidating. At the moment, I’m still not sure what’s going on.

They seem to be shipping default web server configuration files in a ‘Dockerfiles’ repo (which has no Dockerfiles in it as far as I can see?): GitHub - invoiceninja/dockerfiles: Docker files for Invoice Ninja

In the repo there is a Caddyfile which uses app:9000 as its target:

github.com

invoiceninja/dockerfiles/blob/4aa7a7032c890a835ffe6ddf56e83031f8b30ecd/config/caddy/Caddyfile#L3


      
          {$APP_URL} {
              root * /var/www/app/public
              php_fastcgi app:9000
              encode zstd gzip
              file_server browse
          }

And there is a docker-compose.yml which uses invoiceninja/invoiceninja:5 as an app service:

github.com

invoiceninja/dockerfiles/blob/4aa7a7032c890a835ffe6ddf56e83031f8b30ecd/docker-compose.yml#L25-L38


      
          app:
            image: invoiceninja/invoiceninja:5
            env_file: env
            restart: always
            volumes:
              - ./config/hosts:/etc/hosts:ro
              - ./docker/app/public:/var/www/app/public:rw,delegated
              - ./docker/app/storage:/var/www/app/storage:rw,delegated
            depends_on:
              - db
            networks:
              - invoiceninja
            extra_hosts:
              - "in5.localhost:192.168.0.124 " #host and ip

Although it doesn’t expose port 9000, the implication here is that invoiceninja/invoiceninja is a standard, port-9000-available PHP-FPM container, and you should consider exposing port 9000 to the host machine in order to proxy to it. Alternatively, put Caddy and this app together in the same Compose file and have Caddy refer to it over the Docker-Compose-created network.

I know none of this for sure, because the docs are not entirely clear, and I’m unable to test things myself at the moment.

fyrfli · December 15, 2022, 6:14am

I actually tried that since I do have php fpm on the host. I don’t remember what that error was but I do remember it didn’t work … which maybe means my container is … iffy? Lemme check that method again … stand by …

Here is the changed Caddyfile:

https://fyrfli.org {
	log {
		output file /var/log/fyrfli-org.log
	}
	root * /srv/public
	encode gzip
	php_fastcgi unix//run/php/php8.1-fpm.sock {
		root /srv/public
	}
	file_server
}

and this produces no discernable error in either the docker container logs or caddy’s log and a blank screen.

fyrfli · December 15, 2022, 6:19am

So it’s not just me. Thanks for confirming that I am not completely dense.

Yes I found that while they do mention Caddy, there is no example Caddyfile anywhere. I guess I could try the docker-compose and see if that makes a difference. I actually do expose :9000 (copied the wrong docker run command):

docker run -d \
  --name invoiceninja \
  --restart=unless-stopped \
  -v /srv/public:/var/www/app/public \
  -v /srv/storage:/var/www/app/storage \
  -e APP_ENV='production' \
  -e APP_DEBUG=0 \
  -e APP_URL='http://fyrfli.org' \
  -e APP_KEY='app_key' \
  -e APP_CIPHER='AES-256-CBC' \
  -e DB_TYPE='mysql' \
  -e DB_STRICT='false' \
  -e DB_HOST='172.17.0.1' \
  -e DB_DATABASE='db' \
  -e DB_USERNAME='dbuser' \
  -e DB_PASSWORD='dbpassword' \
  -p '8010:80' \
  -p '8011:9000' \
  invoiceninja/invoiceninja

… and no dice.

Which makes me think that this should be a post on their forum not “ours”

BTW: I absolutely am in love with Caddy.

Whitestrake · December 15, 2022, 6:21am

Caddy on the host machine is trying to proxy to the host machine’s FPM socket, not the FPM socket of the container you’re running.

I know this because your docker command does not have a volume (-v) flag relating to the socket, so the container cannot be bind mounting the socket.

Ahh!

So you’re actually exposing the container’s PHP-FPM listener on the host’s port 8011 (Docker published ports come in the format of hostport:containerport).

That means you can’t php_fastcgi localhost:9000, you need to php_fastcgi localhost:8011 from your Caddy server running on the host machine.

We stan a based webserver.

fyrfli · December 15, 2022, 6:26am

Yes. I apologize for the confusion. I tried exposing 9000:9000 and proxying to that and I tried exposing 8011:9000 and proxying to that … same thing. Blank page or “File not found” and I know the index.php is in the same directory being accessed because I can browse to the robots.txt file in that directory.

And honestly, now that we are talking about it … I am beginning to believe the problem is with the container … not Caddy. So let me experiment some more with their installation documentation. Thanks for all your help and again sorry for the confusion.

fyrfli · December 15, 2022, 6:29am

Oh … in my search, I stumbled on this post but the Caddyfile is v1 and I am being lazy learning how to convert v1 to v2.

Whitestrake · December 15, 2022, 6:38am

I have to question, here… are you sure that’s the case?

In both the container and the host?

I note that you’ve translated the volume mount in your Docker run command:

So, while Caddy on your web server can see all the files in your web root… When it tells PHP-FPM to read the file in /srv/public/index.php, that file inside the container isn’t actually there; it’s in /var/www/app/public/index.php!

I think you might need to configure Caddy to tell your PHP-FPM container about this discrepancy.

Where you have root specified as a subdirective to php_fastcgi, instead, specify root /var/www/app/public (leaving the main Caddyfile web root as /srv/public).

fyrfli · December 15, 2022, 6:47am

Ooh … lemme try this… brb

ETA:

Progress… now I get a page but a 500 error!

ETAA:

Got it working! THANKS!!! (the 500 error was because my app key wasn’t correct, in case you were wondering…)

Caddy and the Caddy community rock socks!

Whitestrake · December 15, 2022, 7:14am

Nice! Glad to hear it!

system · January 14, 2023, 4:08am

This topic was automatically closed after 30 days. New replies are no longer allowed.