Caddy + nextcloud (FPM) + collabora

TL;DR: I want to setup the collabora/code container with nextcloud:fpm-alpine to have the office suite and it seems that I have problem for CODE container to respond back to nextcloud through Caddy (WOPI).

I will be descriptive in the post since 10 hours ago I was not understanding a thing about the setup. I am a pure amateur. So I make sure that I make sense for me of, luckily, for others.

Setup

I use podman (5.4,2) with quadlet files to setup my services. One notable thing about this is that Caddy listens to unix-socket for HTTPS thanks to systemd. I write this, but I think it has nothing to do with my problem.

The problem

The current situation is that Nextcloud does not have a webserver since I run the FPM version of it. So the CODE container cannot directly connect to nextcloud container. Then, I am trying to use my caddy container to make the WOPI works. But it does not seem to work at all. Also, I use SSL termination so the nextcloud ↔ CODE does not have to use https.

Detailed setup

Caddyfile

Here the Caddyfile with the relevant sections:

	admin off
	default_bind fd/3 {
		protocols h1 h2
	}
}

cloud.mydomain.com {
	forward_auth authelia:9091 {
		uri /api/authz/forward-auth
		copy_headers Remote-User Remote-Groups Remote-Email Remote-Name
	}
	
	root * /nextcloud-html
	file_server

	php_fastcgi nextcloud:9000 {
		root /var/www/html/
	}

	# Redirects for DAV apps
	# doc: https://docs.nextcloud.com/server/31/admin_manual/configuration_server/reverse_proxy_configuration.html#caddy
	redir /.well-known/carddav /remote.php/dav/ 301
	redir /.well-known/caldav /remote.php/dav/ 301

	@forbidden {
		path    /.htaccess
		path    /data/*
		path    /config/*
		path    /db_structure
		path    /.xml
		path    /README
		path    /3rdparty/*
		path    /lib/*
		path    /templates/*
		path    /occ
		path    /console.php
	}
	respond @forbidden 404
}

office.mydomain.com {
  encode gzip

  reverse_proxy http://collabora:9980  
}

# Add internal HTTP listener for container communication
http://systemd-caddy:8080 {
    # Handle WOPI requests for cloud.mydomain.com
    @wopi {
        host cloud.mydomain.com
        path /index.php/apps/richdocuments/wopi/*
    }
    
    handle @wopi {
        php_fastcgi nextcloud:9000 {
            root /var/www/html/
            env HTTP_HOST cloud.mydomain.com
        }
    }
    respond 403
}
  • The fast CGI works for my Nextcloud service. No problem with this. This is used for external access to my nextcloud.
  • Then I have the normal office subdomain such that the client can connect to the CODE container to edit document.
  • Then I made an http://systemd-caddy:8080 which is supposed to be the route from CODE container to nextcloud. With this part I try to add the “webserver” capability of the Nextcloud container. systemd-caddy is the name of the caddy container in the podman bridge network.

Quadlet files

There are:

  • caddy.container, caddy.socket and main.network
  • nextcloud.container (also on main.network)
  • collabora.container (also on main.network)

Caddy, socket and network

;; caddy.container
[Unit]
AssertPathExists=%h/services/caddy/Caddyfile
AssertPathIsDirectory=%h/services/static

[Container]
Exec=/usr/bin/caddy run --config /etc/caddy/Caddyfile
Image=docker.io/library/caddy
Network=main.network:ip=10.89.0.5
Notify=true ;; for socket activation, I have a caddy.socket 
Volume=nextcloud_html:/nextcloud-html:z
Volume=%h/services/caddy/Caddyfile:/etc/caddy/Caddyfile:Z
Volume=%h/services/static:/static:Z,ro
Volume=caddy_config.volume:/config
Volume=caddy_data.volume:/data

Just the volume needed , and a static IP on the network.

;; main.network
[Network]
Subnet=10.89.0.0/24
Gateway=10.89.0.1

For being exhaustive, here the caddy.socket which wakes up the container:

;; caddy.socket
[Socket]
BindIPv6Only=both

;; fd/3
ListenStream=[::]:443

[Install]
WantedBy=sockets.target

Just listening to the socket on 443.

nextcloud.container

[Unit]
After=redis.service postgres_nextcloud.service
BindsTo=redis.service postgres_nextcloud.service

[Container]
Image=docker.io/library/nextcloud:fpm-alpine
ContainerName=nextcloud

Network=main.network:ip=10.89.0.100
Network=db.network ;; for redis and postgres container

Environment=TRUSTED_PROXIES=10.89.0.5/24 ;; static IP of caddy container
;; Redis and postgres setup removed

Volume=nextcloud_html:/var/www/html:z
Volume=%h/data/downloads:/downloads:Z,ro
Volume=%h/data/media/seminaires:/seminaire:Z,ro

collabora.container

[Container] 
Image=docker.io/collabora/code:25.04.4.1.1
ContainerName=collabora
ExposeHostPort=9980 ;; just documentation, has no effect
Network=main.network
AddHost=cloud.mydomain.com;systemd-caddy:10.89.0.5

Environment=server_name=office.mydomain.com.
Environment=aliasgroup1=https://cloud.mydomain.com:443,http://systemd-caddy:8080
Environment=extra_params="--o:ssl.enable=false --o:ssl.termination=true --o:net.proto=http"

AddCapability=MKNOD

This one is the most interesting. You can see that I set the server_name for external connection. Then I set the ssl.enable=false and ssl.termination=true for SSL termination on CODE’s end. I force the protocol to be http.
Then the aliasgroup1 to allow the WOPI host, nextcloud container, Also, I added the subnetwork alias of the reverse proxy in order to CODE container to connect to it instead of trying to reach the public address.
But, as you will see, it does not work. Maybe I misunderstood something. I tried to enforce this behavior with adding a line to /etc/hosts with the AddHost line which forces my container to resolve them locally, so to use the Caddy configuration. This line should not be necessary since I wrote the alias already.

nextcloud setup (admin settings)

On the nextcloud office setup, I use the container name to connect to collabora collabora:9980 and I allow the whole subnet for the WOPI requests.

The error log

The journalctl for collabora gives me the following errors:

ERR  #31: WOPI::CheckFileInfo failed for URI [https://cloud.mydomain.com/index.php/apps/richdocuments/wopi/files/29604_ocnsrvy1fzd7?access_token=1ddaXKjLaq18i7ElWbMpYaxW0Um3pUHU&access_token_ttl=0]: 0 (Unknown) . Headers:         Body: []| wsd/wopi/CheckFileInfo.cpp:107
ERR  #31: Failed or timed-out CheckFileInfo [https://cloud.mydomain.com/index.php/apps/richdocuments/wopi/files/29604_ocnsrvy1fzd7?access_token=1ddaXKjLaq18i7ElWbMpYaxW0Um3pUHU&access_token_ttl=0]| wsd/wopi/CheckFileInfo.cpp:121
ERR  #31: CheckFileInfo failed for [https%3A%2F%2Fcloud.mydomain.com%3A443%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F29604_ocnsrvy1fzd7], State::Fail| wsd/RequestVettingStation.cpp:341
ERR  #32: WOPI::CheckFileInfo failed for URI [https://cloud.mydomain.com/index.php/apps/richdocuments/wopi/files/29604_ocnsrvy1fzd7?access_token=1ddaXKjLaq18i7ElWbMpYaxW0Um3pUHU&access_token_ttl=0&no_auth_header=&permission=edit]: 0 (Unknown) . Headers:         Body: []| wsd/wopi/CheckFileInfo.cpp:107
ERR  #32: Failed or timed-out CheckFileInfo [https://cloud.mydomain.com/index.php/apps/richdocuments/wopi/files/29604_ocnsrvy1fzd7?access_token=1ddaXKjLaq18i7ElWbMpYaxW0Um3pUHU&access_token_ttl=0&no_auth_header=&permission=edit]| wsd/wopi/CheckFileInfo.cpp:121

It seems that it cannot connect to the caddy container? I can see that the Nextcloud instance asks correctly with a token. But it seems unreachable.

Another problem

So I tried to curl from the collabora container to my caddy container. Notably, I made this query: curl -v -H "Host: cloud.mydomain.com" http://systemd-caddy:8080/index.php/apps/richdocuments/wopi/. Manually I try to emulate the WOPI request that collabora is supposed to produce. But I get

*   Trying 10.89.0.5:8080...
* connect to 10.89.0.5 port 8080 failed: Connection refused
* Failed to connect to systemd-caddy port 8080 after 0 ms: Couldn't connect to server
* Closing connection 0

Which is very weird, I do not understand why I cannot connect to my caddy container. For this, I suspected the added lines to /etc/hosts to be the cause (I am random guessing), but removing the AddHost in the collabora.container does not make any difference.

I checked

  • the variable names are correctly set in the collabora container
  • nextcloud works well
  • the office.mydomain.com/hosting/discovery works well, I have a nice XML for the WOPI protocol.
  • various resources. I found some tutorial in the wiki for collabora with the basic version of nextcloud and another one with end-to-end SSL but they do not fit my situation. I saw a blog on nextcloud but it uses the stack of docker compose and create two instance of caddy including one to “fix” the lack of webserver for the fpm image. I also searched the wiki for related problems, the closest was this one but it’s about problem with a certificate. I read the CODE documentation.

Conclusion

I am out of idea to troubleshot my problem. In the end I have hard time to know where it comes from. I believe that the curl command not working is a very bad sign. But in another hand, I do not understand why collabora still try to get the resource from the external https. I believed that I could route the request through Caddy which seems the simplest solution.

Do you spot any errors or next steps I can do to diagnostics my problems better? I learnt a lot by doing this already, but it seems that I still miss a point for this to be working.

And if read this far, thanks for your time and attention!

It looks like you want the collabora container to connect to caddy on the custom network (main.network), but Caddy is not listening on the custom network.

Socket-activated sockets are created on the host.

Two alternative ideas:

Alternative 1

Modify the file Caddyfile so that Caddy also listens on the custom network

For details, see an example I wrote.

Alternative 2

Connect to caddy on the host’s main network interface instead.

Add

AddHost=systemd-caddy:host-gateway

in the [Container] section in the file collabora.container.

For details, see

example: connect to host’s main network interface using pasta and --add-host=example.com:host-gateway

The network performance (throughput) could be much worse when using Alternative 2 instead of Alternative 1.

1 Like

Thanks for you answer. I have been changing my strategy for this. Also, I just see, you are my hero to understand podman, socket and caddy!

First, I use the Apache image now and not the FPM ones. I wanted to make it a bit simpler and I read online that FPM is great when the setup is distributed with multiple instances, which is not my case at all (and will never be?). I read that using the Apache image for a single instance does not degrade the performance, so I took this image instead.

Then, I read more the NextCloud forum about how to configure the Collabora. It is documented that both service should be able to connect to the https (public) of each server. I think I will follow these recommendations.

Well, your answer is on point. The problem is that caddy is not listening to the custom network. Consequently, one container (nextcloud container) cannot query the collabora container on its public address because hairpin problem, since source IP = destination IP. Also I do not want my container connections to go to the public internet when it can be done in the custom network. Thus, I should focus on using caddy directly and I already looked at both of your solution. I need time to debug/understand more. I believe that the pasta runtime from podman needs additional parameter to access the custom network gateway. More, I think I should change my socket activation configuration. I spend time understanding yours but it was before your change that you did few months ago that enabled container listening to each other passing by caddy. Also, I think I kind of understand the basics but I did not study it so today I understand it badly.

In brief, thank you very much for your contribution to explain podman+socket+caddy and your answer. I need some times to read again your repos from scratch to figure out the issue. I will update my post once it’s done.

I had time this weekend to look at the socket setup and Caddyfile and fill the missing pieces of understanding.

It works by using a special route with caddy passing by the custom network between collabora/nextcloud/caddy. Great. Here is the relevant section.

cloud.mydomain.com {
	forward_auth authelia:9091 {
		uri /api/authz/forward-auth
		copy_headers Remote-User Remote-Groups Remote-Email Remote-Name
	}
	reverse_proxy nextcloud:80
}
cloud.mydomain.com {
	bind 10.89.8.5 {
		protocols h1 h2
	}
	reverse_proxy nextcloud:80
}

office.mydomain.com {
	forward_auth authelia:9091 {
		uri /api/authz/forward-auth
		copy_headers Remote-User Remote-Groups Remote-Email Remote-Name
	}
	encode gzip
  	reverse_proxy http://collabora:9980
}
office.mydomain.com {
	bind 10.89.8.5 {
		protocols h1 h2
	}
	encode gzip
  	reverse_proxy http://collabora:9980
}

where 10.89.8.5 is the static IP of caddy container on the custom network. All good. On the caddy container I wrote: Network=nextcloud.network:ip=10.89.8.5,``alias=office.mydomain.com``,``alias=cloud.mydomain.com.

I have a remaining question. Instead of using a static IP, I wanted to bind to the interface name. So when I add the network option for this interface_name=n0, for instance, and replace the bind directives with bind n0, it does not work. I have the following error message: Error: loading initial config: loading new config: http app module: start: listening on n0:443: listen tcp: lookup n0 on 10.89.8.1:53: no such host from Caddy. Where 10.89.8.1 is the gateway of the custom network. It seems that the container cannot connect to it. I use podman 5.4.2.

I try to understand this and I believe that is because the network runtime pasta limits the custom networks setting. I tried to add Network=pasta:–map-gw line in my Caddy .container file but then in the journal I have caddy[2943403]: Error: can only set extra network names, selected mode pasta conflicts with bridge: invalid argument It seems that I cannot add pasta option when I use multiple networks.

Maybe the fix is not the correct one or maybe there is not fix for this and this is inherent flaw of pasta. But something does not feel right. pasta is the runtime and bridge is the topology of the network. Why both of them seems to conflict? With multiple networks on my caddy container, podman should use pasta as runtime anyway? So why I could not set pasta option for my custom network?

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.