I can't publish notifications into Mercure hub (Caddy out of Docker)

1. Caddy version (caddy version):

v2.4.3 h1:Y1FaV2N4WO3rBqxSYA8UZsZTQdN+PwcoOcAiZTM8C0I=

2. How I run Caddy:

From supervisord

a. System environment:

Debian 11

b. Command:

[program:mercure]
process_name=%(program_name)s
environment=JWT_KEY="m3rcu353cr37pa55pra53DEV" \MERCURE_PUBLISHER_JWT_KEY="m3rcu353cr37pa55pra53DEV" \MERCURE_SUBSCRIBER_JWT_KEY="m3rcu353cr37pa55pra53DEV" \CORS_ALLOWED_ORIGINS="https://pami54.local" \PUBLISH_ALLOWED_ORIGINS="https://pami54.local" \SERVER_NAME="pami54.local:3000"
command=/home/frizquierdo/mercureLinux/mercure run --config /home/frizquierdo/mercureLinux/Caddyfile
numprocs=1
autostart=true
autorestart=true
startsecs=5
startretries=10
redirect_stderr=false
stdout_capture_maxbytes=1MB
stderr_capture_maxbytes=1MB
stdout_logfile=/var/log/supervisor/mercureout.log
stderr_logfile=/var/log/supervisor/mercureerror.log

c. Service/unit/compose file:

Paste full file contents here.
Make sure backticks stay on their own lines,
and the post looks nice in the preview pane.

supervisord

d. My complete Caddyfile or JSON config:

# Learn how to configure the Mercure.rocks Hub on https://mercure.rocks/docs/hub/config
{
	http_port 8081
	{$GLOBAL_OPTIONS}
}

pami54.local:3000

log

tls /etc/apache2/ssl-cert/pami54.local.crt /etc/apache2/ssl-cert/pami54.local.key

route {
	encode zstd gzip

	mercure {
		# Transport to use (default to Bolt)
		transport_url {$MERCURE_TRANSPORT_URL:bolt://mercure.db}
		# Publisher JWT key
		publisher_jwt {env.MERCURE_PUBLISHER_JWT_KEY} {env.MERCURE_PUBLISHER_JWT_ALG}
		# Subscriber JWT key
		subscriber_jwt {env.MERCURE_SUBSCRIBER_JWT_KEY} {env.MERCURE_SUBSCRIBER_JWT_ALG}
		# Extra directives
		cors_origins https://pami54.local
		publish_origins https://pami54.local
		{$MERCURE_EXTRA_DIRECTIVES}
	}

	respond /healthz 200

	respond "Not Found" 404
}

3. The problem I’m having:

The clients can connect and reconnect to the Mercure hub, but when eb application try to publish into the hub, the HTTP Client component log errors:
messenger.WARNING: Error thrown while handling message App\Message\NotificacionMessage. Sending for retry #3 using 4000 ms delay. Error: "Handling "App\Message\NotificacionMessage" failed: Failed to send an update." {"message":{"App\\Message\\NotificacionMessage":[]},"class":"App\\Message\\NotificacionMessage","retryCount":3,"delay":4000,"error":"Handling \"App\\Message\\NotificacionMessage\" failed: Failed to send an update.","exception":"[object] (Symfony\\Component\\Messenger\\Exception\\HandlerFailedException(code: 0): Handling \"App\\Message\\NotificacionMessage\" failed: Failed to send an update. at /var/www/html/pami54.local/vendor/symfony/messenger/Middleware/HandleMessageMiddleware.php:129)\n[previous exception] [object] (Symfony\\Component\\Mercure\\Exception\\RuntimeException(code: 0): Failed to send an update. at /var/www/html/pami54.local/vendor/symfony/mercure/src/Hub.php:104)\n[previous exception] [object] (Symfony\\Component\\HttpClient\\Exception\\TransportException(code: 0): The request was not processed and can be safely retried at /var/www/html/pami54.local/vendor/symfony/http-client/Response/CommonResponseTrait.php:148)\n[previous exception] [object] (Symfony\\Component\\HttpClient\\Exception\\TransportException(code: 0): The request was not processed and can be safely retried at /var/www/html/pami54.local/vendor/symfony/http-client/Chunk/ErrorChunk.php:65)\n[previous exception] [object] (Amp\\Http\\Client\\Connection\\UnprocessedRequestException(code: 0): The request was not processed and can be safely retried at /var/www/html/pami54.local/vendor/amphp/http-client/src/Connection/DefaultConnectionFactory.php:117)\n[previous exception] [object] (Amp\\Http\\Client\\SocketException(code: 0): Connection to 'pami54.local:443' failed at /var/www/html/pami54.local/vendor/amphp/http-client/src/Connection/DefaultConnectionFactory.php:118)\n[previous exception] [object] (Amp\\Socket\\ConnectException(code: 111): Connection to tcp://pami54.local:443 refused at /var/www/html/pami54.local/vendor/amphp/socket/src/DnsConnector.php:108)"} []

It’s a virtualhost for the app domain:

<VirtualHost 192.168.56.2:80>
	ServerName pami54.local
	ServerAlias wwww.pami54.local
	
	Redirect permanent / https://pami54.local
</VirtualHost>

<IfModule mod_ssl.c>
 #SSLStaplingCache "shmcb:${SRVROOT}/logs/ssl_stapling(32768)"
 <VirtualHost 192.168.56.2:443>
   ServerName pami54.local
   ServerAlias wwww.pami54.local
  
   DocumentRoot "/var/www/html/pami54.local/public"
   DirectoryIndex index.php
    
   <Directory "/var/www/html/pami54.local/public/"> 
    AllowOverride All
    Order Allow,Deny
    Allow from All
    #Require local
    Require all granted
   
    <IfModule mod_rewrite.c>
            Options -MultiViews
            RewriteEngine On
            RewriteCond %{REQUEST_FILENAME} !-f
            RewriteRule ^(.*)$ index.php [QSA,L]
    </IfModule>
   </Directory>

   SSLEngine on
   SSLProtocol all -SSLv3 -SSLv2
   SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
   SSLCertificateFile "/etc/apache2/ssl-cert/pami54.local.crt"
   SSLCertificateKeyFile "/etc/apache2/ssl-cert/pami54.local.key"
   SSLUseStapling off

   ErrorLog ${APACHE_LOG_DIR}/pami54-vhost-error.log
   CustomLog ${APACHE_LOG_DIR}/pami54-vhost-access.log combined

   <FilesMatch "\.(cgi|shtml|pl|asp|php)$">
    SSLOptions +StdEnvVars
   </FilesMatch>

   BrowserMatch ".*MSIE.*" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0

   SSLProxyEngine On
   ProxyRequests Off
   ProxyPreserveHost On
   ProxyPass '/.well-known/mercure' 'https://pami54.local:3000/.well-known/mercure' connectiontimeout=600 timeout=600
   ProxyPassReverse '/.well-known/mercure' 'https://pami54.local:3000/.well-known/mercure'
 </VirtualHost>
</IfModule>

4. Error messages and/or full log output:

5. What I already tried:

The idea it’s that Mercure run over port 3000 in the same machine of webserver, and in the vitrtualhost definition exist a reverse proxy rule that redirect /.well-known/mercure to https://pami.local:3000/.well-known/mercure.
As I say, the clients can connect to Mercure hub but the web application can publish on the hub.

6. Links to relevant resources:

/cc @dunglas – who might have an answer for you! :slight_smile:

1 Like

Could you create a reproducer (if possible using Docker) in a dedicated Github repository?
The error basically says that pami54.local:443 isn’t reachable from the PHP server.

If it’s not possible, can you try to connect to pami54.local:443 using curl from the box running the PHP server? This may help find the issue.

2 Likes

Hi @dunglas, I founded the error: I had put twice in the /etc/host file an entry for pami54.local, one with the IP 127.0.0.1 and the other with the IP 192.168.56.2; apparently this conflicted with the caddy server and the web server.

I will put a summary of the configuration:

Symfony 5.4 .env.local.php:

'MERCURE_URL' => 'https://pami54.local/.well-known/mercure',
    'MERCURE_PUBLIC_URL' => 'https://pami54.local/.well-known/mercure',
    'MERCURE_JWT_SECRET' => 'm3rcu353cr37pa55pra53DEV',

Symfony 5.4 mercure.yaml:

mercure:
    hubs:
        default:
            url: '%env(MERCURE_URL)%'
            public_url: '%env(MERCURE_PUBLIC_URL)%'
            jwt:
                secret: '%env(MERCURE_JWT_SECRET)%'
                publish: ['notif/unreaded/{user}']
                subscribe: ['notif/unreaded/{user}']

The /etc/host file:

127.0.0.1	localhost
127.0.1.1	debiandev.local	debiandev
192.168.56.2	pami54.local

Apache2 virtualhost file:

<VirtualHost 192.168.56.2:80>
	ServerName pami54.local
	ServerAlias wwww.pami54.local
	
	Redirect / https://pami54.local
</VirtualHost>

<IfModule mod_ssl.c>
 #SSLStaplingCache "shmcb:${SRVROOT}/logs/ssl_stapling(32768)"
 <VirtualHost 192.168.56.2:443>
   ServerName pami54.local
   ServerAlias wwww.pami54.local
  
   DocumentRoot "/var/www/html/pami54.local/public"
   DirectoryIndex index.php
    
   <Directory "/var/www/html/pami54.local/public/"> 
    AllowOverride All
    Order Allow,Deny
    Allow from All
    #Require local
    Require all granted
   
    <IfModule mod_rewrite.c>
            Options -MultiViews
            RewriteEngine On
            RewriteCond %{REQUEST_FILENAME} !-f
            RewriteRule ^(.*)$ index.php [QSA,L]
    </IfModule>
   </Directory>

   SSLEngine on
   SSLProtocol all -SSLv3 -SSLv2
   SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
   SSLCertificateFile "/etc/apache2/ssl-cert/pami54.local.crt"
   SSLCertificateKeyFile "/etc/apache2/ssl-cert/pami54.local.key"
   SSLUseStapling off

   ErrorLog ${APACHE_LOG_DIR}/pami54-vhost-error.log
   CustomLog ${APACHE_LOG_DIR}/pami54-vhost-access.log combined

   <FilesMatch "\.(cgi|shtml|pl|asp|php)$">
    SSLOptions +StdEnvVars
   </FilesMatch>

   BrowserMatch ".*MSIE.*" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0

   SSLProxyEngine On
   ProxyRequests Off
   ProxyPreserveHost On

   ProxyPass "/.well-known/mercure" "https://pami54.local:3000/.well-known/mercure" connectiontimeout=600 timeout=600
   ProxyPassReverse "/.well-known/mercure" "https://pami54.local:3000/.well-known/mercure"
 </VirtualHost>
</IfModule>

Mercure is running over supervisord worker:

[program:mercure]
process_name=%(program_name)s
environment=JWT_KEY="m3rcu353cr37pa55pra53DEV" \MERCURE_PUBLISHER_JWT_KEY="m3rcu353cr37pa55pra53DEV" \MERCURE_SUBSCRIBER_JWT_KEY="m3rcu353cr37pa55pra53DEV" \CORS_ALLOWED_ORIGINS="https://pami54.local" \PUBLISH_ALLOWED_ORIGINS="https://pami54.local" \SERVER_NAME="pami54.local:3000" \ADDR="pami54.local:3000"
command=/home/frizquierdo/mercureLinux/mercure run --config /home/frizquierdo/mercureLinux/Caddyfile
numprocs=1
autostart=true
autorestart=true
startsecs=5
startretries=10
redirect_stderr=false
stdout_capture_maxbytes=1MB
stderr_capture_maxbytes=1MB
stdout_logfile=/var/log/supervisor/mercureout.log
stderr_logfile=/var/log/supervisor/mercureerror.log

Caddyfile of mercure package:

# Learn how to configure the Mercure.rocks Hub on https://mercure.rocks/docs/hub/config
{
	#I set it to a random port for HTTP because otherwise I couldn't run Apache2 on port 80 on the same host. 
    #Also, topics are privates.
	http_port 8081
	https_port 3000
	auto_https disable_redirects
}

pami54.local:3000

log

tls /etc/apache2/ssl-cert/pami54.local.crt /etc/apache2/ssl-cert/pami54.local.key

route {
	encode zstd gzip

	mercure {
		# Transport to use (default to Bolt)
		transport_url {$MERCURE_TRANSPORT_URL:bolt://mercure.db}
		# Publisher JWT key
		publisher_jwt {env.MERCURE_PUBLISHER_JWT_KEY} {env.MERCURE_PUBLISHER_JWT_ALG}
		# Subscriber JWT key
		subscriber_jwt {env.MERCURE_SUBSCRIBER_JWT_KEY} {env.MERCURE_SUBSCRIBER_JWT_ALG}
		# Extra directives
		cors_origins https://pami54.local
		publish_origins https://pami54.local
		{$MERCURE_EXTRA_DIRECTIVES}
	}

	respond /healthz 200

	respond "Not Found" 404
}

**Thanks for your attention!!! Mercure is great within the Symfony ecosystem **

3 Likes