Caddy V2 + Varnish + Wordpress using Docker with SSL

First of all, I am noob in this stuff so do not be harsh. I would like to share my experience, because there are some pitfals in the process of deploying this stack and I spent a lot of hours tinkering with the caddyfile and docker-compose. I do not want others to tear their hair out when whey will try to accomplish this.

I am building this whole stuff on raspberry pi, so if you plan to do it on a different platform, you will have to use different containers.

Varnish is a caching service that creates static pages out of your dynamic site. The problem is it uses HTTP. So we need to configure it behind Caddy. So that Caddy could manage all the HTTPS stuff.
The scheme for connection would be:

Caddy:443 => Varnish:8090 => Caddy:8091 => Wordpress:9000

I assume you already have Docker and Docker-compose installed.

folder setup:

. Caddyfile
. .env
. docker-compose.yml
. php.ini
./caddy/Dockerfile
./varnish/default.vcl

docker-compose.yml:

version: '3.3'
services:
  # Database
  database:
    image: linuxserver/mariadb
    container_name: database
    volumes:
      - ./db:/config
    restart: always
    env_file: 
      - .env
    environment: 
      - MYSQL_DATABASE=blog_wp
      - MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD
      - MYSQL_USER=$MYSQL_USER
      - MYSQL_PASSWORD=$MYSQL_PASSWORD
    networks:
      - blog-network

  # Wordpress
  wordpress:
    depends_on:
      - database
    image: wordpress:php7.4-fpm-alpine
    container_name: wordpress
    restart: always
    user: "root:root"
    env_file: 
      - .env
    environment:
      - WORDPRESS_DB_HOST=database:3306
      - WORDPRESS_DB_USER=$MYSQL_USER
      - WORDPRESS_DB_PASSWORD=$MYSQL_PASSWORD
      - WORDPRESS_DB_NAME=blog_wp
    volumes:
      - ./php.ini:/usr/local/etc/php/conf.d/custom.ini
      - ./wordpress:/var/www/html
    networks:
      - blog-network

  # Webserver
  caddy:
    container_name: webserver
    build:
      context: caddy
    restart: always
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./wordpress:/var/www/html
      - ./caddy_data:/data
      - ./Caddyfile:/etc/caddy/Caddyfile
    networks:
      - blog-network

  varnish:
    container_name: varnish
    image: viktorli/varnish_for_rpi:latest
    volumes:
      - ./varnish/default.vcl:/etc/varnish/default.vcl
    ports:
      - 8090:8090
    networks:
      - blog-network

networks:
  blog-network:
    driver: bridge

Caddyfile:

{
	email you@mail.com
}

example.com {
	reverse_proxy varnish:8090  #IP of Docker host with Varnish LISTENING port
}


:8091 {
        @cachedFiles {
            path *.jpg *.jpeg *.png *.gif *.ico *.js *.css *.woff *.woff2 *.ttf
        }
        header @cachedFiles Cache-Control "public, max-age=604800, must-revalidate"
        encode gzip
        php_fastcgi wordpress:9000
        file_server        
}

default.vcl

#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and https://www.varnish-cache.org/trac/wiki/VCLExamples for more examples.

# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;

# Default backend definition. Set this to point to your content server.
backend default {
    .host = "caddy";
    .port = "8091";
}

sub vcl_recv {
    # Happens before we check if we have this in cache already.
    #
    # Typically you clean up the request here, removing cookies you don't need,
    # rewriting the request, etc.
if (req.restarts == 0) {
if (req.http.X-Forwarded-For) {
  set req.http.X-Forwarded-For = client.ip;
  }
}
if (req.http.Authorization || req.method == "POST") {
  return (pass);
}
if (req.url ~ "/feed") {
  return (pass);
}
set req.http.cookie = regsuball(req.http.cookie, "wp-settings-\d+=[^;]+(; )?", "");
set req.http.cookie = regsuball(req.http.cookie, "wp-settings-time-\d+=[^;]+(; )?", "");
if (req.http.cookie == "") {
  unset req.http.cookie;
  }
}

sub vcl_backend_response {
    # Happens after we have read the response headers from the backend.
    #
    # Here you clean the response headers, removing silly Set-Cookie headers
    # and other mistakes your backend does.
}

sub vcl_deliver {
    # Happens when we have all the pieces we need, and are about to send the
    # response to the client.
    #
    # You can do accounting or modifying the final object here.
}

ABOUT THE SETUP

As suggested by:

I did reconfigured docker-compose.yml and Caddyfile accordingly.
Now it is working.

Have you considered using this plugin instead?

You could drop Varnish altogether and just use one instance of Caddy. Seems like a mess to have 3 servers running instead of just one.

That said, I think a confusion you’re having with Docker has to do with how to refer to one service from another.

Docker ships with an internal DNS server that is configured to resolve service names to the container IP addresses. So instead of 192.168.200.184:9000, you should use wordpress:9000 and instead of localhost:80 you should use caddy:80, and so on.

I think that’s why bridge mode wasn’t working for you. Because localhost means “this container”.

Thank you for the input.

Does it generate static files from php?

Docker ships with an internal DNS server that is configured to resolve service names to the container IP addresses. So instead of 192.168.200.184:9000, you should use wordpress:9000 and instead of localhost:80 you should use caddy:80, and so on.

I think that’s why bridge mode wasn’t working for you. Because localhost means “this container”.

Yes. But if you set network_mode: host in the docker-container then this DNS naming does not work. And Varnish itself does not seem to connect the correct way if it is not on the host machines network together with caddy. I do not know where is the problem.

No, cache-handler caches the responses in memory. It uses olric under the hood.

Also just quoting the bit I edited in after in case you missed it (cause I was still editing when you replied):

Varnish does exactly that. It generates static files from wordpress. Which is great I think.

But you wouldn’t need host mode if you use the right names to connect.

Have you tried this from Varnish?

backend default {
    .host = "caddy";
    .port = "80";
}

Yes. I did. And it works if I connect to the varnish listening port :8090
But when I further configure reverse_proxy in Caddyfile to manage ssl it does not work. I receive too many redirects message. I can not understand why.

But when I do the same when Caddy and Varnish on one localhost network - it works.

Try using curl -vL <url> to make the request, see what it looks like. It’ll show you what redirects are being attempted.

Thank you! I did not know that. I will try to debug.

I did the cahnges you usggested and edited the initial post. With this settings I can access website using both HTTP and HTTPS, so I tried redirect directive, but if I apply it I get redirection error.
My Caddyfile:

https://viktorli.hopto.org {
	reverse_proxy varnish:8090 
}

http://viktorli.hopto.org  {
  redir https://viktorli.hopto.org
}

:80 {
        root * /var/www/html
        @cachedFiles {
            path *.jpg *.jpeg *.png *.gif *.ico *.js *.css *.woff *.woff2 *.ttf
        }
        header @cachedFiles Cache-Control "public, max-age=604800, must-revalidate"
        encode gzip
        php_fastcgi wordpress:9000
        file_server       
}

curl -vL --max-redirs 5 https://viktorli.hopto.org gives me:

* Connected to viktorli.hopto.org (46.109.12.149) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_CHACHA20_POLY1305_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=viktorli.hopto.org
*  start date: Dec  7 19:51:03 2020 GMT
*  expire date: Mar  7 19:51:03 2021 GMT
*  subjectAltName: host "viktorli.hopto.org" matched cert's "viktorli.hopto.org"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x1911880)
> GET / HTTP/2
> Host: viktorli.hopto.org
> User-Agent: curl/7.64.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 302
< age: 0
< date: Wed, 09 Dec 2020 12:31:33 GMT
< location: https://viktorli.hopto.org
< server: Caddy
< server: Caddy
< via: 1.1 varnish (Varnish/5.0)
< x-varnish: 2
< content-length: 0
<
* Connection #0 to host viktorli.hopto.org left intact
* Issue another request to this URL: 'https://viktorli.hopto.org/'
* Found bundle for host viktorli.hopto.org: 0x1915250 [can multiplex]
* Re-using existing connection! (#0) with host viktorli.hopto.org
* Connected to viktorli.hopto.org (46.109.12.149) port 443 (#0)
* Using Stream ID: 3 (easy handle 0x1911880)
> GET / HTTP/2
> Host: viktorli.hopto.org
> User-Agent: curl/7.64.0
> Accept: */*
>
< HTTP/2 302
< age: 0
< date: Wed, 09 Dec 2020 12:31:33 GMT
< location: https://viktorli.hopto.org
< server: Caddy
< server: Caddy
< via: 1.1 varnish (Varnish/5.0)
< x-varnish: 4
< content-length: 0
<
* Connection #0 to host viktorli.hopto.org left intact
* Issue another request to this URL: 'https://viktorli.hopto.org/'
* Found bundle for host viktorli.hopto.org: 0x1915250 [can multiplex]
* Re-using existing connection! (#0) with host viktorli.hopto.org
* Connected to viktorli.hopto.org (46.109.12.149) port 443 (#0)
* Using Stream ID: 5 (easy handle 0x1911880)
> GET / HTTP/2
> Host: viktorli.hopto.org
> User-Agent: curl/7.64.0
> Accept: */*

If I comment out the redirect block:

#http://viktorli.hopto.org  {
# redir https://viktorli.hopto.org
#}

Everything works well.
If I use curl -vL --max-redirs 5 https://viktorli.hopto.org it gives me:

* Connected to viktorli.hopto.org (46.109.12.149) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_CHACHA20_POLY1305_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=viktorli.hopto.org
*  start date: Dec  7 19:51:03 2020 GMT
*  expire date: Mar  7 19:51:03 2021 GMT
*  subjectAltName: host "viktorli.hopto.org" matched cert's "viktorli.hopto.org"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x693880)
> GET / HTTP/2
> Host: viktorli.hopto.org
> User-Agent: curl/7.64.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200
< accept-ranges: bytes
< age: 0
< content-type: text/html; charset=UTF-8
< date: Wed, 09 Dec 2020 12:36:26 GMT
< link: <https://viktorli.hopto.org/wp-json/>; rel="https://api.w.org/"
< server: Caddy
< server: Caddy
< vary: Accept-Encoding
< via: 1.1 varnish (Varnish/5.0)
< x-powered-by: PHP/7.4.13
< x-varnish: 2

Why the redirect is not working?

I also tried changing Caddyfile to:

http://viktorli.hopto.org  {
  redir https://{host}{uri} permanent
}

https://viktorli.hopto.org {
	reverse_proxy varnish:8090  
}

:80 {
        root * /var/www/html
        @cachedFiles {
            path *.jpg *.jpeg *.png *.gif *.ico *.js *.css *.woff *.woff2 *.ttf
        }
        header @cachedFiles Cache-Control "public, max-age=604800, must-revalidate"
        encode gzip
        php_fastcgi wordpress:9000
        file_server       
}

But it does not work aswell.
Also tried just the domain name without http or https in front:

viktorli.hopto.org {
	reverse_proxy varnish:8090  
}

:80 {
        root * /var/www/html
        @cachedFiles {
            path *.jpg *.jpeg *.png *.gif *.ico *.js *.css *.woff *.woff2 *.ttf
        }
        header @cachedFiles Cache-Control "public, max-age=604800, must-revalidate"
        encode gzip
        php_fastcgi wordpress:9000
        file_server       
}

Did not help either

UPD
Got the mistake.
I also have :80 directive which is causing the problem.
Edited the initial post

Well the problem is you’re having Varnish make the request on port 80, but you also turn on a redirect on port 80 (http:// is port 80). You can’t do that, that’s in conflict.

Instead, I recommend serving your wordpress backend on a different port like 8080 or something, and have Varnish hit that. That way, you won’t need to worry about the redirect being overriden because it wouldn’t be.

Just change :80 in your Caddyfile to :8080, then in your Varnish config change .port = "80"; to .port = "8080"; and you should be good to go.

Port 8080 would only be accessible to docker services only, not to the outside world. That way you’re ensuring that outside services can’t just skip past HTTPS by making HTTP requests.

1 Like

This topic was automatically closed after 30 days. New replies are no longer allowed.