Redir doesn't add a regex group to URL

Help
1

1. Caddy version (caddy version): 2.0.0

2. How I run Caddy: Dockerfile with the Caddyfile added with a docker-compose.yml

FROM caddy:2.0.0-builder AS builder

RUN caddy-builder \
    github.com/hairyhenderson/caddy-teapot-module \
    github.com/caddy-dns/cloudflare

FROM caddy:2.0.0

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

a. System environment:

Linux Ubuntu 20.04.1 LTS, Docker

b. Command: Not really

c. Service/unit/compose file:

  caddy:
    build:
      context: ./caddy
      dockerfile: Dockerfile
    restart: always
    volumes:
      - site:/srv/site
      - ./.caddy_data:/data
      - ./.caddy_config:/config
      - ./caddy/etc/Caddyfile:/etc/caddy/Caddyfile
    environment:
      - ACME_AGREE=true
    ports:
      - "2015:2015"
      - "80:80"
      - "443:443"
    env_file:
      - ./conf/caddy.env
      - ./conf/caddy_sensitive.env

d. My complete Caddyfile or JSON config:

{
    default_sni {$SERVER_NAME}
}

{$SERVER_NAME} {    
    import /etc/{$TLS_MODE} 
    import /etc/basic_auth.conf
    root * /srv/site/pub
    encode zstd gzip

    @blocked {
        path /media/customer/* /media/downloadable/* /media/import/* /media/custom_options/* /errors/*
    }
    respond @blocked 403

    @notfound {
        path_regexp reg_notfound \/\..*$|\/errors\/.*\.xml$|theme_customization\/.*\.xml
    }
    respond @notfound 404

    @static {
        path_regexp reg_static ^/static/(version\d*/)?(.*)$
    }
    rewrite @static /static/{http.regexp.reg_static.2}

    file_server
    php_fastcgi fpm:9000

    log {
        output file /var/log/caddy.log
    }

    header /media X-Frame-Options "SAMEORIGIN"
    header /static X-Frame-Options "SAMEORIGIN"
    header / X-Content-Type-Options "nosniff"

}

3. The problem I’m having:

There is a @static matcher that works perfectly in a production environment. In prod all files are compiled statically and if they are served statically that will be caught in this matcher and the following redirect.

So, I want to run the same server in developer mode, then all files are not compiled and instead these files will be server via a php file instead. When this application is hosted under Apache the .htaccess file for the static dir has following:

<IfModule mod_rewrite.c>
    RewriteEngine On

    # Remove signature of the static files that is used to overcome the browser cache
    RewriteRule ^version.+?/(.+)$ $1 [L]

    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-l

    RewriteRule .* ../static.php?resource=$0 [L]
    # Detects if moxieplayer request with uri params and redirects to uri without params
    <Files moxieplayer.swf>
     	RewriteCond %{QUERY_STRING} !^$
     	RewriteRule ^(.*)$ %{REQUEST_URI}? [R=301,L]
     </Files>
</IfModule>

I’ve been trying to figure out how to make the same logic.

  1. Static file is requested, filter out “version1234443” and grab the last group
  2. If regex group is a path that exists on disk, serve it
  3. Otherwise redirect to static.php with the file as a parameter

I’ve tried quite a few different options similar to:

    @static {
        path_regexp reg_static ^/static/(version\d*/)?(.*)$
        file /static/{re.reg_static.2}
    }
    rewrite @static /static/{re.reg_static.2}

    @dynamic {
        path_regexp reg_dynamic ^/static/(version\d*/)?(.*)$
        not file /static/{re.reg_dynamic.2}
    }
    rewrite @dynamic /static.php?resource={re.reg_dynamic.2}

4. Error messages and/or full log output:

The redirect comes back to the browser but WITHOUT the resource variable added. The redirect ends with ?resource= and the redirect request ends up in a 404.

5. What I already tried:

I’ve tried several different options based on this subject with no luck so far.

6. Links to relevant resources:

2

FROM caddy:2.0.0-builder

I recommend using Caddy v2.2.0 instead, there’s plenty of great fixes since v2.0 that you can make use of.

Please take a look at the docs on Docker Hub again, the instructions for building are very slightly different (use xcaddy now, and each plugin is prefixed with --with)

That said, the trouble with your rewrite is due to the order of operations. Unfortunately, you can’t rely on matchers to run in the order you define them in your Caddyfile. They’ll be run in an arbitrary order. That means you can’t rely on the result of the regexp in other matchers in the same named matcher block.

You can work around this limitation though, by using the handle directive to make sure the matchers are run in sequence (I’m using a feature introduced in v2.1 here, single line named matchers):

@staticPath path_regexp reg_static ^/static/(version\d*/)?(.*)$
handle @staticPath {
	@static file /static/{re.reg_static.2}
	rewrite @static /static/{re.reg_static.2}

	@dynamic not file /static/{re.reg_static.2}
	rewrite @static /static.php?resource={re.reg_dynamic.2}
}

So this way, we’re delegating use of the path_regexp matcher to inside of a new subroute, so we guarantee that the regexp has run before we use it. It also lets you remove a bit of duplication since you don’t need to repeat the regexp.

Turns out, you can actually shorten it even more here by using the try_files directive, which is a shortcut for a file matcher plus a rewrite:

@staticPath path_regexp reg_static ^/static/(version\d*/)?(.*)$
handle @staticPath {
	try_files /static/{re.reg_static.2} /static.php?resource={re.reg_dynamic.2}
}

I’m not 100% certain that this will work (I’m sketchy on whether the query is maintained in the rewrite here) but I hope it does! Note that the handle is still needed here, because the try_files directive is special and can’t take matcher arguments, since it itself encodes a matcher as part of the shortcut.

1 Like
3

Unfortunately I didn’t make it, the redirect is not working for some reason in this setup but it might be something else in the Apache config I’m missing.

4

That’s not enough information for me to help any further. You’ll need to elaborate on what’s not working, what behaviour you’re seeing.

5

No worries, I didn’t expect you to magically find my problems. :slight_smile:
I couldn’t figure out what’s missing and had to give up at the time. This is a Magento 2.x installation based on Caddy instead of Apache or Nginx (fact is, I ran Magento on Nginx before it was supported as well). In Magento there’s a Mode setting that can carry any of the value [default, developer, production]. Default is a hybrid mode and production requires the code to be pre-compiled for performance. In developer mode all the compiling/transpile occurs on the fly to make it easier to do code changes.
The Apache configuration snippet is partly responsible for the developer config and the static directory is supposed to be ignored. But I can’t get it to work and I suspect there’s something else missing.

6

This is the same config but for an Nginx server. If someone sees something obvious I may overcome this and finally make a fully working developer setup. :slight_smile:

location /static/ {
   # Uncomment the following line in production mode
   # expires max;

   # Remove signature of the static files that is used to overcome the browser cache
   location ~ ^/static/version {
       rewrite ^/static/(version\d*/)?(.*)$ /static/$2 last;
   }

   location ~* \.(ico|jpg|jpeg|png|gif|svg|js|css|swf|eot|ttf|otf|woff|woff2|html|json)$ {
       add_header Cache-Control "public";
       add_header X-Frame-Options "SAMEORIGIN";
       expires +1y;

       if (!-f $request_filename) {
           rewrite ^/static/(version\d*/)?(.*)$ /static.php?resource=$2 last;
       }
   }
   location ~* \.(zip|gz|gzip|bz2|csv|xml)$ {
       add_header Cache-Control "no-store";
       add_header X-Frame-Options "SAMEORIGIN";
       expires    off;

       if (!-f $request_filename) {
          rewrite ^/static/(version\d*/)?(.*)$ /static.php?resource=$2 last;
       }
   }
   if (!-f $request_filename) {
       rewrite ^/static/(version\d*/)?(.*)$ /static.php?resource=$2 last;
   }
   add_header X-Frame-Options "SAMEORIGIN";
}
7

You said

Which is too vague. What exactly isn’t working? What behaviour are you seeing? What’s in your logs? What’s your current Caddyfile, after the changes I recommended?

8

This is what I have at the moment.
Like I wrote in the OP, it will end up as a 404.

{
    default_sni {$SERVER_NAME}
}

{$SERVER_NAME} {    
    import /etc/{$TLS_MODE} 
    import /etc/basic_auth.conf
    root * /srv/site/pub
    encode zstd gzip

    @blocked {
        path /media/customer/* /media/downloadable/* /media/import/* /media/custom_options/* /errors/*
    }
    respond @blocked 403

    @notfound {
        path_regexp reg_notfound \/\..*$|\/errors\/.*\.xml$|theme_customization\/.*\.xml
    }
    respond @notfound 404

    @static {
        path_regexp reg_static ^/static/(version\d*/)?(.*)$
    }
    rewrite @static /static/{http.regexp.reg_static.2}

    file_server
    php_fastcgi fpm:9000

    log {
        output file /var/log/caddy.log
    }

    header /media X-Frame-Options "SAMEORIGIN"
    header /static X-Frame-Options "SAMEORIGIN"
    header / X-Content-Type-Options "nosniff"

}
9

I took some time this night and got it to work. I tried the longer suggestion and found a typo, I fixed that and I also had to manually remove some generated files in other directories to get it take effect.
The short version may work too, but that I have to check another night. :slight_smile:

@staticPath path_regexp reg_static ^/static/(version\d*/)?(.*)$
handle @staticPath {
	@static file /static/{re.reg_static.2}
	rewrite @static /static/{re.reg_static.2}

	@dynamic not file /static/{re.reg_static.2}
    # typos: should not be @static nor re.reg_dynamic.2
	rewrite @dynamic /static.php?resource={re.reg_static.2}
}
2 Likes
10

This topic was automatically closed after 30 days. New replies are no longer allowed.

12

Aha. My bad, copy paste troubles :sweat_smile:

13

So, now my Caddy setup for Magento 2 (currently running versions between 2.2 and 2.4.1) supports all running modes. I am really happy with this setup, so easy to get going.

2 Likes