Caddyfile assistance with odd directory structure

a-s-o · February 12, 2017, 10:05pm

Hi, I am just looking for help with structuring my Caddyfile. I can’t figure out how to do rewrites internally so they don’t change the end user url. Alternatively, I would like to allow browsing of a directory ourside the root folder.

I have the following directory structure:

www
\ downloads
  - v1.1.zip
  - v1.2.zip
  - etc
\ blog
  - 201612-december-update.md
  - 201611-november-update.md
\ release
  - index.md
  - blog.md
  - about.md
  \ templates
    - default.html
    - blog.html
  \ css
    - main.css
    - app.css
  \ js
    - main.js
    - app.js

The reasoning behind the above directory structure is that the release folder contains all the application-y stuff and the content is outside. The application gets updated regularly whereas the
content will occasionally change.

I would like the following outcome:

myurl.com           ->  index.md (using default.html template)
myurl.com\about     ->  about.md (using default.html template)

myurl.com\downloads ->  browse www\downloads directory

myurl.com\app ->        proxy to a nodejs app

myurl.com\{css,js}  ->  serve the js and css in release folder

myurl.com\blog      ->  blog.md (using default.html template)
myurl.com\blog\201612-december-update 
                    ->  201612-december-update.md (using blog.html template)

As mentioned above I have tried two different approaches, but I am nowhere close
to the above outcome.

1) setting release as the root of the site and rewriting to other urls

*:80 {
  root /var/www/release

  gzip
  tls off
  ext .md

  rewrite /downloads {
    to /../downloads{uri}
  }

  browse /downloads

  rewrite /blog {
    to /../blog/{uri}
  }

  proxy /app localhost:3000

  markdown {
    template templates/default.html
    template blog templates/blog.html
  }
}

2) setting www as the root of the site and rewriting the other urls

*:80 {
  root /var/www

  gzip
  tls off
  ext .md

  browse /downloads

  rewrite / /release
  rewrite /about /release/about
  rewrite /blog /release/blog
  rewrite /css /release/css
  rewrite /js /release/js

  proxy /app localhost:3000

  markdown {
    template release/templates/default.html
    template blog release/templates/blog.html
  }
}

My issues are as follows:

In approach number 1, I cannot browse the downloads and the blog posts do not work. The main /blog page works but none of the posts at /blog/201612-december-update will show up.

In approach number 2, urls do not look as intended; for example the homepage url becomes myurl.com/release (similar to a redirect; I thought rewrite just performed an internal rewrite). The same thing happens to the about and main blog page. Additionally, in exact opposite of approach 1, the main blog does not work but the individual posts work.

I suspect this may be fixed, if I move the static assets inside the release folder, but that is a limitation of the deployment method I am using (anistrano.deploy for ansible).

Can someone please point out if this is possible and if I am doing something totally out of whack? I have tried various mix and matches of the above and still no luck.

Whitestrake · February 13, 2017, 1:24am

What about a multi-vhost (multi-root) setup:

# Handle:
# myurl.com           ->  index.md
# myurl.com\about     ->  about.md
# myurl.com\{css,js}  ->  serve the js and css
myurl.com:80 {
    root /var/www/release
    gzip
    # tls off is unnecessary as specifying port :80 disables Automatic HTTPS
    ext .md
    markdown {
        template templates/default.html
    }
}

# Handle:
# myurl.com\blog      ->  blog.md
# myurl.com\blog\201612-december-update 
#                     ->  201612-december-update.md
myurl.com:80/blog {
    root /var/www/blog
    gzip
    ext .md
    markdown {
        template templates/blog.html
    }
}

# Handle:
# myurl.com\downloads ->  browse www\downloads
myurl.com:80/downloads {
    root /var/www/downloads
    gzip
    browse # Maybe? Not sure how you want to let users browse this
}

# Handle:
# myurl.com\app ->        proxy to a nodejs
myurl.com:80/app {
    proxy / localhost:3000 {
        transparent # Almost always a sane idea when proxying to backends
    }
}

Completely untested, of course. Might run into some weird issues with base paths in URIs and such (not 100% sure), should work well but might need some tweaking.

P.S. Kudos for having a really nice, well written, and very comprehensive first post!

a-s-o · February 13, 2017, 3:32am

Thank you @Whitestrake; that seems like a good direction. My initial results are positive using this method. A lot of the requirements work, except for some base url related issues as you suspected. I am going to continue tweaking and see what I can get it to do and will post back.

I like how each endpoint becomes more configurable so I can try other tricks with it.

Appreciate your response.

Whitestrake · February 13, 2017, 4:44am

No worries, will look forward to seeing what you end up with.

a-s-o · February 19, 2017, 10:27pm

Hi, is there any ways to get more debugging information out of caddy. I am trying to out different configurations in my caddy file and run into a lot of “500 Internal Server Error”. I see no information logged to console and none to logfile supplied with -log flag.

Is there any other diagnostic information? Perhaps a verbose log level?

Currently, I am seeing this error while working with the rewrite and markdown directives.

Thanks

matt · February 19, 2017, 11:22pm

It should almost certainly be logging to the -log if not the errors directive log… are you sure?

a-s-o · February 20, 2017, 12:19am

Great, the errors directive worked; I had not tried that. There was nothing in logfile specified with -log flag, but I presume those are more application level logs. Makes sense.

Thanks for your help and quick response.

Edit: Looks like errors visible is what I needed all along. Sorry, should have read the docs more carefully.

matt · February 20, 2017, 3:43am

Glad you figured it out! Maybe the new website will make things related to logging a little clearer.

a-s-o · February 20, 2017, 6:47pm

Hi gents,

Finally after a lot of trial an error, I got my tests to pass. Here is what I ended up with:

*:80 {
  errors /var/www/errors.log
  root /var/www/current

  ext .md

  browse /downloads

  markdown /blog {
    template blog release/templates/blog.html
    template post release/templates/post.html
  }

  rewrite {
    if {path} not_has /downloads
    if {path} not_has /blog
    to ./release/{path}.html ./release/{path} ./release/{path}/ ./static/{path}/ ./static/{path}
  }
}

Notably missing from above is the proxy to nodejs, which I believe should be simple enough to add by adding a proxy directive and another exclusion to the rewrite block. I also flattened the directory structure a bit but the crucial pain point of the “release” folder being a nested in the root is still intact and solved by rewrite.

Thank you both for your help and also open sourcing and working on the project. Please let me know if you see some obvious improvements to the above. I am still quite new to this but happy with how it all turned out so far.

Whitestrake · February 20, 2017, 11:02pm

Looks good. The only thing I can think of is to be aware that those if statements check anywhere in the path, so a request like /somefolder/blog-post-title.md might not get redirected properly. Doesn’t look like it’ll be an issue with your naming convention.

system · May 21, 2017, 11:14pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.