HLS M3U8 Reverse Proxy Caching

(b2c) #1

I’m trying to reverse proxy and cache a HLS/M3U8 live streaming. In HLS live streaming, m3u8 playlist file contains video “chunk” file name with TS extension which is exactly 10 seconds long. So I’m trying to cache those TS files. There is no reason to download same TS files for every new user in my reverse proxy server.

example.com/live/index.m3u8
example.com/live/feed865/1000.ts
example.com/live/feed346/1010.ts
example.com/live/feed424/1020.ts

and my caddyfile

proxy /tv example.com/live {
	without /tv
	header_downstream Access-Control-Allow-Origin "*"
} 
cache {
     match_path /tv/
     match_header Content-Type application/octet-stream   # only for ts file
     status_header X-Cache-Status
     default_max_age 1m  # in live streaming, cached file don't have any value longer than 10 seconds.
     path C:\caddy\tmp\caddy-cache
 }

Is this best way to do that? Thanks in advance.

(Matthew Fay) #2

Looks pretty good to me - well thought out and configured to cache only what you need it to, only for as long as necessary.

(b2c) #3

I’m having two issues using this config. :zipper_mouth_face:

  1. Cached responses are having 3 server name headers! It should be one!

MyServer # I added custom name using header
nginx 1.9.5 # proxy server upstream, I can rename it using header_downstream
Caddy # It’s adding from cache plugin

  1. Server is caching css/js of other folder and creating cached files, though I clearly defined /tv/ path only!
(Matthew Fay) #4

Caddy preserves the headers it receives from upstream.

You can strip the upstream Server header as it comes back with header_downstream -Server.

  • header_upstream sets headers to be passed to the backend. The field name is name and the value is value . This option can be specified multiple times for multiple headers, and dynamic values can also be inserted using request placeholders. By default, existing header fields will be replaced, but you can add/merge field values by prefixing the field name with a plus sign (+). You can remove fields by prefixing the header name with a minus sign (-) and leaving the value blank.
  • header_downstream modifies response headers coming back from the backend. It works the same way header_upstream does.

https://caddyserver.com/docs/proxy

As for the latter, that sounds odd indeed. Unless the CSS and JS is being served with a Content-Type of application/octet-stream - which it really shouldn’t - I’d wager it could be a bug… Worth taking up with the plugin author over at https://github.com/nicolasazrak/caddy-cache.

1 Like
(b2c) #5

Thanks, now it comes down 2 from 3 using your provided solution.

header_downstream -Server

Caddy # Getting this from Cache Plugin
Maddy # Added by myself using header

About caching, I checked content type for a image file which was HIT.

Accept-Ranges: bytes
Cache-Control: max-age=2592000
Content-Length: 43391
Content-Type: image/jpeg
Date: Tue, 28 May 2019 04:31:02 GMT
Etag: “qqn4spr”
Last-Modified: Sun, 19 May 2019 06:23:52 GMT
Server: Caddy
Server: Maddy
X-Cache-Status: hit

I guess I should open a github issue there. :grinning:

(Matthew Fay) #6

I wouldn’t have thought the cache plugin would add Server: Caddy, but then again, the header behaviour is supposed to override by default. So either the plugin or the Caddy project itself might need an issue to track this.

(b2c) #7

Yes that’s right. If I add a cache block in config file, only then Caddy is added as a 2nd server header in cached files. @matt