Using Caddy for incident management in a home network

Wiki
1

Scope

This article serves as an adjunct to the wiki article Using Caddy as a reverse proxy in a home network. It provides some ideas on how Caddy can be used to help you manage customer perceptions when a service you’re offering goes offline unexpectedly or is taken down for scheduled maintenance.

Assumptions

  1. You use a Caddyfile, though I’m sure this can be adapted to JSON as well.
  2. You use a supported CDN, such as Cloudflare, with your Caddy reverse proxy.
  3. You have some way to inform website visitors of outages and scheduled maintenance. You might want to check out Statuspage, which has a free plan.

Topology

*Permission to reuse the diagram below courtesy of @matt.

For the moment, assume that Service A and Service B are not being served by sub-paths. but are being served by subdomains and example.com serves a backend service with a CIDR address 192.168.0.2/24.

The following scenarios are considered below:

  1. Taking the domain offline.
  2. Taking a subdomain offline.
  3. Common infrastructure failure.
  4. Taking a subdirectory offline.
  5. Split behaviour.

Domain Considerations

This snippet comes in handy when dealing with a domain.

(online) {
  @offline expression `"{args.0}" == "no"`
  handle @offline {
    # Do whatever e.g. respond/redir 
  }
}

For example:

example.com {
  import online yes   # Set to no and reload Caddy to take the domain offline
   ...
}

Subdomain Considerations

The map directive works particularly well if you’re using a wildcard certificate for subdomains. In the example below, subdomain1.example.com has been taken offline.

*.example.com {
  map {labels.2} {online} {
    subdomain1  no   # Service A
    subdomain2  yes  # Service B
    ...
  }
  @offline expression `{online} == "no"`
  handle @offline {
    # Do whatever e.g. respond/redir 
  }
  ...
}

Common Infrastructure Considerations

If a common infrastructure element (such as the WAN link, router or Caddy) serving the various backend services fails, all those services are no longer available externally. In this event, you may be able to configure the CDN to redirect customers elsewhere.

If, for example, you’re using the Cloudflare CDN, Cloudflare can be configured to temporarily redirect example.com and its subdomains to another URL. If you set this up beforehand, the redirection can easily be switched on and off as circumstances dictate.

example.com
example.com1118×705 83.3 KB

Subdirectory Considerations

If you use subdirectories, use this approach instead of the one described above for the domain. It’s a little trickier to get your head around, but it does provide you with the flexibility to take a sub-path offline. This example assumes that Service A and Service B are now served by sub-paths.

example.com {
  ...
  map {path} {online} {
    ~^/service-a.*  yes
    ~^/service-b.*  yes
    ~^/.*           yes
  }
  @offline expression `{online} == "no"`
  handle @offline {
    # Do whatever e.g. respond/redir 
  }
  ...
}

For a deeper appreciation of this structure, refer to the forum thread Path handling issue using the map directive

Split behaviour

While it’s useful to switch an offered service from online to offline and then back again, there are times when it may be necessary for the service to remain online internally, but appear offline externally. You might want to do this, for instance, while testing out some changes before making a service generally available.

A third split state can be set up to achieve this, It will require an additional filter to be added to the online snippet and map examples above. You will also need your local network address. For the example above, the network CIDR address would be 192.168.0.0/24.

Using the example network, this filter is added to the online snippet:

  @split {
    expression `"{args.0}" == "split"`
    not remote_ip 192.168.0.0/24
  }
  handle @split {
    # Do whatever e.g. respond/redir
  }

To activate a split state with the snippet, set import online split and reload Caddy.

Using the example network, this filter is added to the map examples:

  @split {
    expression `{online} == "split"`
    not remote_ip 192.168.0.0/24
  }
  handle @split {
    # Do whatever e.g. respond/redir
  }

To set up a split state for a specified subdomain or subdirectory, just set its online state to split within the map table and then reload Caddy.

Note: For the split feature to work, the local DNS server must resolve the domain name to the Caddy IP address. For further detail, refer to Tri-state switch partially working.

References

  1. Using Caddy as a reverse proxy in a home network
  2. Serving tens of thousands of domains over HTTPS with Caddy
  3. Best practice approach when a service goes offline?
  4. Migrate to using a wildcard certificate
  5. Variable assignment?
  6. map (Caddyfile directive) — Caddy Documentation
  7. Path handling issue using the map directive
  8. Tri-state switch partially working
  9. Load balancing queries
4 Likes
2

For a whole bunch of reasons (see Load balancing queries), it’s not a good idea to tightly couple the map and reverse_proxy directives together. This wiki article has been updated to reflect that.

1 Like