1. The problem I’m having:
In Caddy2, I want to use rewrite to intercept commonly used spider crawlers. If it is a crawler accessing the path of the product page, then rewrite it. But in Caddy2, I don’t know how to match spider crawlers, when I write the header like this_ Regexp agent1 User Agent baiduspider|360Spider|Sogou Spider, caddy service cannot start. Can you help me write a regular expression to intercept spider crawlers.
root * C:\root
@product {
header_regexp agent1 User-Agent baiduspider|360Spider|Sogou Spider
path_regexp product ^/detail/detail(\d+).html$
}
rewrite @product /product/{http.regexp.product.1}.html
file_server
}
2. Error messages and/or full log output:
PASTE OVER THIS, BETWEEN THE ``` LINES.
Please use the preview pane to ensure it looks nice.
Caddy service cannot start, when modified to
header_regexp agent1 User-Agent baiduspider ，starts normally。
3. Caddy version:
v2
4. How I installed and ran Caddy:
a. System environment:
b. Command:
PASTE OVER THIS, BETWEEN THE ``` LINES.
Please use the preview pane to ensure it looks nice.
c. Service/unit/compose file:
PASTE OVER THIS, BETWEEN THE ``` LINES.
Please use the preview pane to ensure it looks nice.
d. My complete Caddy config:
PASTE OVER THIS, BETWEEN THE ``` LINES.
Please use the preview pane to ensure it looks nice.
