Rewrite to Robots.txt

in nginx this is possible:

location /robots.txt {return 200 “User-agent: *\nDisallow: /\n”;}

In my proxies I would like to map requests for robots.txt to either the solution above or a robots.txt file somwhere on my caddy server, using the same file for all proxies.

Thanks, John

Try this. Note that you need to build from source or wait for 0.9.2 to use status.

header /robots.txt {
  User-agent *
  Disallow /

status 200 /robots.txt
1 Like


Thank you very much for your answer! :slight_smile:

Kind Regards, John

I did solve it with writing the X-Robots-Tag for all requests instead.

But it would be nice to be able to write a response body with the status command, is this something you plan to do? If not, do think it’s a bad idea?

Kind Regards

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.