Rewrite to Robots.txt

[john-tjust-stratsys]

in nginx this is possible:

location /robots.txt {return 200 “User-agent: *\nDisallow: /\n”;}

In my proxies I would like to map requests for robots.txt to either the solution above or a robots.txt file somwhere on my caddy server, using the same file for all proxies.

Thanks, John

[abiosoft (Abiola Ibrahim)]

Try this. Note that you need to build from source or wait for 0.9.2 to use status.

header /robots.txt {
  User-agent *
  Disallow /
}

status 200 /robots.txt

[john-tjust-stratsys]

Hello,

Thank you very much for your answer!

Kind Regards, John

[john-tjust-stratsys]

I did solve it with writing the X-Robots-Tag for all requests instead.

But it would be nice to be able to write a response body with the status command, is this something you plan to do? If not, do think it’s a bad idea?

Kind Regards