Rewrite sub.domain.com/robots.txt to another file

[orangpelupa]

• explain what you are trying to do,
  I have a robots.txt on sub.doman.com/robots.txt that i want to replace with another robots.txt. For example with the ones on C:\caddy\robots.txt or the one in domain.com/robots.txt

both domain.com and sub.comain.com are in the same network, under different VM.
subdomaincom is served thru reverse proxy.

the problem basically is that a nextcloud snap VM blocks all bots thru robots.txt. i want to replace that via rewrite to robots.txt that allows all bot.

• show what you have already tried,
  already tried
  rewrite /robots.txt c:\caddy\robots.tx

and

header /robots.txt {
User-agent : *
Disallow :
}

status 200 /robots.txt

thnkyou
pardon my english

[Whitestrake (Matthew Fay)]

@orangpelupa

rewrite /robots.txt c:\caddy\robots.tx

This will have Caddy act as though the client had requested example.com/c:\caddy\robots.tx. The rewrite directive isn’t used to select a path on disk, it’s used to modify the URI, which Caddy combines with the site root to determine which path to serve.

If you’ve got root /var/www/html and a file at /var/www/html/robots.txt, you won’t need to rewrite, as the path will be correct. What you need to do is tell Caddy not to proxy this request to your Nextcloud snap (assuming you’re proxying to it).

To do that, use the except subdirective, i.e. except /robots.txt, and ensure the web root and robots.txt file line up.

https://caddyserver.com/docs/proxy