Hello,
Just started using Caddy. Got a webserver going that display a static html splash page. I have added the nobots plugin, created and added a bomb.gz, then used this example from the documentation:
nobots "bomb.gz" { "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" "Googlebot/2.1 (+http://www.google.com/bot.html)" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" "Googlebot-News" "Googlebot-Image/1.0" "Googlebot-Video/1.0" "compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html" "Mediapartners-Google" "AdsBot-Google (+http://www.google.com/adsbot.html)" "AdsBot-Google-Mobile-Apps" "APIs-Google (+https://developers.google.com/webmasters/APIs-Google.html)" }
Caddy server is running and displaying the page.
How can I test if nobots is working as intended now?
Also, with the snippet above, am I correct to understand it will stop every indexing bots pretty much?
Thanks