Access log full of bots 404ing on existing index.php. What am I doing wrong?


(Devin Wilson) #1

The access log for my site is full of things like this:

199.59.150.180 - [10/Sep/2017:00:34:31 -0700] "GET /index.php HTTP/1.1" 404 8303 "-" "Twitterbot/1.0"
199.59.150.182 - [10/Sep/2017:00:34:31 -0700] "GET /index.php HTTP/1.1" 404 8303 "-" "Twitterbot/1.0"
216.244.66.238 - [10/Sep/2017:01:08:48 -0700] "GET /index.php HTTP/1.1" 404 8303 "-" "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)"
5.196.87.10 - [10/Sep/2017:01:37:18 -0700] "GET /index.php HTTP/1.1" 404 2526 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"
51.255.71.119 - [10/Sep/2017:01:37:19 -0700] "GET /index.php HTTP/1.1" 404 2526 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"
217.182.132.60 - [10/Sep/2017:02:55:58 -0700] "GET /index.php HTTP/1.1" 404 2526 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"
216.244.66.238 - [10/Sep/2017:02:58:37 -0700] "GET /index.php HTTP/1.1" 404 8303 "-" "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)"
46.229.164.100 - [10/Sep/2017:03:00:42 -0700] "GET /index.php HTTP/1.1" 404 2526 "-" "Mozilla/5.0 (compatible; SemrushBot/1.2~bl; +http://www.semrush.com/bot.html)"

Problem is, index.php exists, and loading it in a browser loads my site just fine.

I’m thinking it’s a problem with the rewrite rules to make my CMS (Kirby) work:

rewrite {
    if_op or
    if {file} ends_with .md
    if {file} ends_with .mdown
    if {uri} starts_with /site/
    if {uri} starts_with /kirby/
    to /index.php
}

rewrite /panel {
    to {path} {path}/ /panel/index.php
}

rewrite {
    to {path} {path}/ /index.php
}

Anyone have any clues?


(Devin Wilson) #2

A search of the caddy process log (which I didn’t previously have enabled) shows that some Chinese biomedical corporation has pointed their domain at my IP. Hmm.


(system) #3

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.