Re^6: Unable to connect

[jdporter]

At about 1210 UTC today (corresponding to 08:10 server time), I modified Super Search to disallow Anonymous Monk usage. This should be considered experimental and temporary, to see if/how it affects site responsiveness.

[karlgoethebier]

Please see also Reduce the Google crawl rate and File a special request to report a problem with unusually high crawl rate.

[jdporter]

Interesting, but I don't see any evidence that google is the origin of the large number of hits.

[hippo]

You wrote:

I just did some log file messin', and found that there have been a huge number of hits on the site from the address range 66.249.64. to 66.249.79.

And these are listed in the Googlebot ranges. Therefore, the evidence does suggest that the "huge number of hits" are down to the googlebot crawler.

[karlgoethebier]

So now it is proven. Or we regard it as proven. Or the majority consider it proven - it doesn't matter. According to pure doctrine returning anything other than 200 should work - even 418. That's about what Google suggests at the given location. And writing them a nice little email would also have a certain entertainment value. Maybe then this nightmare with the non-existent performance will soon come to an end.

[choroba]

Just a few minutes ago, I got "Unable to connect" when trying to refresh the RAT.