Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask

Re: (OT) Robots disallow

by kyle (Abbot)
on Apr 01, 2009 at 15:24 UTC ( #754738=note: print w/ replies, xml ) Need Help??

in reply to (OT) Robots disallow

As others have noted, you're already doing the Right Thing to rid yourself of robots that play by the rules. You might be able to discourage badly behaved robots by creating a tar pit for it to wander into. The problems with this are many, and I'm not inclined to discuss them in any detail, but if you're in desperate times and looking for desperate measures, it's an idea worth consideration and probably rejection.

Comment on Re: (OT) Robots disallow

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://754738]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (4)
As of 2015-10-14 04:12 GMT
Find Nodes?
    Voting Booth?

    Does Humor Belong in Programming?

    Results (320 votes), past polls