Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re^2: On improving Perlmonks site availability

by LanX (Saint)
on Mar 30, 2026 at 23:00 UTC ( [id://11167489]=note: print w/replies, xml ) Need Help??


in reply to Re: On improving Perlmonks site availability
in thread On improving Perlmonks site availability

If there was any "intelligence" behind the scraping, it would have long realized that we have only dozens of posts with new content per week.

The AI business is hungry and greedily tries to swallow as much content as possible and many actors are throwing billions into training more bots.

(Update: Moved the rest to Redirecting anonymous requests to offline copy on other servers)

Cheers Rolf
(addicted to the Perl Programming Language :)
see Wikisyntax for the Monastery

  • Comment on Re^2: On improving Perlmonks site availability

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11167489]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (2)
As of 2026-04-16 01:27 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?
    hippoepoptai's answer Re: how do I set a cookie and redirect was blessed by hippo!
    erzuuliAnonymous Monks are no longer allowed to use Super Search, due to an excessive use of this resource by robots.