|laziness, impatience, and hubris|
Any spider framework?by Anonymous Monk
|on Jan 06, 2012 at 08:03 UTC||Need Help??|
Anonymous Monk has asked for the
wisdom of the Perl Monks concerning the following question:
I want to get all urls like 'http://site/fixed_string/random_string.html' from one site. Where should I start?
Is there any spider framework, support proxy,cache, and so on, suit my need?
Or, If I start from scratch, using LWP, is there some guide for write a spider?