Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Re^4: Any spider framework?

by jdrago999 (Pilgrim)
on Jan 08, 2012 at 04:54 UTC ( #946825=note: print w/replies, xml ) Need Help??


in reply to Re^3: Any spider framework?
in thread Any spider framework?

As the author of WWW::Crawler::Lite, I am also appalled at the use of that regexp for URL detection! (What was I thinking?)

I am quite pressed for time at the moment, but I will put the module on github and re-release it with the patches/updates suggested on RT already.

FWIW I use this module in several places (and have for some time now). While there are perhaps some more "robust" spiders/crawlers out there, I wasn't able to find one as simple to use and understand as W:C:L.

Once the github + pause uploads are completed, I'll re-post here.

Thanks!

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://946825]
help
Chatterbox?
[1nickt]: Discipulus, Corion, try this: D.A. Golden's blog
[Lady_Aleena]: Your Mother, that's the command line grep, not perl's grep.
[Your Mother]: …how on Earth could Perl’s grep recurse?
[Tanktalus]: perldigious: I'm not looking to get rich. I'm looking for interesting work, with good continuity of employment, and maintenance of our current lifestyle. (An improvement wouldn't be bad, either, but that's not a requirement ;) )
erix enjoys the confusion++
[Your Mother]: :P
[Tanktalus]: Without confusion, I'd be out of a job ;)
[Tanktalus]: Already. :D
[Lady_Aleena]: Your Mother, I wish it did have a way to put in a directory to search.
[Your Mother]: That’s silly. It’s a string op tool.

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (11)
As of 2017-05-23 19:33 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?