Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Re^4: Any spider framework?

by jdrago999 (Pilgrim)
on Jan 08, 2012 at 06:40 UTC ( #946828=note: print w/ replies, xml ) Need Help??


in reply to Re^3: Any spider framework?
in thread Any spider framework?

OK!

As promised, the patches/updates/POD have been applied, github now hosts the code and I've put the newest release on github at https://github.com/jdrago999/WWW-Crawler-Lite

Thanks everyone for your suggestions and time...

Now you can get the HTML::LinkExtor version of link-parsing by specifying 'link_parser => "HTML::LinkExtor"' in the constructor. Otherwise you get the 'default' (original, regexp-based) way.

Maybe this could be changed...actually...to use something slick like Web::Query to get at that information (which, for me, was the whole point).


Comment on Re^4: Any spider framework?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://946828]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (5)
As of 2014-09-19 21:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (147 votes), past polls