Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re^4: Any spider framework?

by jdrago999 (Pilgrim)
on Jan 08, 2012 at 06:40 UTC ( #946828=note: print w/replies, xml ) Need Help??


in reply to Re^3: Any spider framework?
in thread Any spider framework?

OK!

As promised, the patches/updates/POD have been applied, github now hosts the code and I've put the newest release on github at https://github.com/jdrago999/WWW-Crawler-Lite

Thanks everyone for your suggestions and time...

Now you can get the HTML::LinkExtor version of link-parsing by specifying 'link_parser => "HTML::LinkExtor"' in the constructor. Otherwise you get the 'default' (original, regexp-based) way.

Maybe this could be changed...actually...to use something slick like Web::Query to get at that information (which, for me, was the whole point).

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://946828]
help
Chatterbox?
[Lady_Aleena]: Well, not much of my code. A few little things people here like. The big things though, the more important to me code, people here seem to loathe.
[atcroft]: Lady_Aleena: My early code was pretty horrible... but somewhere early on I learned not to take criticism of my code personally... which was probably a good thing, considering the kind of code I wrote at the time. :\
[Lady_Aleena]: I am hungry, so I'm going to go make something to eat. I hope you figure it out atcroft. Have a nice night!
[atcroft]: I have one script that I have started re-writing several times over the years, and have yet to get it working in a cleaner/better form. (It is some smelly code-I did not even know about hashes at the time I wrote it.)
[atcroft]: Enjoy, Lady_Aleena!

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (6)
As of 2017-05-27 05:27 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?