Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re: Automating website navigation

by ckj (Chaplain)
on May 06, 2013 at 10:32 UTC ( #1032235=note: print w/ replies, xml ) Need Help??


in reply to Automating website navigation

For all such work there are lots of packages like:

1. WWW::Mechanize : This is the best module for this kind of task. You can use it for almost all kind of websites except extensive dynamic websites where pages are created with request at run time, while if you can find the input parameters then you can use this module and believe me I've used WWW::Mecahnize for all my tasks till now.

2. LWP: This is the parent classs of WWW::Mechanize and only difference is that here you have to create your own modules for some of the operations whicha are already created in WWW::Mechanize, but since this is the parent so you can use it in any way.

3. WWW::Mechanize::Firefox: Best for dynamic websites means where javascript is widely used to generate pages, where automation is very complex with simple WWW::Mechanize.

4. WWW::Scripter: another WWW::Mechanize-workalike with Javascript support.

Please go through cpan and you might found more URLS for your task, but if you'll ask anyone then these four are the best packages referred for this kind of work. I hope this one will help you or let me know if you've any more queries.


Comment on Re: Automating website navigation
Re^2: Automating website navigation
by samir_gambler (Novice) on May 06, 2013 at 10:41 UTC
    thanks for the reply....even I had thought for the third option WWW::Mechanize::Firefox but it is not available under Perl package manager. Can you please guide me how to install and all the dependent package to be installed as I only know how to install through PPM. Thanks in Advance.
        I don't remember exactly...I was googling and I came to some forum where it was mentioned about this package....
Re^2: Automating website navigation
by samir_gambler (Novice) on May 06, 2013 at 12:53 UTC
    I have installed WWW::Mechanize::Firefox package but while running the sample code. I am getting error "Failed to connect to , problem connecting to "localhost", port 4242: No connecti on could be made because the target machine actively refused it. at C:/Perl/site /lib/MozRepl/Client.pm line 144". Moreover it seems I am unable to start mozrepl in Firefox 20.0.1. Can you please suggest where I am getting it wrong. Thanks
      Finally I got it working. It was the issue with mozrepl in Firefox 20.0.1. I downgraded my firefox to 19 and its working fine... Thanks for your help...

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1032235]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (12)
As of 2014-07-24 20:12 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (167 votes), past polls