Beefy Boxes and Bandwidth Generously Provided by pair Networks vroom
P is for Practical
 
PerlMonks  

To Automate through a Dynamic Page like that of Twitter, Using www:mechanize:firefox

by Matt™ (Acolyte)
on Sep 21, 2012 at 13:14 UTC ( #994898=perlquestion: print w/ replies, xml ) Need Help??
Matt™ has asked for the wisdom of the Perl Monks concerning the following question:

To Automate through a dynamic page like that of twitter Using www:mechanize:firefox .

Guten Tag Monks, I need your help to automate through a Dynamic Page like that of twitter. In twitter the initial loading of tweets are about 20 Nos. We need to scroll to the bottom in-order to load another 20 more. So my humble question is, how can i do it via www:mechanize:firefox. It is basically a GET that is happening there but I can't use www:mechanize for this. I am pretty new to www:mechanize:firefox, so pls be kind to answer my question. Thanks in advance.

Bitte Hilfe!, Monks.

Danke.

Comment on To Automate through a Dynamic Page like that of Twitter, Using www:mechanize:firefox
Re: To Automate through a Dynamic Page like that of Twitter, Using www:mechanize:firefox
by Corion (Pope) on Sep 21, 2012 at 13:38 UTC

    There are just two steps to do that:

    1. Find out how a human navigates the page
    2. Replicate those steps using WWW::Mechanize::Firefox

    You haven't told us where you have problems, so I suggest you start at step one.

[reply]

      Thanks Corion. We need to scroll down the page of twitter in-order to get the next set of 20 tweets, Can we replicate the Scroll down process in WWW::Mechanize::Firefox or, is there any alternate way? Thanks in advance.

[reply]

        There is no direct way with WWW::Mechanize::Firefox. But you can run arbitrary Javascript through it, so you can use that to scroll the page down.

        Personally, I would look at what function gets called to load more information.

        Also, you are certainly aware of the Twitter API and/or Net::Twitter?

[reply]

Back to Seekers of Perl Wisdom

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://994898]
Approved by herveus
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (5)
As of 2013-05-19 18:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best material for plates (tableware) is:









    Results (397 votes), past polls