Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

How to get WWW::Mechaize::Firefox to get text?

by zerocred (Beadle)
on Apr 18, 2014 at 18:15 UTC ( [id://1082798]=perlquestion: print w/replies, xml ) Need Help??

zerocred has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I'm trying to automate logging in to a complex javascript webpage.

After inputting username/password the javascript asks an extra question from a list of 6 known questions & answers before actually logging in. The question is in a iframe popup.

The question appears like this - simple text: <span id="LoginQuestion">What is your pets name?</span>

I have an xpath&css locators too

I can't figure out how to lift simple text with WWW-Mechanize-Firefox given the xpath that points at it.

  • How to grab get_text with Mechanize_Firefox?

  • How to select a particular iframe in Mechanize/Firefox (like in Selenium), is it even necessary in Mech FF?

  • Should I look at another tool?

  • I tried $mech->content (with a regex), but it returns some other html not what I see with Firefox/Firebug...I think it comes from wrong iframe.

    I did it before using Selenium which has $sel->get_text(xpath) sort of thing

    Thanks in advance

    Replies are listed 'Best First'.
    Re: How to get WWW::Mechaize::Firefox to get text?
    by Corion (Patriarch) on Apr 19, 2014 at 10:48 UTC

      You will need to look inside of the iframe.

      Most likely, if you already have the appropriate DOM element via ->xpath, the ->{innerHTML} property can help you.

      If you are already using ->xpath, maybe you want to use xpath( "/path/to/element/text()" )?

      For selecting frames, have you looked at what WWW::Mechanize::Firefox has to say about frames?

        Thanks Corion!

        So here it is:

        #!/usr/bin/perl -w use strict; use Data::Dumper; use WWW::Mechanize::Firefox; my $mech = WWW::Mechanize::Firefox->new( launch => '/usr/bin/firefox', tab=>'current', frames => 1, ); $mech->get('http://www.example.com'); # wait for JS to render the login button my $retries = 10; #by xpath while ($retries-- and ! $mech->is_visible( {xpath => '//*[@id="login"] +' } )) { sleep 1; }; die "Timeout while waiting for site to appear" if 0 > $retries; print "waitiing retries = $retries\n"; #type the uid & pw $mech->field( username => 'username'); #by name $mech->field( password => 'password'); #by name #sync cos hangs here otherwise $mech->click( {id=>'login', synchronize => 0}); #by id here my $string; my @elements = $mech->xpath('xpath/selector/to/secret/question'); #by +xpath here for(my $x=0;$x<=$#elements;$x++){ $string = $elements[$x]->{innerHTML}; #get the string print "secret question is elements[$x] = $string\n"; #in case ther +e are multiple matches } #ok you need some regex here to decide which question is being asked a +nd reply with corresponding answer $mech->field( SecQuestion => 'secret answer'); #by name here $mech->click( {id=>"confirm", synchronize => 0});

        There seemed to be issues about the match of the secret question array having multiple responses.

        You asked for many elements but seem to only want a single item. Did you forget to pass the 'single' option with a true value? Pass 'all => 1' to suppress this message and receive the count of item +s.
        I think it was trouble shooting lines of code I had added that put me in the wrong iframe... or something.

        Thanks for the hints, they really helped!

        z

        Thanks for the hints. I was already using

        frames=>1

        It is a iframe within a iframe... and Firebug xpath seems only references within the current iframe.

        It's a unique id ... I haven't cracked it yet. I shall report again later!

    Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Domain Nodelet?
    Node Status?
    node history
    Node Type: perlquestion [id://1082798]
    Approved by Athanasius
    help
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this?Last hourOther CB clients
    Other Users?
    Others browsing the Monastery: (2)
    As of 2024-04-20 03:14 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      No recent polls found