Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re^2: How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing?

by Darth Ed (Beadle)
on Aug 21, 2021 at 07:13 UTC ( [id://11135999] : note . print w/replies, xml ) Need Help??


in reply to Re: How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing?
in thread How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing?

So you have to tell Mechanize to wait for an DOM-element to appear which represent the state you consider "finished".

Sure, I get that, but I haven't been able to figure out how to do that with WWW::Mechanize::PhantomJS. I don't see an is_visible method listed in the POD.... I'd be fine with just setting a timer for 3 seconds, but, again, I don't see how to do that either.

Replies are listed 'Best First'.
Re^3: How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing?
by Corion (Patriarch) on Aug 21, 2021 at 12:04 UTC
      Note that WWW::Mechanize::PhantomJS is not under development anymore. The program phantomjs also is not developped anymore.
      It's not? I was not aware of that. I don't see anything in the WWW::Mechanize::PhantomJS POD about that, and the most recent release was less than a year ago....

      You're certainly correct about the phantomjs program. I now see that notice at the top of https://phantomjs.org/.

      Unfortunately, I need this to work on a fairly old version of Linux (CentOS 6), so I don't think WWW::Mechanize::Chrome is an option. Maybe WWW::Mechanize::Firefox will work though....

Re^3: How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing?
by LanX (Saint) on Aug 21, 2021 at 11:11 UTC
    Using one of the selectors in WWW::Mechanize::PhantomJS to find a "finished" characteristic (DOM element or attribute or combination or ...) should return an empty result unless it exists. (See all EXTRACTION-METHODS)

    And even if this didn't work, you could ->eval your own JS-code which does those checks in a loop for you and only returns after success.

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery

Re^3: How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing?
by Corion (Patriarch) on Aug 21, 2021 at 07:39 UTC

    Maybe just use sleep to wait for three seconds?

      Maybe just use sleep to wait for three seconds?
      Sleep between the call to the get method and the call to the decoded_content method? I tried that. No change to the content.

      Is there a different arrangement of methods you recommend? Do you have an example that does something like this?

        > Sleep between the call to the get method and the call to the decoded_content method? I tried that. No change to the content.

        I can't find any decoded_content method documented in WWW::Mechanize::PhantomJS (?)

        Maybe try $mech->content( %options ) ?

        Otherwise are you sure your JS-App really works inside PhantomJS? It wouldn't be the first case of browser incompatibilities.

        update

        The HTTP::Response object returned by WWW::Mechanize has a decoded_content method, I don't know how and if subclassing was done here, but I wouldn't expect that to work with JS.

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        Wikisyntax for the Monastery