Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Re^7: How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing? (update)

by Darth Ed (Beadle)
on Aug 24, 2021 at 00:55 UTC ( [id://11136044] : note . print w/replies, xml ) Need Help??


in reply to Re^6: How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing? (update)
in thread How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing?

OK, so I think that's the problem. I added print $_->{message},"\n" for $mech->js_errors(); to my code, and I'm getting a mess of TypeError: undefined is not a function (evaluating ...) JavaScript errors that I don't see in any major web browser.

How very disappointing. I guess I'll have to try to get WWW::Mechanize::Firefox working...

Or maybe I can inject some polyfills? Seems others have encountered this sort of problem with PhantomJS:
https://github.com/tom-james-watson/phantomjs-polyfill
https://github.com/mathiasbynens/String.prototype.startsWith
I might try that first... I'm thinking the best way to do that is to copy ghostdriver/src/main.js and add the polyfills to that?
Well, no such luck with that. Adding the polyfills to the main.js didn't seem to help at all. I give up!
  • Comment on Re^7: How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing? (update)

Replies are listed 'Best First'.
Re^8: How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing?
by LanX (Saint) on Aug 24, 2021 at 10:36 UTC
    Sorry, but I find this whole shooting in the dark approach dubious.

    First you should make clear that this app works in a normal browser on your old system. Then try either a FF v54 (or less) or a supported Chrome.

    If none succeeds your out of luck, either switch your OS or try an alternative like Selenium.

    If you want to participate in a horse race don't start by doping a donkey, just get a horse.

    Anyway your JS bug doesn't sound like something which can be fixed with polyfills.

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery

Re^8: How do I get WWW::Mechanize::PhantomJS to wait until the JavaScript on the web page has finished executing? (update)
by bliako (Monsignor) on Aug 24, 2021 at 10:06 UTC

    perhaps stupid suggestion, but shouldn't the polyfills be injected before any other JS-from-server being run (not after)? Is there a way to first inject the polyfills and then load the page, without "erasing" them? Hmmm tricky. Perhaps fetch the page, even with errors. Then insert the polyfills. Then initiate a re-run which will already have the poly-fills added. The latter perhaps with triggering an event or running a JS function. My first call would be to check if polyfills can be run OK with your current setup. Make a local html test file and load them in WWW::Mechanize::*. If test is successful then you may have a way to solve your problem.

    bw, bliako