Re: Waiting for page to load with WWW::Mechanize::Chrome

[Corion]

The kinds of events that Chrome makes available to the outside world are different from the events that Firefox makes available.

If you need to wait 3 seconds or more, then most likely as LanX already said, some Javscript only runs after the page is considered complete (by the browser) and then loads additional information. LanX's approach of waiting for a page element is the approach I think is most fruitful. Unfortunately, WWW::Mechanize::Chrome doesn't have the ->wait_until_* methods of WWW::Mechanize::Firefox yet, but maybe you can submit a patch for that?

[nysus]

This isn't a patch but one possible solution. I wrote a wrapper for WWW::Mechanize::Chrome in Moose. Note that that get method is overridden. Every 1/10th of a second it checks to see if new elements have been loaded on the page. If there were, it waits another 1/10th of a second. The length of time can be increased for slower connections.

package WWW::Mechanize::Chrome::MooseWrapper ;

use Moose;
use MooseX::NonMoose;
use Time::HiRes qw(usleep);
use File::Temp 'tempdir';
use Log::Log4perl qw(:easy);
use Data::Dumper qw(Dumper);
use namespace::autoclean;
extends 'WWW::Mechanize::Chrome';

sub FOREIGNBUILDARGS {
  my $class = shift;
  my %args = ( @_ == 1 ? %{ $_[0] } : @_);

  # set default launch args
  $args{launch_exe}     = $args{launch_exe}       || "/Applications/Go
+ogle Chrome.app/Contents/MacOS/Google Chrome";
  $args{launch_arg}     = $args{launch_arg}       || [ '--disable-noti
+fications' ];
  $args{data_directory} = $args{data_directory}   || tempdir();
  $args{incognito}      = exists $args{incognito}  ? $args{incognito} 
+: 1;
  return %args;
}

### Public methods ###

sub scroll_to_bottom {
  my $s = shift;
  $s->eval( 'window.scroll(0,document.body.scrollHeight)' );
}

sub scroll_to_top {
  my $s = shift;
  $s->eval( 'window.scroll(0,0)' );
}

sub get_element_count {
  my $s = shift;
  my ($el_count) = $s->eval( 'document.getElementsByTagName("*").lengt
+h' );
  return $el_count;
}

around get => sub {
  my $orig = shift;
  my $s = shift;
  my $url = shift;
  $s->SUPER::get($url);
  my $el_count = $s->get_element_count;

  # wait 1/10th sec for more of the page to load
  usleep 100000;

  while ($el_count != $s->get_element_count) {
    # wait 1/10th sec for more of the page to load
    usleep 100000;
    $el_count = $s->get_element_count;
  }
};



### Private methods ###


__PACKAGE__->meta->make_immutable;
1; # Magic true value