Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

Re^9: Need help with WWW::Mechanize and Chrome cookies

by bakiperl (Beadle)
on Jul 11, 2021 at 12:20 UTC ( #11134913=note: print w/replies, xml ) Need Help??

in reply to Re^8: Need help with WWW::Mechanize and Chrome cookies
in thread Need help with WWW::Mechanize and Chrome cookies

Here is the issue:
First, let's start with the html file that I have used to test the script (WMC.html)
<html> <head> <meta content="text/html; charset=utf-8" http-equiv="Content-Type" /> <title>Testing hyperlink file Downloads</title> </head> <body> <h2>Testing Download of hyperlinked files using WWW::Mechanize::Chrome +</h2> <p></p> Let's try downloading this <a href="/my_Files/csv_File.csv">CSV File</ +a> <br/><br/> </body> </html>
Now here is the Perl script.
#!/usr/bin/perl -w use Log::Log4perl qw(:easy); use WWW::Mechanize; use WWW::Mechanize::Chrome; use strict; my $cookie_dir = 'C:/Users/some_user/AppData/Local/Google/Chrome/User +Data/Default/'; #chrome cookies path #my $mech = WWW::Mechanize::Chrome->new( data_directory => $cookie_dir +); my $mech = WWW::Mechanize->new(); my $uri = URI->new( "" ); $mech->get( $uri ); unless ($mech->success) { my $mesg = $mech->response->status_line; print $mesg; goto FINISH; } my $path = "/path"; my @links = $mech->find_all_links( url_regex => qr/\.csv/i ); my @urls = map { $_->[0] } @links; for my $foo (@urls) { my $filename = $path.$foo; $mech->get($foo, ':content_file'=>$filename); my $file_content = $mech->get($foo); print $file_content->content(); } print "Success\n"; FINISH :
When I use the WWW::Mechanize instance, the script runs fine. It prints and saves the file content to disk.
However, when the WWW::Mechanize::Chrome instance is used I get the following error message:
Cannot navigate to invalid URL -32000 at C:/Perl/perl/site/lib/Chrome/DevToolsProtocol/ line + 490

Replies are listed 'Best First'.
Re^10: Need help with WWW::Mechanize and Chrome cookies
by Corion (Patriarch) on Jul 11, 2021 at 15:20 UTC
Re^10: Need help with WWW::Mechanize and Chrome cookies
by Corion (Patriarch) on Jul 11, 2021 at 14:09 UTC

    This is really weird - what is the value of the URL you're trying to navigate to when that error occurs? Where does that error occur?

      I sent you the url.

      The error occurs on both of these calls:
      $mech->get($foo, ':content_file'=>$filename); my $file_content = $mech->get($foo);

        Yes. So what is the value of $foo? Did you print it? Is the loop necessary for the bug to happen, or does the error happen with the hardcoded value for $foo as well?

        From the URL you sent to me, it seems that WWW::Mechanize returns absolute URLs, while WWW::Mechanize::Chrome returns whatever URLs are contained in the file. Simple printing of the values can confirm that to you.

        Until WWW::Mechanize::Chrome achieves API parity with WWW::Mechanize in that part, consider converting the URL to absolute yourself.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11134913]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (2)
As of 2023-03-25 21:27 GMT
Find Nodes?
    Voting Booth?
    Which type of climate do you prefer to live in?

    Results (63 votes). Check out past polls.