Greetings dear Monks,
I have been trying to, for my work project, get a Perl program that can automatically download all the packages linked to by a single Google web-page. Nothing nefarious of course, but there are literally hundreds of 20-100 MB sized files there, and I don't fancy doing this manually. I've actually been instructed to build such a program.
The program isn't the hard part per se, as I've already completed the task in multiple ways, theoretically. The problem is, when I run even the following test code (with the extra "Use" statements left over from previous attempts):
#!/usr/bin/perl
use LWP::UserAgent;
use LWP::Simple;
use URI::URL;
use WWW::Mechanize;
use HTML::LinkExtor;
my $url = 'http://foo.bar.baz';
getprint('http://foo.bar.baz');
$user = LWP::UserAgent->new();
$user->get($url);
I get the following error in command prompt (I am using Windows 8, don't ask):
500 Status read failed: A non-blocking socket operation could not be c
+ompleted i
mmediately. <URL:http://foo.bar.baz>
What am I doing wrong with my approach? Is there a way to fix/bypass this? Could a different programming language get the job done? I've gotten the script to download the files successfully (I tried on one of them manually with the script using LWP::Simple to save the file on my disk), but the page that links the downloads is unreadable apparently.
UPDATE: I've tried wget, curl, and a few other things - even a method that worked yesterday to grab a test file off the net, and today the method I used to download a test file off the net with perl, doesn't work.
Every time I use lwp to connect ANYWHERE on the net with perl now, it gives me the "500 Status Read Failed" error, "a non-blocking socket operation could not be completed immediately". I'm completely baffled by this. I can connect to local html files with lwp, but not anything on the internet, and I have no firewalls up.
