Hi Monks,
I have a few problems that I hope you might be able to help with. I'm trying to get pull a set of data from a number of webservers about every 10 seconds. Each set contains a url of data, and two urls of images. I need to save each set either in a directory based on the name or a file based on the name.
My problems are:
1. I can't figure out how to match the returned data with the name
2. I'm unable to get this working using basic authentication.
This is what I have so far, but I'm stumped and need help...
#!/usr/bin/perl
use strict;
use warnings;
require HTTP::Request;
use Data::Dumper;
use HTTP::Async;
#load the list of targets from a file
#file format is:
#server_name,username,password,https://{url}
my $file='conf.cfg';
my @data;
open(my $fh, '<', $file) or die "Can't read config file '$file' [$!]\n
+";
while (my $line = <$fh>) {
chomp $line;
my @fields = split(/,/, $line);
push @data, \@fields;
}
my $async = HTTP::Async->new;
$async->proxy_host("proxy.my.company.com");
$async->proxy_port("8080");
$async->timeout(10);
$async->max_request_time(10);
$async->slots(100);
while(1){
foreach(@data) {
my $page;
my $name=$_->[0];
my $account=$_->[1];
my $password=$_->[2];
my $url=$_->[3] . "/info.html";
my $port=$_->[3];
$port=~s/.*://g;
my $method=$_->[3];
$method=~s/:\/\/.*//g;
my $img1=$_->[3]."/image1.jpg";
my $img2=$_->[3]."/image2.jpg";
my $url=$method . "://" . $addr . ":" . $port . "/status/captures"
+;
my $now = time;
$async->add( HTTP::Request->new( GET => $url) );
$async->add( HTTP::Request->new( GET => $img1) );
$async->add( HTTP::Request->new( GET => $img2) );
while ( my $response = $async->wait_for_next_response ) {
my $url = sprintf "%-20s,%-25s", $venue, $url;
my $len = sprintf "%8s", length $response->content;
my $et = sprintf "%5.3f", time - $now;
print "$url has length $len and loaded in $et s \n";
if ($response->content eq "Timed out"){
$page="Timed out";
}
else{
$page=$response->content;
}
open (FILE, '>', $name);
print FILE $page;
close (FILE);
}
}
sleep(10);
}
exit(0);
-
Are you posting in the right place? Check out Where do I post X? to know for sure.
-
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big>
<blockquote> <br /> <dd>
<dl> <dt> <em> <font>
<h1> <h2> <h3> <h4>
<h5> <h6> <hr /> <i>
<li> <nbsp> <ol> <p>
<small> <strike> <strong>
<sub> <sup> <table>
<td> <th> <tr> <tt>
<u> <ul>
-
Snippets of code should be wrapped in
<code> tags not
<pre> tags. In fact, <pre>
tags should generally be avoided. If they must
be used, extreme care should be
taken to ensure that their contents do not
have long lines (<70 chars), in order to prevent
horizontal scrolling (and possible janitor
intervention).
-
Want more info? How to link
or How to display code and escape characters
are good places to start.