Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

LWP download: how to handle open/save/cancel prompt

by avid (Novice)
on Feb 15, 2006 at 21:48 UTC ( #530531=perlquestion: print w/replies, xml ) Need Help??

avid has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to download some excel files that are generated by pressing a link. At the press, the original site redirects the request to another site that generates the excel and the net result is a open/save/cancel/more_info prompt box. I need to save the file from the script. I could log-on to the website correctly by using LWP and am stuck there. The open/save prompt box is a seperate popup window, so I cannot view code behind it. I am a newby in this so do not know mechanism behind popups. Here is my code:
my $HOST = 'http://webserver_address'; my $DESTINATION = '/rel_path/nt/GetStats.asp' print "Enter Username: "; chop( $c0 = <STDIN> ); print "\n"; system "stty -echo"; print "Enter Password: "; chop( $c1 = <STDIN> ); print "\n"; system "stty echo"; package MyAgent; use base 'LWP::UserAgent'; sub get_basic_credentials { return "$c0", "$c1"; } package main; my $agent = MyAgent->new; push @{ $agent->requests_redirectable }, 'POST'; $agent->cookie_jar( HTTP::Cookies->new() ); my $response = $agent->get( "$HOST" ); print $response->code; #check after the initial login success if ( $response->is_success ) { print "\nGot OK\n"; print "Reading Excel...\n"; $response = $agent->request( POST $HOST . $DESTINATION, [ Prod => '8345', User => "$c0", Code => "ONET", URL => "http://anotherurl/getdata?action=li +ve_report&ID=3", ] ); if ( $response->is_success ) { $tmp = $response->content; } else { die "Request failed, got status [" . $response->status_line . "] +\n"; } }
Actual link that works from a browser:
http://webserver_address/rel_path/nt/GetStats.asp?Prod=8345&User=AVID& +Code=ONET&URL=http://another_url/getdata?action=live_report&ID=3
Runlog:
Enter Username: avid Enter Password: 200 Got OK Reading Excel... Request failed, got status [500 Internal Server Error]
I am also new to perl and gathered the above code by search-paste-play with the codes. Not knowing anything internal to lwp. :( Any help or insight will be greatly appreaciated.

Replies are listed 'Best First'.
Re: LWP download: how to handle open/save/cancel prompt
by dannoura (Pilgrim) on Feb 15, 2006 at 22:23 UTC

    Try using an HTTP sniffer to find out what goes on when you press either of the buttons (the Live HTTP Headers extension for Firefox also works). After you find out you can work out what needs to be done.

Re: LWP download: how to handle open/save/cancel prompt
by polettix (Vicar) on Feb 16, 2006 at 01:10 UTC
    There are a bunch of things that may go wrong, the sniffing suggestion is to be taken in high consideration. I'd also suggest to switch to WWW::Mechanize, which is a sub-class of LWP::UserAgent that helps a lot in this kind of "stateful" navigation (but beware that any Javascript will not be handled). Good luck!

    Flavio
    perl -ple'$_=reverse' <<<ti.xittelop@oivalf

    Don't fool yourself.
Re: LWP download: how to handle open/save/cancel prompt
by johnnywang (Priest) on Feb 16, 2006 at 05:41 UTC
    Yes, get a sniffer like Ethereal ( free at www.ethereal.com), capture your browser's interaction with the site. The popup is likely just a javascript confirmation, could be on the same page, or be sent as a response, then redirect. All these will be obvious after you look at the source of the page and at what goes on the wire.
Re: LWP download: how to handle open/save/cancel prompt
by avid (Novice) on Feb 17, 2006 at 21:49 UTC
    Thanks folks for these prompt replies.
    I will play with these and will post the results or any solution to this page.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://530531]
Approved by kutsu
Front-paged by kutsu
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (4)
As of 2021-10-20 14:37 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My first memorable Perl project was:







    Results (81 votes). Check out past polls.

    Notices?