Retrieving Data from URL

[arunhorne]

Monks,

I have a URL http://ca.expasy.org/cgi-bin/get-sprot-fasta?Q9PKH8 that results in text/plain output. I want to store this to a file. How can I go about doing this using Perl... the URL will be generated many times with different parameters hence need for automation.

Thanks, Arun

[lshatzer]

Check out LWP::Simple, it has a nice function getstore, where it will store the results in a file.

[merlyn]

LWP is your friend. Use LWP. Install it, then perldoc lwpcook for many examples of usage.

-- Randal L. Schwartz, Perl hacker

[arunhorne]

Thanks, LWP worked brilliantly.

Just as an aside for anyone looking at this thread in the future, I am behind a firewall and so had to add the following code to make it all work.

use LWP::Simple qw(&getstore &is_success $ua);

The above imports the $ua variable and the subs I needed, $ua (User Agent, see LWP::UserAgent) is used to set proxy settings.

# Initialise the proxy
$ua->proxy(http => 'http://wwwcache.ex.ac.uk:8080'); 
#$ua->proxy_authorization_basic( 'username', 'password' );

The above actually sets the proxy. I didn't need any proxy auth, but you might, hence the line is there but commented out. Once this is done the subs like getstore worked great.

Hope this helps in the future.

Best wishes and thanks again, Arun