Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Presentation: Intro to Perl and LWP

by hossman (Prior)
on Mar 02, 2005 at 00:10 UTC ( #435663=perlmeditation: print w/ replies, xml ) Need Help??

I few weeks ago my boss asked me to give a short presentation on Perl and LWP to a bunch of our developers. The audience was mainly Java/JSP folks with minimal perl exposure, and the goal of the presentation was to make them aware of some of the "quick wins" that they could achieve by writting little test scripts and utilities in Perl

I figured these slides might come in handy for someone else down the road, so I decided to post them here.

NOTE: I'm the kind of person who likes my slides to be very basic, and just display the stuff that you don't want to have to convey oraly. The majority of the information from my talk was verbal; but anyone with a good background on LWP should be able to use these slides as a starting point to give the exact same talk without much preperation.


Perl & LWP HTTP Connections In Perl

What is HTTP? A simple to read (and write) protocol... lynx -source -mime_header http://www.cnet.com/bogus telnet www.cnet.com 80 GET /bogus HTTP/1.0 Host: www.cnet.com $ telnet www.cnet.com 80 Trying 216.239.115.141... Connected to c10-ssa-xw-lb.cnet.com. Escape character is '^]'. GET /bogus HTTP/1.0 Host: www.cnet.com HTTP/1.1 404 Not Found Date: Fri, 04 Feb 2005 00:43:02 GMT Server: Apache/2.0 Accept-Ranges: bytes P3P: CP="CAO DSP COR CURa ADMa DEVa PSAa PSDa IVAi IVDi CONi OUR OTRi +IND PHY ONL UNI FIN COM NAV INT DEM STA" Connection: close Content-Type: text/html Expires: Fri, 04 Feb 2005 00:43:02 GMT ... <html>

LWP is easy to use #!/usr/local/bin/perl use strict; use warnings; use LWP::UserAgent; use HTTP::Request; my $url = 'http://www.cnet.com/bogus/'; my $ua = LWP::UserAgent->new; my $request = HTTP::Request->new(GET => $url); my $response = $ua->request($request); if ($response->is_success) { print $response->content; } else { print $response->status_line, " <URL:$url>\n"; }

LWP::Simple is even easier #!/usr/local/bin/perl use strict; use warnings; use LWP::Simple; getprint('http://www.cnet.com/bogus/');

First, a Perl refresher #!/user/local/bin/perl # Tell your OS this is a perl script use strict; # If you write a script without these use warnings; # ... two lines, you owe me 10 buck +s use Module::Name; # "use" is like java's "import" my $stuff = undef; # "my" is how you declare variables # ... "undef" is like java's "null" my $bat = get_bat($name); # Function calls are just like java my $bar = ['Hoss',2,3,$stuff]; # square brackets make an array ref my $baz = {a => 1, b => 2}; # currly braces make a hash (ie: Map) + ref my $first = $bar->[0]; # -> dereferences attributes/methods $baz->{'c'} = 3; # ... like "." in java. $bat->do_something(); # if ($bar->[1]) { # "if" and "print" work the way you t +hink print "$first is true\n"; # Double quotes evaluate variables } # ... prints: Hoss is true

More fun with LWP::Simple #!/usr/local/bin/perl use strict; use warnings; use LWP::Simple; my $data = get('http://www.cnet.com/'); if (defined $data) { print $data; } if (is_success(getstore('http://mysimon.com/', './mysimon.html'))) { print "fetched mysimon front door just fine\n"; } if (RC_NOT_MODIFIED != mirror('http://mp3.com','./mp3.html')) { print "Something interesting happened with mp3.com\n"; } my ($type, $length, $mod, $exp, $server) = head('http://news.com'); print "Content Type is $type, and the Server is $server\n";

LWP::UserAgent is more "full featured" #!/usr/local/bin/perl use strict; use warnings; use LWP::UserAgent; my $ua = LWP::UserAgent->new(); $ua->cookie_jar({}); $ua->agent('Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7)'); $ua->timeout(42); # seconds $ua->proxy('http', 'http://proxy.sn.no:8001/'); $ua->no_proxy('com.com'); my $response1 = $ua->get('http://www.cnet.com/'); my $response2 = $ua->head('http://news.com/'); my $response3 = $ua->post('http://bugz.foo.com/postbug.cgi', {'descr' => 'your bug description', 'component' => 'your bug component', 'priority' => 'P1' });

HTTP::Request & HTTP::Response #!/usr/local/bin/perl use strict; use warnings; use LWP::UserAgent; use HTTP::Request; use HTTP::Response; my ($ua, $res) = (LWP::UserAgent->new(), undef); # these two lines are equivilent $res = $ua->get('http://cnet.com/'); $res = $ua->request(HTTP::Request->new('GET','http://cnet.com/')); # but you can do a lot more with the Request object... my $req = HTTP::Request->new('GET','http://builder.com/foo'); $req->header(Accept => 'text/x-dvi, text/html, text/*, */*'); $res = $ua->request($req); # the Response object also contains a lot of usefull info... print "The Status code was: ", $res->code(), "\n"; print "The Content Type was: ", $res->header('Content-Type'), "\n"; print "The URL was: ", $res->request()->uri(), "\n"; while ($res = $res->previous()) { print "Redirected from: ", $res->request()->uri(), "\n"; }

Previous script's output $ ./example-script.pl The Status code was: 200 The Content Type was: text/html The URL was: http://builder.com.com/ Redirected from: http://builder.cnet.com/webbuilding/pages/foo Redirected from: http://builder.com/foo

Other cool features worth looking at * Use proxies * POST complex requests (including file uploads) * Authenticate using Basic Credentials * Read/write cookies from disk (in Mozilla or Microsoft formats) * Parse/Submit forms auto-magically * LWP:RobotUA, LWP::Parallel::UserAgent, LWP::UserAgent::FramesReady

One caveat to LWP::Simple #!/usr/local/bin/perl use strict; use warnings; # use LWP::Simple this way, to ensure a proper LWP::UserAgent object # (that speaks HTTP/1.1) is used by all methods. # # Without it some methods use a lightweight client that only # speaks HTTP/1.0, which your server might treat differently, # and might give you different results then you expect. use LWP::Simple qw(:DEFAULT $ua); my $data = get('http://cnet.com/bogus/'); # http://www.perlmonks.org/?node=412544

More Info perldoc LWP perldoc LWP::Simple perldoc LWP::UserAgent perldoc HTTP::Request::Common perldoc lwpcook http://www.perlmonks.org/?node=168684 http://search.cpan.org/search?query=LWP http://search.cpan.org/dist/WWW-Mechanize/

Comment on Presentation: Intro to Perl and LWP
Select or Download Code
Re: Presentation: Intro to Perl and LWP
by machinecraig (Monk) on Mar 02, 2005 at 05:06 UTC
    ++! I thought this was an excellent intro to making productive use of LWP; you did a good job of presenting an ultra-quick run through of perl as well - I laughed out loud when I read your comment re: use strict + warnings.

    Thanks for the inspiring read hossman!
Re: Presentation: Intro to Perl and LWP
by crypix (Acolyte) on Mar 04, 2005 at 03:37 UTC
    I think this is a terrific intro for Perl/LWP. Well done if I might say.. ~crypix
      Is a intro for who never saw Perl! And I can tell you that for the Delphi and Java folks Perl is very strange!

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://435663]
Approved by davidj
Front-paged by ybiC
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (8)
As of 2014-12-25 05:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (159 votes), past polls