Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??

I few weeks ago my boss asked me to give a short presentation on Perl and LWP to a bunch of our developers. The audience was mainly Java/JSP folks with minimal perl exposure, and the goal of the presentation was to make them aware of some of the "quick wins" that they could achieve by writting little test scripts and utilities in Perl

I figured these slides might come in handy for someone else down the road, so I decided to post them here.

NOTE: I'm the kind of person who likes my slides to be very basic, and just display the stuff that you don't want to have to convey oraly. The majority of the information from my talk was verbal; but anyone with a good background on LWP should be able to use these slides as a starting point to give the exact same talk without much preperation.


Perl & LWP HTTP Connections In Perl

What is HTTP? A simple to read (and write) protocol... lynx -source -mime_header http://www.cnet.com/bogus telnet www.cnet.com 80 GET /bogus HTTP/1.0 Host: www.cnet.com $ telnet www.cnet.com 80 Trying 216.239.115.141... Connected to c10-ssa-xw-lb.cnet.com. Escape character is '^]'. GET /bogus HTTP/1.0 Host: www.cnet.com HTTP/1.1 404 Not Found Date: Fri, 04 Feb 2005 00:43:02 GMT Server: Apache/2.0 Accept-Ranges: bytes P3P: CP="CAO DSP COR CURa ADMa DEVa PSAa PSDa IVAi IVDi CONi OUR OTRi +IND PHY ONL UNI FIN COM NAV INT DEM STA" Connection: close Content-Type: text/html Expires: Fri, 04 Feb 2005 00:43:02 GMT ... <html>

LWP is easy to use #!/usr/local/bin/perl use strict; use warnings; use LWP::UserAgent; use HTTP::Request; my $url = 'http://www.cnet.com/bogus/'; my $ua = LWP::UserAgent->new; my $request = HTTP::Request->new(GET => $url); my $response = $ua->request($request); if ($response->is_success) { print $response->content; } else { print $response->status_line, " <URL:$url>\n"; }

LWP::Simple is even easier #!/usr/local/bin/perl use strict; use warnings; use LWP::Simple; getprint('http://www.cnet.com/bogus/');

First, a Perl refresher #!/user/local/bin/perl # Tell your OS this is a perl script use strict; # If you write a script without these use warnings; # ... two lines, you owe me 10 buck +s use Module::Name; # "use" is like java's "import" my $stuff = undef; # "my" is how you declare variables # ... "undef" is like java's "null" my $bat = get_bat($name); # Function calls are just like java my $bar = ['Hoss',2,3,$stuff]; # square brackets make an array ref my $baz = {a => 1, b => 2}; # currly braces make a hash (ie: Map) + ref my $first = $bar->[0]; # -> dereferences attributes/methods $baz->{'c'} = 3; # ... like "." in java. $bat->do_something(); # if ($bar->[1]) { # "if" and "print" work the way you t +hink print "$first is true\n"; # Double quotes evaluate variables } # ... prints: Hoss is true

More fun with LWP::Simple #!/usr/local/bin/perl use strict; use warnings; use LWP::Simple; my $data = get('http://www.cnet.com/'); if (defined $data) { print $data; } if (is_success(getstore('http://mysimon.com/', './mysimon.html'))) { print "fetched mysimon front door just fine\n"; } if (RC_NOT_MODIFIED != mirror('http://mp3.com','./mp3.html')) { print "Something interesting happened with mp3.com\n"; } my ($type, $length, $mod, $exp, $server) = head('http://news.com'); print "Content Type is $type, and the Server is $server\n";

LWP::UserAgent is more "full featured" #!/usr/local/bin/perl use strict; use warnings; use LWP::UserAgent; my $ua = LWP::UserAgent->new(); $ua->cookie_jar({}); $ua->agent('Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7)'); $ua->timeout(42); # seconds $ua->proxy('http', 'http://proxy.sn.no:8001/'); $ua->no_proxy('com.com'); my $response1 = $ua->get('http://www.cnet.com/'); my $response2 = $ua->head('http://news.com/'); my $response3 = $ua->post('http://bugz.foo.com/postbug.cgi', {'descr' => 'your bug description', 'component' => 'your bug component', 'priority' => 'P1' });

HTTP::Request & HTTP::Response #!/usr/local/bin/perl use strict; use warnings; use LWP::UserAgent; use HTTP::Request; use HTTP::Response; my ($ua, $res) = (LWP::UserAgent->new(), undef); # these two lines are equivilent $res = $ua->get('http://cnet.com/'); $res = $ua->request(HTTP::Request->new('GET','http://cnet.com/')); # but you can do a lot more with the Request object... my $req = HTTP::Request->new('GET','http://builder.com/foo'); $req->header(Accept => 'text/x-dvi, text/html, text/*, */*'); $res = $ua->request($req); # the Response object also contains a lot of usefull info... print "The Status code was: ", $res->code(), "\n"; print "The Content Type was: ", $res->header('Content-Type'), "\n"; print "The URL was: ", $res->request()->uri(), "\n"; while ($res = $res->previous()) { print "Redirected from: ", $res->request()->uri(), "\n"; }

Previous script's output $ ./example-script.pl The Status code was: 200 The Content Type was: text/html The URL was: http://builder.com.com/ Redirected from: http://builder.cnet.com/webbuilding/pages/foo Redirected from: http://builder.com/foo

Other cool features worth looking at * Use proxies * POST complex requests (including file uploads) * Authenticate using Basic Credentials * Read/write cookies from disk (in Mozilla or Microsoft formats) * Parse/Submit forms auto-magically * LWP:RobotUA, LWP::Parallel::UserAgent, LWP::UserAgent::FramesReady

One caveat to LWP::Simple #!/usr/local/bin/perl use strict; use warnings; # use LWP::Simple this way, to ensure a proper LWP::UserAgent object # (that speaks HTTP/1.1) is used by all methods. # # Without it some methods use a lightweight client that only # speaks HTTP/1.0, which your server might treat differently, # and might give you different results then you expect. use LWP::Simple qw(:DEFAULT $ua); my $data = get('http://cnet.com/bogus/'); # http://www.perlmonks.org/?node=412544

More Info perldoc LWP perldoc LWP::Simple perldoc LWP::UserAgent perldoc HTTP::Request::Common perldoc lwpcook http://www.perlmonks.org/?node=168684 http://search.cpan.org/search?query=LWP http://search.cpan.org/dist/WWW-Mechanize/

In reply to Presentation: Intro to Perl and LWP by hossman

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • Outside of code tags, you may need to use entities for some characters:
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others pondering the Monastery: (9)
    As of 2014-09-02 19:41 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      My favorite cookbook is:










      Results (29 votes), past polls