Beefy Boxes and Bandwidth Generously Provided by pair Networks Russ
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??

Hello good monks,

I've recently come up with something I haven't seen before. Granted, this hardly means it hasn't been done before; I just haven't run across it previously. If there's prior art, I'd be interested by all means!

We all know about LWP::UserAgent and WWW::Mechanize -- and I've certainly gained much purchase from using only those modules for a long time -- but there's a certain class of problems that I end up solving repeatedly, in a very similar way. That is: making HTTP requests, reacting to the responses, possibly making more requests, responding to those responses, ad infinitum. I've never liked using imperative Perl code to write this kind of bread and butter request/response/request/response chain.

It struck me that this is a straightforward recursive problem. It can be solved easily with a data structure to describe the requests, their responses, further requests for those responses, and so on. This way the actual HTTP processing (i.e., the use of LWP::UserAgent) is isolated to a simple recursive routine (with good, consistent error handling!), but the definition of the expected behavior is somewhere else.

Now, in case anyone is curious, I'll include an incomplete, and probably broken example. This is what I've been using to develop the idea. This is supposed to be a last.fm track submission client. Right now it does authentication and submits a single bogus track.

Here's the definition:

my $REQUESTS = [ { uri => "http://post.audioscrobbler.com", params => [ hs => 'true', p => '1.1', c => $CLIENT_ID, v => $CLIENT_VER, u => $USER, ], responses => [ { name => 'Successful handshake', content_pattern => qr/^UPTODATE\n([^\n]+)\n([^\n]+)\nI +NTERVAL (.*)/, requests => sub { my ($md5, $submit, $interval) = @_; my $hash = md5_hex(md5_hex($PASS) . $md5); print "Pausing $interval\n"; sleep $interval; return ( { uri => $submit, params => [ u => $USER, s => $hash, 'a[0]' => 'Test Artist', 't[0]' => 'Test Track', 'b[0]' => 'Test Album', 'm[0]' => '', 'l[0]' => 5*60, 'i[0]' => strftime("%Y-%m-%d %H:%M:%S" +, gmtime), ], responses => [ { name => 'Successful submission', content_pattern => qr/^OK\nINTERVA +L (.*)/, requests => sub { print "Pausing $_[0]\n"; sleep $_[0]; return; }, }, { name => 'Failed submission', content_pattern => qr/^FAILED ([^\ +n]*)/, requests => sub { print "Failure: $_[0]\n"; return; }, }, { name => 'Failed authentication', content_pattern => qr/^BADAUTH/, }, ] }, ); }, }, { content_pattern => qr/^UPDATE/ }, { content_pattern => qr/^FAILED/ }, { content_pattern => qr/^BADUSER/ }, ], }, ];

And here's the handler routine

sub handle_requests { my ($agent, $requests, @params) = @_; my @requests; if (ref $requests eq 'ARRAY') { @requests = @$requests; + } elsif (ref $requests eq 'CODE') { @requests = $requests->(@params +); } for (@requests) { my $uri = URI->new($_->{uri}); $uri->query_form($_->{params}); print "Request: $uri\n"; my $response = $agent->get($uri); unless ($response->is_success) { die $response->status_line; } for (@{ $_->{responses} }) { if (my @groups = $response->content =~ $_->{content_patter +n}) { my $name = $_->{name} || "Matched $_->{content_pattern +}"; print "Response: $name\n"; handle_requests($agent, $_->{requests}, @groups); } } } }

I wasn't entirely sure if I should post this code, because this Meditation is really intended to be about the idea. I can think of all kinds of ways to improve the code (in terms of readability, conciseness, completeness, correctness), but I posted it to help explain the idea, in case my textual description is lacking. I'd really like to know, (1) if there is already a well-known -- or even not-well-known -- implementation of this kind of thing out there somewhere, and (2) if this is interesting enough to anybody else that I should spend more time working on it.

Thanks for any response!


In reply to Data driven HTTP interaction by revdiablo

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • Outside of code tags, you may need to use entities for some characters:
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others contemplating the Monastery: (9)
    As of 2014-04-16 23:04 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      April first is:







      Results (436 votes), past polls