Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

POE::Component::RSSAggregator breaks LWP::Simple::get

by blahblahblah (Priest)
on Jan 19, 2007 at 06:16 UTC ( #595408=perlquestion: print w/replies, xml ) Need Help??

blahblahblah has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

Inspired by the Advanced Perl Programming book, I'm trying to use a POE module for the first time. As part of my script I also want to download a small web page, so I'm trying to use LWP::Simple. When I run my script, it gets to the LWP::Simple::get call and hangs. I have to use task manager to kill it (the cmd prompt won't respond to Ctrl-C). I took out most of my code and reproduced the problem with this simple script:

my $url = "http://wfmu.org/listen.m3u?show=21788&archive=32709"; use LWP::Simple; print "getting $url ...\n"; my $content = LWP::Simple::get($url); die "Problem!" unless defined $content; print "content: $content\n"; use POE qw(Component::RSSAggregator); # bad line __END__ With "bad line" commented out, it works: F:\cgi\wfmu>perl testget.pl getting "http://wfmu.org/listen.m3u?show=21788&archive=32709" content: http://archive.wfmu.org:5555/archive/NU/nu070117.mp3 With "bad line" present, it hangs: F:\cgi\wfmu>perl testget.pl getting "http://wfmu.org/listen.m3u?show=21788&archive=32709"
It works if I just "use POE", so maybe it's specifically the PoCo::RSSAggregator module that's the problem.?

I've poked around in the module source a very little bit looking for the problem, but I didn't see anything. Being completely unfamiliar with how POE works, I thought I'd ask for help before spending a lot of time on something obvious that I might be overlooking.

Thanks for any suggestions,
Joe

Replies are listed 'Best First'.
Re: POE::Component::RSSAggregator breaks LWP::Simple::get
by bingos (Vicar) on Jan 19, 2007 at 10:23 UTC

    First up, it is not a good idea to mix blocking code such as LWP::Simple with POE.

    POE has various components for doing what LWP does, including POE::Component::Client::HTTP which you should already have installed, as it is a prereq of POE::Component::RSSAggregator

    Your snippet of code worked for me on NetBSD, which leads me to suspect that you are using very old versions of POE and POE::Component::RSSAggregator. What versions are you using ?

      First up, it is not a good idea to mix blocking code such as LWP::Simple with POE.

      And therein lies the rub with POE.

      Once you invite it into your code, forget everything else you know about writing Perl. Forget all those familiar modules that you've been using forever. Forget reading a file, or querying a directory, or asking the user a quick question, because unless there is a module in the POE::* namespace that does exactly what you want to do, and that module has been thoroughly tested and kept up to date with all the other POE::* modules you're gonna need, you're stuffed.

      Because once you go the 'build our own cooperative scheduler' route, everything--and that means everything--in your program has to be be broken up into small bite-sized chunks, and retain state across those chunks, so that the non-preemptive scheduler doesn't get locked out.

      It like using Lego. So long as everything you want to build has straight sides and is either a multiple of some fixed sized unit, or there is a (usually expensive) off-the-shelf part to suit, it's great. But if you want curved sides; or cirular holes; or 120° angles; or anything else that is vaguely custom--you're stuffed.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        Once you invite it into your code, forget everything else you know about writing Perl. ....

        I call bogus.

        File and directory access is generally insignificant. If you're writing high-performance programs that simply can't wait for the filesystem, POE and IO-AIO get along nicely.

        As we've seen elsewhere in the thread, POE-Component-Generic can wrap an asynchronous interface around all those blocking modules you know and love, assuming there isn't a POE::Component to your liking and you're not sufficiently motivated to write one.

        Oh, right, and some POE components aren't maintained as well as they might be. Welcome to CPAN.

        Because once you go the 'build our own cooperative scheduler' route, everything--and that means everything--in your program has to be be broken up into small bite-sized chunks, and retain state across those chunks, so that the non-preemptive scheduler doesn't get locked out.

        Semi-accurate but exaggerated.

        It really depends on the program in question. It's generally accepted that well-written code is already decomposed into bite-sized pieces: small functions and methods that do one thing well, and larger ones that are composed of glue logic between calls to the smaller ones. In this case, you may find that a lot of your code will work as-is.

        POE provides ways to maintain state between cooperative callbacks, but the POE-agnostic parts of a program don't even need that.

        A program that requires major restructuring to work in a cooperative environment may already have bigger problems.

        It like using Lego. So long as everything you want to build has straight sides and is either a multiple of some fixed sized unit, or there is a (usually expensive) off-the-shelf part to suit, it's great. But if you want curved sides; or cirular holes; or 120 angles; or anything else that is vaguely custom--you're stuffed.

        Completely bogus.

        Lego are proprietary hardware. Most users can't fabricate their own bricks. POE is a suite of free, open-source software libraries, with internal and external extension APIs. If you can't find just the right component, you're encouraged to write it and share it with the rest of the world.

        I feel the same way. I'm always amazed when people casually suggest using POE, or a similar non-blocking I/O scheme. It makes trivial things difficult. That's why the forking/threading approach is so popular: you can ignore these issues most of the time.
Re: POE::Component::RSSAggregator breaks LWP::Simple::get
by rcaputo (Chaplain) on Jan 19, 2007 at 19:28 UTC

    So after chiming in on the side discussion, I actually tried to figure out your problem. Unfortunately, I can't reproduce it, so I wonder if something else is at fault:

    1) poerbook:~/projects/disorganized/support% perl perlmonks-poco-rssaggregator.perl
    getting http://wfmu.org/listen.m3u?show=21788&archive=32709 ...
    content: http://archive.wfmu.org:5555/archive/NU/nu070117.mp3
    
    1) poerbook:~/projects/disorganized/support% cat perlmonks-poco-rssaggregator.perl
    my $url = "http://wfmu.org/listen.m3u?show=21788&archive=32709";
    use LWP::Simple;
    print "getting $url ...\n";
    my $content = LWP::Simple::get($url);
    die "Problem!" unless defined $content;
    print "content: $content\n";
    use POE qw(Component::RSSAggregator);  # bad line
    1) poerbook:~/projects/disorganized/support% perl -v
    
    This is perl, v5.8.8 built for darwin-thread-multi-2level
    
    Copyright 1987-2006, Larry Wall
    
    Perl may be copied only under the terms of either the Artistic License or the
    GNU General Public License, which may be found in the Perl 5 source kit.
    
    Complete documentation for Perl, including FAQ lists, should be found on
    this system using "man perl" or "perldoc perl".  If you have access to the
    Internet, point your browser at http://www.perl.org/, the Perl Home Page.
    
    1) poerbook:~/projects/disorganized/support% uname -a
    Darwin poerbook.local 8.8.0 Darwin Kernel Version 8.8.0: Fri Sep  8 17:18:57 PDT 2006; root:xnu-792.12.6.obj~1/RELEASE_PPC Power Macintosh powerpc
    
Re: POE::Component::RSSAggregator breaks LWP::Simple::get
by blahblahblah (Priest) on Jan 20, 2007 at 02:15 UTC
    Thanks for all the feedback. I just checked my various versions:

    Perl: 5.8.4 from ActiveState (Win XP)
    POE: 0.29     ** 2.5 yrs old!!!!
    POE::Component::RSSAggregator: 1.022

    I haven't downloaded the latest POE yet, but I will try it.

    I just installed these modules last night, using ActiveState's PPM. I guess the repository I'm pointing to is out of date. Now I have to remember how to install the module manually (do I still need to get my own "make"? I haven't done this on windows in years) and/or find an up-to-date PPM repository.

    Thanks again,
    Joe

    UPDATE:
    I got POE version 0.9917 from http://theoryx5.uwinnipeg.ca/cgi-bin/ppmserver?urn:/PPMServer58 and now my script works.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://595408]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (4)
As of 2019-12-13 02:07 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?