Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Doing an Input->process->Output cycle with AnyEvent? (was: error: "recursive blocking wait detected")

by isync (Hermit)
on Oct 16, 2010 at 19:34 UTC ( #865719=perlquestion: print w/ replies, xml ) Need Help??
isync has asked for the wisdom of the Perl Monks concerning the following question:

Trying to wrap my head around event programming, I ran into this error. My script:
#perl use AnyEvent; my $input_waiting = AnyEvent->condvar; my $output_waiting = AnyEvent->condvar; my $input_from_stdin = AnyEvent->io ( fh => \*STDIN, # which file handle to check poll => "r", # which event to wait for ("r"ead data) cb => sub { # what callback to execute my $input = <STDIN>; # read it push(@input, $input); $input_waiting->send; $output_waiting->recv; print shift(@output) ."\n"; } ); while(1){ # this "blocks" (while handling events) till the callback # calls ->send $input_waiting->recv; ## do some possibly lengthy processing push(@output, 'done'); $output_waiting->send; ## finish this cycle, ## AnyEvent docs say: "You can only wait once on a condition - add +itional calls are valid but will return immediately." ## but we need to stop this loop again, so this: $input_waiting = AnyEvent->condvar; }
As you can see, I am trying to implement a simple Input->process->Output mechanism, like for example chatbots use it. Where the printing of the output should be delayed as long as the processing takes. Is my overall design design okay for that?

Second, as I understand the condvar/signalling logic in AnyEvent, I can use a ->rcv once to halt my processing loop, but after that I can't tell it to go back to sleep as "You can only wait once on a condition". So I need to redeclare a new condvar - is that right?

Third, I need to halt my processing until some input arrives in @input. By doing a ->send in my input waiter I trigger the processing to take place. But how do I halt the input waiter until @output contains some value?
I can't return the @output from some other point in my code as what it is done here via STDIN/STDOUT may as well be a http connected client where I can't simply append data. I need to halt the code in $input_from_stdin for as long as the processing needs so I keep this "sessing" alive there.

Comment on Doing an Input->process->Output cycle with AnyEvent? (was: error: "recursive blocking wait detected")
Download Code
Re: Doing an Input->process->Output cycle with AnyEvent? (was: error: "recursive blocking wait detected")
by ambrus (Abbot) on Oct 16, 2010 at 21:55 UTC

    Try to wrap your head around it again in some other way. As a guideline, you almost never want to call the recv method of AnyEvent condvars from event handlers.

Re: Doing an Input->process->Output cycle with AnyEvent? (was: error: "recursive blocking wait detected")
by BrowserUk (Pope) on Oct 16, 2010 at 22:13 UTC

    Sorry for this, but I'm going to ask what might be a really dumb question.

    As I read your post: you can't process until you've obtained some input; and you can output until you've processed that input. Correct?

    If so, that sounds like a serial process to me:

    while( <> ) { ## do something to $_ print; }

    So why would you use a behemoth like AnyEvent to do that?


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      I expected this to come up. And to be honest: I am not quite sure if AnyEvent should *really* stay onboard or not...

      Well, the verbose explanation is that this input output thing here is part of a larger system, where multiple things go on simultaneously. AnyEvent's timers come in handy, I've got kind of a main-loop... stuff like that. So I thought: I think that's where others would throw an event framework into the mix. So when I needed to add this input/output interface here, a component which influences the various parts of the system, I thought it would be a consistent move to model this with AnyEvent's tools as well.
      Wrong asumption?
        So when I needed to add this input/output interface here, a component which influences the various parts of the system, I thought it would be a consistent move to model this with AnyEvent's tools as well. Wrong asumption?

        Hm. Let me be up-front and say: I don't understand why anyone would voluntarily choose to use an event-driven system on a pre-emptive, multi-tasking operating system. The phrase that comes to mind is: "Like buying a dog and barking yourself".

        It always reminds me of one of those plate-spinning acts. The guy or girl is running around splitting their time between trying to load up new plates onto poles, whilst keeping the existing ones spinning fast enough to stop them falling. Everyone gets behind them cheering them on. Unwittingly helping them out by collectively "whoooa"ing, whenever one of them starts to wobble violently. Eventually they either complete the poles available or their timeslot comes to an end and everyone applauds with admiration. But in the end, we all know they're all going to come crashing down.

        In the past, I've had to program such systems, because there was no other option available. They can be made to work well, and can be very efficient. But at what expense? They are so complicated and fragile. Difficult to program; a nightmare to debug; and horrendously expensive to maintain. One change to the specification; or the need to move to different hardware; and you're into a game of having to re-tune every part of the system.

        And once you're locked into a given framework, nearly everything you want to do requires a "special version" of everyday modules. Just witness the hundreds of modules that turn up when you search for AnyEvent::* or POE::*. And see how many of them are specially adapted versions of common, well used modules like ...::LWP::* and ...::NET::* and ...::IO::*. And then there are the dependency chains. A bug arises and a fix is a applied to one of those core modules, but then you have to wait for that fix to make it's way through the chain into the specialist version you're using.

        Their proponents will tell you horror stories of deadlocks and inversion chains, and all sorts of other nasties, if you bring up threads; but as you've discovered, you don't avoid synchronisation issues with an event-based system. Indeed, you have to use it to try re-create a simple serial sequence of DoA() then DoB() then DoC().

        You want to read-process-write, see the loop above. You need to do other things while that goes on:

        async{ while( <> ) { ## process $_ print; } };

        Now you can!

        Let that well-tested, highly tuned, pre-emptive multi-tasking scheduler take care of making sure that when things need doing, they get done. None of this interrupting everything you're doing, every 1/10th of a second, to run around everywhere else asking (polling): Does anything need doing? Does anything need doing?

        Don't feel the need to counter this--it's just my view on things--but I've yet to see an event-driven application that wasn't simpler written and maintained using threading. That's actually a challenge that I'd love to see someone take up. But I doubt they will. I'll let you decide the reason why not :)


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Doing an Input->process->Output cycle with AnyEvent? (was: error: "recursive blocking wait detected")
by james2vegas (Chaplain) on Oct 17, 2010 at 00:16 UTC
    Wrap the creation of your condvars and AE::io objects in your while loop and use callbacks on the condvars instead of procedural code to do the sequential tasks, something like this:
    #!/usr/bin/perl use strict; use warnings; use AnyEvent; while (1) { my $cv = AE::cv; my (@output, @input); my $input_waiting = AE::cv; my $output_waiting = AE::cv; my $input_from_stdin = AE::io *STDIN, 0, sub { my $input = <STDIN>; # read it push(@input, $input); $input_waiting->send; }; # what to do when $input_waiting has sent $input_waiting->cb( sub { push(@output, 'done'); $output_waiting->send; }); # what to do when $output_waiting has sent $output_waiting->cb( sub { print shift(@output) ."\n"; $cv->send; }); # wait for the main condvar $cv->recv; }
    the while block ensures that new condvars and AE::io objects get created on each iteration of the loop. I used the simpler AE API, but that shouldn't affect the functioning of the code. You could probably get by without the $cv condvar, but it may be helpful as this is part of a larger application. You should look at the begin and end methods of AE::condvar which enables the use of single condvars as mergepoints for events, and Object::Event for a lightweight event registration and emitting library.
      Thanks for your design input. As I mentioned in my previous reply to BrowserUk, I am tempted to walk away from event based programming for the task, at least for now.

      But before I do I will give it a last try with your proposed structure. As I've learned from your post, redeclaring condvars over and over again seems to be okay to send the overall system into stop and start states, as I can use a cv only once.
      I've already looked into the begin/end methods a bit, but so far did not need its "counter" mechanism. We'll see.

      Again, thanks!
Re: Doing an Input->process->Output cycle with AnyEvent? (was: error: "recursive blocking wait detected")
by zentara (Archbishop) on Oct 17, 2010 at 11:37 UTC
    This how you might do it with the Glib event loop system.
    #!/usr/bin/perl use warnings; use strict; use Glib; my $main_loop = Glib::MainLoop->new; Glib::IO->add_watch (fileno 'STDIN', [qw/in/], \&watch_callback, 'STDI +N'); #just to show it's non blocking my $timer1 = Glib::Timeout->add (1000, \&testcallback, undef, 1 ); $main_loop->run; sub watch_callback { # print "@_\n"; my ($fd, $condition, $fh) = @_; my $line = readline STDIN; print $line; #always return TRUE to continue the callback return 1; } sub testcallback{ print "\t\t\t".time."\n"; } __END__

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://865719]
Approved by Corion
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (9)
As of 2014-07-25 03:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (167 votes), past polls