AnyEvent for I/O events

[elmex]

As POE seems to be very popular these days I wanted to share an alternative way of handling non-blocking I/O (and other events). For this I include a simple made up example, where I will show how to wait for a filehandle "readable" event and wait for a child process to finish.

But first a quick overview/introduction to AnyEvent:

AnyEvent provides us with 'watchers', which watch out for events. Events like a filehandle becoming readable or writable, a timer that reaches his timeout, a signal or a condition becoming true.

To handle all these events AnyEvent needs an event loop. The interesting thing here is, that AnyEvent can use other event loops, like the one provided by Glib, which would allow easy integration with Gtk2 programs. Aside from that AnyEvent knows how to interface with the event loops of the Event and EV module, Tk and even Coro. For the case that none of these event loops is either loaded or their module installed on the system it comes with it's own pure Perl implementation.

The major advantage comes in when you write a module and you need to wait for I/O events or need timers: you don't need to decide which event loop to use or even come up with your own select() implementation. If your module uses AnyEvent it can be used in POE, IO::Async, Gtk2, Tk, Coro, and other modules that also use AnyEvent.

Now a simple example where I commented the most interesting parts:

#!/opt/perl/bin/perl
use strict;
use AnyEvent;
use POSIX;
use IO::Handle;
use Time::HiRes qw/usleep/;

pipe my $par, my $chld;

$par->autoflush (1);
$chld->autoflush (1);

my $pid = fork;
defined $pid
   or die "fork failed: $!";

if ($pid) {
   # parent:

   # first we create a condition variable, which allows
   # us to wait for a 'condition' to become true.
   my $cvar = AnyEvent->condvar;


   # then we setup a watcher which will watch for the 'readable'
   # event for the parent end of the pipe. The callback we
   # pass will be executed when the filehandle becomes readable.
   my $read_watcher = AnyEvent->io (fh => $par, poll => 'r', cb => sub
+ {
      my $r = sysread $par, my $data, 1024;

      if (not defined $r) {
         $! == EAGAIN and return;
         die "sysread failed: $!";
      } else {
         print "Data from child: [\n$data]\n";
      }
   });

   # after that a child watcher will be installed, where we wait
   # for the child to terminate. (AnyEvent installs a SIGCHLD handler
   # for us here).
   my $child_watcher = AnyEvent->child (pid => $pid, cb => sub {
      print "Child terminated\n";

      # when the child terminated we are finished, and the broadcast
      # method will let the call to 'wait' below return.
      $cvar->broadcast;
   });

   # here we start to wait for the 'broadcast' in the child
   # watcher. Under the hood AnyEvent starts one of the available
   # event loops for us (might be Glib, Event, EV, Tk or even it's
   # own pure Perl implementation via select(), depending on what is
   # available or loaded).
   $cvar->wait;
} else {
   # child:
   my $cnt = 0;

   while (1) {
      usleep ((int rand (500000)) + 100000);

      syswrite $chld, "iteration count: ".++$cnt."\n"
         or die "write not successful: $!";

      if ($cnt > 10) { exit 0 }
   }
}

Instead of calling the wait method there you could also start an event loop explicitly. For example if you would like to use the EV module you could use the loop and unloop functions instead of wait and broadcast:

   use EV;
   use AnyEvent;

   # ...

   my $child_watcher = AnyEvent->child (pid => $pid, cb => sub {
      print "Child terminated\n";
      EV::unloop;
   });

   EV::loop;

Some links of the modules noticed:

• AnyEvent at CPAN
• POE on CPAN
• EV module on CPAN - The C bindings to libev (a libevent replacement)

[d3xter]

Great article about great module. I think AnyEvent might be real standard for event driven applications.

I would like to see some benchmarks and see if multi-threaded application is faster that one-threaded with event loop. I already found some Java benchmark (http://www.theserverside.com/discussions/thread.tss?thread_id=26700) and saw that Linux with NPTL and one connection per one thread mode can be really faster than many connections per one thread mode.

[elmex]

The performance of AnyEvent depends on the event loop used. If you use EV as event loop it will most likely be VERY fast and scalable (as the author of EV/libev optimized it a lot, see also: libev vs. libevent benchmark). If you use Tk or Glib as event loop the performance might not be the best.

AnyEvent is after all just a very thin abstraction layer.

The performance benchmark of multi-threaded vs. one-threaded with event loop is mostly a comparison of apples to bottles of beer:
For example: If you just have two threads per socket: one reader and one writer, your performance will most likely suck a lot, especially when your connection count rises that design wont scale well. But you could have an event loop per thread, and each thread could handle a specified amount of connections, which could be a way better solution.

So threads and event loops are after all two very orthogonal concepts.

Also note that enabling threads when compiling perl5 will degrade the overall performance of that perl5 build.

[Anonymous Monk]

i like the idea behind AnyEvent, but would like to see more building blocks for performing common tasks, like POE has. i'm particularly interest in writing a web client and am not looking forward to creating the equivalent of POE::Component::Client::HTTP.

[Corion]

See Coro::LWP. The basic idea of Coro is to subvert non-Coro modules by making their sockets coro-enabled. I haven't tried it under Windows, but as 4.7 now has support for ActiveState and Strawberry Perl, it might even work there.

[elmex]

There are currently a wide range of AnyEvent utility modules in the work. Searching for AnyEvent on CPAN will show that there is AnyEvent::AIO for real asynchronous IO integrated with event loops. Also AnyEvent::BDB, AnyEvent::HTTPD and Net::IRC3 and Net::XMPP2 also have AnyEvent support. AnyEvent::Handle provides a more highlevel way of handling filehandles with AnyEvent, see this example:

use AnyEvent::Handle;
my $cv = AnyEvent->condvar;

my $ae_fh =
    AnyEvent::Handle->new (
       fh => \*STDIN,
       on_eof => sub { $cv->broadcast }
    );

$ae_fh->push_read_line (sub {
   my ($ae_fh, $line) = @_;
   print "Got line [$line]\n";

   $ae_fh->push_read (sub {
      my ($ae_fh) = @_;
      print "Got additional data:[\n".$ae_fh->rbuf."]\n";
      
      if ($ae_fh->rbuf =~ s/^.*\bend\b//s) {
         print "'end' detected, stopping program\n";
         $cv->broadcast;
         return 1;
      }
      
      return 0;
   });
});

$cv->wait;

(Note: See also the examples in the distribution of AnyEvent and it's POD docs.)

There are also utility functions for non-blocking socket creation planned for next releases of AnyEvent. And one day there will hopefully also a HTTP client module with AnyEvent support, perhaps someone would like to write it?

[Anonymous Monk]

The AnyEvent module family grows steadily - AnyEvent::HTTP seems to have hit CPAN just now, and while it isn't as advanced as LWP, it might do what you need. Besides, AnyEvent isn't exclusive as other such "frameworks" - you can mix POE components easily with anyevent modules and code.

[elmex]

There is a whole set of modules now in the AnyEvent suite:

• AnyEvent::DNS - fully asynchronous DNS resolution
• AnyEvent::BDB - truly asynchronous berkeley db access
• AnyEvent::AIO - truly asynchronous file and directrory I/O
• AnyEvent::DBI - asynchronous DBI access
• AnyEvent::HTTP - simple but non-blocking HTTP/HTTPS client
• AnyEvent::FastPing - quickly ping a large number of hosts

Modules that also implement protocols are:

• Net::IRC3 - An event system independend IRC protocol module
• Net::XMPP2 - An implementation of the XMPP Protocol

The great thing is, that you can combine all these protocols and modules in one application or module without threading and without having to choose a specific event loop implementation.

I wrote Net::IRC3 and Net::XMPP2, and was really happy I could shift the choice of the event loop to the actual application programmer, who uses my module, making it a lot more useful (This way even the POE or IO::Async world can benefit from my modules).