Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Prima + MCE::Hobo demonstration

by marioroy (Deacon)
on Apr 29, 2017 at 07:03 UTC ( #1189155=CUFP: print w/replies, xml ) Need Help??

Respected Monks,

Starting with MCE 1.828 and MCE::Shared 1.825, running MCE with Prima is possible. The following is based on the Tk + MCE::Hobo demonstration (2nd example in the post). I tested Prima + MCE on Linux using CentOS 7.x and Windows with Strawberry Perl 5.22.x.

use strict; use warnings; use MCE::Hobo; use MCE::Shared; use Prima qw( Application Buttons Label ); my $que = MCE::Shared->queue(); my $msg = MCE::Shared->scalar("Start"); my $hobo = MCE::Hobo->create("bg_task"); my $wm = Prima::MainWindow->new( size => [ 250, 200 ], text => 'Hello world!', onDestroy => \&quit ); my $lbl1 = Prima::Label->create( owner => $wm, size => [ 220, 50 ], text => 'Prima + MCE Demo', alignment => ta::Center, valignment => ta::Middle, pack => {} ); my $btn1 = Prima::Button->new( owner => $wm, size => [ 120, 50 ], text => $msg->get, onClick => \&fun, pack => {} ); my $btn2 = Prima::Button->new( owner => $wm, size => [ 120, 50 ], text => 'Quit', onClick => sub { $::application->close }, pack => {} ); my $timer = Prima::Timer->create( timeout => 100, onTick => sub { my $new_text = $msg->get; if ($new_text ne $btn1->text) { $btn1->set( text => $msg->get ); } } ); $timer->start; run Prima; sub fun { $que->enqueue("some event"); return; } sub quit { $timer->stop; $hobo->exit->join; $::application->close; } sub bg_task { while ( my $event = $que->dequeue ) { $msg->set("Step One"); sleep 1; $msg->set("Step Two"); sleep 1; $msg->set("Step Three"); } }

So that the quit function isn't called twice, I'm only calling $::application->close inside the Quit handler. That closes the window which then triggers the MainWindow's onDestroy handler.

Regards, Mario

Update: Updated the timer handler. Thanks zentara.

Update: On the Mac, mouse clicks between windows is greatly improved by setting an option in XQuartz -> Preferences -> Windows -> Click-through Inactive Windows. When enabled, clicking on an inactive window will cause that mouse click to pass through to that window in addition to activating it.

Update: To improve performance on the Mac, set XQuartz -> Preferences -> Output -> Colors to Thousands. Then relaunch XQuartz for the option to take effect. Prima for the most part runs very well. Thank you, Dmitry.

Replies are listed 'Best First'.
Re: Prima + MCE::Hobo demonstration
by zentara (Archbishop) on Apr 29, 2017 at 11:57 UTC
    Hi marioroy, I'm glad to see someone use Prima, as it is an under-appreciated toolkit. I did notice one unusual aspect of running your code. On my linux system, slackware, I notice a flickering in the $btn1 text as the background process runs. It almost looks like a "feature", indicating which step is active. :-)

    Here is an unoptimized hack to fix the problem in the timer. It looks like this is a perfect opportunity to use a "state" variable in the timer, but this is just a brute force method, which makes it clear what is happening.

    my $text = ''; my $timer = Prima::Timer->create( timeout => 100, onTick => sub { my $new_text = $msg->get; if ($new_text ne $text){ $btn1->set( text => $new_text ); $text = $new_text; } } );

    I'm not really a human, but I play one on earth. ..... an animated JAPH

      Hi zentara. That's nice++. Prima is new to me. I tried calling $btn1->text to obtain the string and that works too.

      my $timer = Prima::Timer->create( timeout => 100, onTick => sub { my $new_text = $msg->get; if ($new_text ne $btn1->text) { $btn1->set( text => $msg->get ); } } );
Re: Prima + MCE::Hobo demonstration
by Discipulus (Prior) on Apr 29, 2017 at 13:37 UTC
    Thanks marioroy for this nth wonderful example!

    I had not the time to investigate Prima before: when I checked no one at PM was using it. Now maybe the time to give it a serious check out is probably arrived.

    The example runs fine and is a perfect skeleton code to work on!

    I'm now at home on a machine where MCE::Hobo was not present; i've seen a stability canary chirping in my screen! thanks also for your care and attention you put in everything you release.

    *** *** The stability canary says: chirp! chirp! (it seems to be quite hap +py) ***

    I succesfully tested with strawberry 5.14.2 win7 Prima 1.51 MCE::Hobo 1.825

    L*

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.

      Hi Discipulus. Interesting... IO::FDPass along with Sereal::Decoder and Sereal::Encoder are listed in the recommends section inside the Makefile.PL for MCE::Shared. It happens to be that Canary::Stability is a dependency for IO::FDPass.

      Folks running a recent Strawberry Perl binary will be happy to know that there's a ppm package already made. Just realized now that 1.46 was released about a year ago. Prima is at 1.51, currently.

      C:\perl-5.22.2.1>ppm PPM interactive shell (11.11_03) - type 'help' for available commands. PPM> install Prima Version 1.46 of 'Prima' is already installed. Remove it, or use 'verify --upgrade Prima'. PPM> exit Quit! C:\perl-5.22.2.1>

      I received a tip from Discipulus via email that Prima installs fine on Strawberry Perl with the following command. The --skiptest argument is necessary to overcome a failing mouse simulation test.

      cpanp i Prima --skiptest

      Thank you, Discipulus.

Re: Prima + MCE::Hobo demonstration
by marioroy (Deacon) on May 01, 2017 at 18:45 UTC

    Respected Monks,

    An upcoming release (MCE::Shared 1.826) continues forward with regards to decreasing memory consumption. In doing so, I took out all the code pertaining to condvar, handle, and queue from inside MCE/Shared/Server.pm and re-factored the bits into MCE/Shared/Condvar.pm, Handle.pm, and Queue.pm respectively. Furthermore, I enabled AUTOLOAD inside MCE/Shared.pm and moved all the sugar-type functions inside it.

    For testing, the following script loads the relevant modules and constructs a shared queue.

    use strict; use warnings; use Prima qw( Application Buttons Label ); #use Parallel::ForkManager; #use MCE; #use MCE::Loop; use MCE::Hobo; use MCE::Shared; my $que = MCE::Shared->queue(); #busy loop, review top at OS level, press Ctrl-C to exit for ( 1 .. 2e9 ) { 1 }

    Results reported by top (resident memory size):

    20.7 Mib Prima + Parallel::ForkManager + MCE::Shared->queue v1.824 20.9 Mib Prima + Parallel::ForkManager + MCE::Shared->queue v1.825 20.7 Mib Prima + Parallel::ForkManager + MCE::Shared->queue v1.826 21.2 Mib Prima + MCE::Loop + MCE::Shared->queue v1.824 20.4 Mib Prima + MCE::Loop + MCE::Shared->queue v1.825 19.3 Mib Prima + MCE::Loop + MCE::Shared->queue v1.826 21.1 Mib Prima + MCE Core API + MCE::Shared->queue v1.824 20.4 Mib Prima + MCE Core API + MCE::Shared->queue v1.825 19.1 Mib Prima + MCE Core API + MCE::Shared->queue v1.826 18.8 Mib Prima + MCE::Hobo + MCE::Shared->queue v1.824 19.1 Mib Prima + MCE::Hobo + MCE::Shared->queue v1.825 18.4 Mib Prima + MCE::Hobo + MCE::Shared->queue v1.826 14.7 Mib Prima (only) 8.4 Mib Parallel::ForkManager + MCE::Shared v1.826 7.1 Mib MCE::Loop + MCE::Shared v1.826 6.8 Mib MCE Core API + MCE::Shared v1.826 6.1 Mib MCE::Hobo + MCE::Shared v1.826

    Q. Why is memory footprint important for MCE::Hobo?

    A. I have an add-on module using MCE::Hobo to send-recv data, similar to Parallel::Fork::BossWorkerAsync. Someone has requested this for a long time and decided okay will do it. But first, wanted to reduce memory consumption.

    MCE::Shared 1.826 will be released sometimes this week.

    Regards, Mario

      Greetings,

      I reached my goal in reaching below 2M per worker. Testing involved running 200 workers with and without threads for the MCE demonstration and without threads for MCE::Hobo.

      MCE::Flow

      use strict; use warnings; use threads; # comment out for child processes use MCE::Flow; use MCE::Shared; use Time::HiRes 'time'; my $n = MCE::Shared->scalar(0); my $s = time; sub task { # do something, plenty time to see top my $v; $v = $n->incr() for 1 .. 933; } mce_flow { max_workers => 200 }, \&task; MCE::Flow::finish; printf "duration: %0.3f\n", time - $s; print $n->get(), "\n"; __END__ main process, shared-manager process, and 200 workers # with threads: e.g. use threads 498M MCE 1.827, MCE::Shared 1.824 437M MCE 1.828, MCE::Shared 1.825 <- big reduction 399M MCE 1.829, MCE::Shared 1.826 <- more reduction # without threads MCE 1.829, MCE::Shared 1.826: ~ 1744K per worker

      MCE::Hobo

      use strict; use warnings; use MCE::Hobo; use MCE::Shared; use Time::HiRes 'time'; my $n = MCE::Shared->scalar(0); my $s = time; sub task { # do something, plenty time to see top my $v; $v = $n->incr() for 1 .. 933; } MCE::Hobo->create(\&task) for 1 .. 200; MCE::Hobo->waitall; printf "duration: %0.3f\n", time - $s; print $n->get(), "\n"; __END__ main process, shared-manager process, and 200 workers MCE::Shared 1.826: ~ 1356K per worker

      Well, I never imagined for MCE::Hobo to consume less than 1.4M per worker, let alone MCE::Flow running below 2M. MCE 1.829 and MCE::Shared 1.826 will be released later this week.

      Regards, Mario

        For completeness, here's the code using Parallel::ForkManager. Calling MCE::Shared->init() is important to spread out IPC across 12 data channels. Not doing so, means all 200 workers share a single data channel and take much longer to run. Init is called automatically for MCE, MCE::Hobo and threads.

        Parallel::ForkManager

        use strict; use warnings; use Parallel::ForkManager; use MCE::Shared; use Time::HiRes 'time'; my $n = MCE::Shared->scalar(0); my $s = time; my $fork_manager = new Parallel::ForkManager(200); $fork_manager->set_waitpid_blocking_sleep(0); foreach my $child ( 1 .. 200 ) { my $pid = $fork_manager->start($child) and next; # init optionally takes an integer value MCE::Shared->init($child); my $v; $v = $n->incr() for 1 .. 933; $fork_manager->finish($child); } $fork_manager->wait_all_children; printf "duration: %0.3f\n", time - $s; print $n->get(), "\n"; __END__ main process, shared-manager process, and 200 workers Parallel::ForkManager: ~ 1200K per worker

        It's great knowing that MCE::Hobo isn't far behind regarding memory consumption. What I've learn is that simply loading modules doesn't tell the whole story. One must capture memory consumption while running for a better picture. :)

        Regards, Mario

        MCE 1.829 and MCE::Shared 1.826 have been released. These include the updates for reducing memory consumption, especially for MCE::Shared::Server.

Re: Prima + MCE::Hobo demonstration
by marioroy (Deacon) on May 02, 2017 at 21:01 UTC

    Compiling Prima on Mac OS X.

    These days, OS X doesn't ship with X11. Fortunately, XQuartz is available from https://www.xquartz.org. That provides the Mac with X11 support and ships with freetype2 and PNG libraries including header files. XQuartz installs under /opt/X11.

    Below, one may need to run with configure --prefix=/opt/X11 when building the GIF and JPEG libraries. For unknown reasons, Prima does not check /usr/local on my system.

    GIF: https://sourceforge.net/projects/giflib/files/

    tar xjf giflib-5.1.4.tar.bz2 cd giflib-5.1.4 ./configure --prefix=/opt/X11 make sudo make install

    JPEG: http://www.ijg.org/files/

    tar xf jpegsrc.v9b.tar.gz cd jpeg-9b ./configure --prefix=/opt/X11 make sudo make install

    Prima: https://metacpan.org/pod/Prima

    tar xf Prima-1.51.tar.gz cd Prima-1.51 PKG_CONFIG_PATH=/opt/X11/lib/pkgconfig perl Makefile.PL X11BASE=/opt/X +11 WITH_GTK2=0 make sudo make install

    Unfortunately, the fonts aren't as nice as running a Tk app. It did find freetype2 when building Prima. For help, I sent the author an email about why fonts have a staircase look.

    Regards, Mario

    Update: The author confirm Prima selecting bitmap (core X11 fonts) over freetypes.

      Dmitry committed fixes into master to ensure freetype fonts are selected over bitmap (core X11 fonts).

      On the Mac, the Makefile.PL now checks for pkgconfig residing under /opt/X11/lib. Not having to set PKG_CONFIG_PATH nor specify the path for X11BASE is awesome.

      The following applies to master and should work for Prima 1.52 and later releases.

      unzip Prima-master.zip cd Prima-master perl Makefile.PL WITH_GTK2=0 make sudo make install

      Thank you, Dmitry. Prima now looks great on the Mac.

        Hi Mario,

        On my Mac setup, you actually don't even need to say WITH_GTK2=0.

        /dk

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: CUFP [id://1189155]
Approved by haukex
Front-paged by Corion
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (3)
As of 2017-06-24 21:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    How many monitors do you use while coding?















    Results (562 votes). Check out past polls.