Parallel::ForkManager + MCE::Shared demonstrationby marioroy (Deacon)
|on Apr 24, 2017 at 02:26 UTC||Need Help??|
In preparation for the upcoming MCE 1.828 and MCE::Shared 1.825 releases, am testing various modules with MCE::Shared. One of which is Parallel::ForkManager.
Discipulus introduced me to zentara recently. zentara am pleased to meet you. Discipulus, imho, folks may choose any parallel module of their liking. It doesn't need to be MCE and the reason for this thread. I like Parallel::ForkManager too.
Some time back, zentara wrote a Parallel::ForkManager + IPC::Shareable demonstration. Fast forward 2.5 years and here's another way. MCE::Shared provides users of Parallel::ForkManager with threads-like sharing capabilities. Below is zentara's example updated with MCE::Shared bits.
Not to worry, MCE::Shared performs reasonably well.
The following are recommended modules for MCE::Shared.
Q. Why is Sereal beneficial?
A. The main reason is for extra performance. To ensure minimum memory consumption, there's no reason to load the Storable module if Sereal is available in Perl. This is handled transparently.
Q. Why is IO::FDPass beneficial?
A. Being able to construct a shared condvar, handle, or queue while the shared-manager is running greatly adds to the ease-of-use. These involve handles behind the scene. Basically, am able to send the relevant fd descriptors to the shared-manager. Without FDPass, one must be careful to construct Condvar, Handle, and Queue first before other shared objects and later starting the shared-manager manually. Note: MCE and MCE::Hobo starts the shared-manager if not already started.
Q. What is MCE::Shared->init all about?
A. For MCE, MCE::Hobo, and threads (via CLONE), MCE::Shared->init() is called automatically. It assigns the worker 1 of 12 data channels for use during IPC. Calling init is totally optional. If the worker is sending data one time, probably not necessary. On the other hand, if doing lots of IPC, then yes worth it.