http://www.perlmonks.org?node_id=939047

Gary Yang has asked for the wisdom of the Perl Monks concerning the following question:

Hi BrowserUk, use Async is a good solution. Since return value from paralleled subs is hash reference, I use the AsyncData use Async; $proc = AsyncData->new(sub {...}); But, I got errors. It says, "Magic number checking on storable string failed at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/Storable.pm line 416, at /usr/lib/perl5/site_perl/5.8.8/Async.pm line 128". I google searched. But, did not find any good explanation and solution. Do you have any idea? Gary

Replies are listed 'Best First'.
Re: Parallel Modules ?
by Corion (Patriarch) on Nov 20, 2011 at 09:10 UTC

    You don't tell us which modules you looked at. I would look at threads.

Re: Parallel Modules ?
by BrowserUk (Patriarch) on Nov 20, 2011 at 09:28 UTC
    Any parallel module can handle this case?

    Yes, threads. Ostensibly it should be as simple as:

    use threads; use Your::Module; my @AoH = map $_->join, map{ async( \&{ "Your::Module::$_" } ) } qw[ sub1 sub2 sub3 sub4 sub5 sub5 sub7 sub8 sub9 sub10 ]; ## do something with the 10 hashrefs here

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      Hi BrowserUk,

      use Async is a good solution. The return value from paralleled subs is hash reference, I use the AsyncData

      use Async;
      $proc = AsyncData->new(sub {...});

      But, I got errors. It says, "Magic number checking on storable string failed at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/Storable.pm line 416, at /usr/lib/perl5/site_perl/5.8.8/Async.pm line 128". I google searched. But, did not find any good explanation and solution. Do you have any idea?

      Gary

        use Async is a good solution.

        Err, no. Use Async is a terrible solution. look at the errors it is giving you.

        I suggested use threads, which would be a good solution. For a start you wouldn't be having the errors you now have,


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Parallel Modules ?
by Anonymous Monk on Nov 20, 2011 at 09:16 UTC
Re: Parallel Modules ?
by sundialsvc4 (Abbot) on Nov 20, 2011 at 13:36 UTC

    All right ... where do these subroutines fetch their data from?   Have you thoroughly established that every component of that entire system possesses the capability to deliver 10 times as much data per second as they presently do?

    Have you taken a murderously-close look at the algorithm, to verify beyond all reproach that, each time they execute, they are fetching precisely what they must fetch and nothing more than is needed to solve the problem, and that they are never fetching anything that they have already requested before?

    I am cordially saying that because, way too often, I have watched efforts to “parallelize” something produce a final product that is noticeably(!) slower than its predecessor.   Very fine parallelization engines do of course exist and several of these have already been cited.   But I suggest that you cross-examine your algorithm first.