Beefy Boxes and Bandwidth Generously Provided by pair Networks
Come for the quick hacks, stay for the epiphanies.

Re^3: Parrot, threads & fears for the future.

by Jenda (Abbot)
on Oct 23, 2006 at 13:10 UTC ( #580019=note: print w/replies, xml ) Need Help??

in reply to Re^2: Parrot, threads & fears for the future.
in thread Parrot, threads & fears for the future.

You can only transparently paralelize map{} if the executed block is side-effect-free. Perl is far too dynamic for the compiler/optimizer to safely check that and you might very well want to have side-effects in your map{}s so you'd have to have two map{}s, a paralellisable one and a garanteed-to-be-serial one. It's one thing to paralelize a purely functional, side-effect free language and to paralelize something that may do whatever it bloody wishes. Everything comes for a prize, even freedom.

  • Comment on Re^3: Parrot, threads & fears for the future.

Replies are listed 'Best First'.
Re^4: Parrot, threads & fears for the future.
by Corion (Patriarch) on Oct 23, 2006 at 13:11 UTC

    Sure - but I'm not talking about doing that for Perl5, but postulating it for Perl6, where such stuff is still possible.

Re^4: Parrot, threads & fears for the future.
by tphyahoo (Vicar) on Oct 23, 2006 at 15:10 UTC
Re^4: Parrot, threads & fears for the future.
by AK108 (Friar) on Oct 25, 2006 at 23:43 UTC
    What if there was a pragma for enabling parallelization? It could be use parallel, in the spirit of integer.
    my @list = ('aaaa' .. 'zzzz'); use parallel 'map'; # Just for maps my @ordlist = map ord, @list; # Defaults only # Probably something like any, all, map, grep, and maybe even sort use parallel; # Sort by last letter in parallel @list = sort {($a =~ /\w+(\w)/) <=> ($b =~ /\w+(\w)/)} @list; # This subroutine can operate in parallel use parallel 'foreach'; foo($_) foreach @list; no parallel; # disable everywhere else

      I think it would be safer to have a "parallel map" function instead. In either way the problem is that people would not normaly think neither about adding the pragma nor using the parallel map. Plus there is another problem. The question is not just whether perl may execute the map in parallel, but also in how many chunks. You would of course not want to create a thread for each and every element of the array/list especially if it's huge and the operation is simple. The number of threads you will want to use depends on the complexity and nature of the action you want to do with the elements - you will want more threads if you will spend most time waiting for a network response, you will want less if it's just computation ... So without knowing what's the operation, the implementation of the mapp will not know how to parallelize the operation to achieve good performance. So even though mapp is a nice interface to parallelization, you will want to be able to give it a few more hints to ensure it works well.

        I think that a compile-time switch that sets the maximum number of threads would be a good solution. This way, you could compile Perl with support for 8 threads on an 8 core processor, and have it be used completely.

        Each thread could get one item to start with, and as it finishes, it could get an additional item. If it's just running a built-in function on the list, then each thread could be given 1/n of the list (where n is the number of threads). Otherwise, Perl could time the items and detect about how many would be good to send based on the time.

        Perl would still need to get data back from the threads to give back to the core, so sending data shouldn't be a problem either. But I've never used threads, so I may be off.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://580019]
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others avoiding work at the Monastery: (3)
As of 2023-12-04 13:49 GMT
Find Nodes?
    Voting Booth?
    What's your preferred 'use VERSION' for new CPAN modules in 2023?

    Results (25 votes). Check out past polls.