Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re^2: using parallel processing to concatenate a string, where order of concatenation doesn't matter

by tphyahoo (Vicar)
on Oct 18, 2006 at 14:26 UTC ( #579085=note: print w/ replies, xml ) Need Help??


in reply to Re: using parallel processing to concatenate a string, where order of concatenation doesn't matter
in thread using parallel processing to concatenate a string, where order of concatenation doesn't matter

Thanks, that's intriguing, but my perl doesn't seem to like that code.

hartman@ds0207:~/pfmArena> perl test2.pl Bogus Parallel::Queue: "fork" and "thread" are exclusive at test2.pl l +ine 17 hartman@ds0207:~/pfmArena> perl -e '@ARGV=("test2.pl"); while (<>) {di +e $_ if $. == 17}' my $mgr = Parallel::Queue->construct( 'thread' ); hartman@ds0207:~/pfmArena> hartman@ds0207:~/pfmArena> perl -V | grep -i thread osname=linux, osvers=2.6.16, archname=i586-linux-thread-multi config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr -Dinstallusr +binperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Duseshrplib=tr +ue -Doptimize=-O2 -march=i586 -mtune=i686 -fmessage-length=0 -Wall -D +_FORTIFY_SOURCE=2 -g -Wall -pipe' usethreads=define use5005threads=undef useithreads=define usemulti +plicity=define cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS +-DDEBUGGING -fno-strict-aliasing -pipe -Wdeclaration-after-statement +-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGI +NG -fno-strict-aliasing -pipe -Wdeclaration-after-statement' libs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E - +Wl,-rpath,/usr/lib/perl5/5.8.8/i586-linux-thread-multi/CORE' PERL_MALLOC_WRAP THREADS_HAVE_PIDS USE_ITHREAD +S /usr/lib/perl5/5.8.8/i586-linux-thread-multi /usr/lib/perl5/site_perl/5.8.8/i586-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.8/i586-linux-thread-multi hartman@ds0207:~/pfmArena> perl -V | grep -i ithread usethreads=define use5005threads=undef useithreads=define usemulti +plicity=define PERL_MALLOC_WRAP THREADS_HAVE_PIDS USE_ITHREAD +S hartman@ds0207:~/pfmArena>
Any ideas?


Comment on Re^2: using parallel processing to concatenate a string, where order of concatenation doesn't matter
Download Code
Re^3: using parallel processing to concatenate a string, where order of concatenation doesn't matter
by diotalevi (Canon) on Oct 18, 2006 at 15:52 UTC

    Tell it to use threading by default: use Parallel::Queue 'thread';.

    ⠤⠤ ⠙⠊⠕⠞⠁⠇⠑⠧⠊

      Can't... get it... to work...

      I tried as you suggested...

      hartman@ds0207:~/Desktop/misc/idealoSubversionArena/parallelListProces +sing> ./testpq.pl Can't locate object method "new" via package "Threads" (perhaps you fo +rgot to load "Threads"?) at /usr/lib/perl5/site_perl/5.8.8/Parallel/Q +ueue.pm line 197. hartman@ds0207:~/Desktop/misc/idealoSubversionArena/parallelListProces +sing> head testpq.pl #!/usr/bin/perl use threads::shared; use Parallel::Queue 'thread'; print concatenate_parallel( ['a' .. 'z'], 4 ) . "\n"; sub concatenate_parallel { my $result :shared; my @input = map { my $string = $_;

      Also tried other permutations with spelling variations, but I feel like I'm stumblinb around in the dark. Parallel::Queue indicates that forking works, but threading is "still in progress".

      Are you able to run this code?

      I am really keen to make progress on this, because I am hoping to use this as a basis for threadedreduce which would serve as an abstraction layer for all sorts of list processing that could potentially be speeded up by using threading, but without changing anything in the "meat" of the program (which would be a function builder that uses threadedreduce or nonthreadedreduce, each having the same effect).

      See

      Could there be ThreadedMapReduce (and/or ForkedMapReduce) instead of DistributedMapReduce?

      and my followup, with threadedreduce stub code, at Re^2: Could there be a ThreadedMapReduce (instead of DistributedMapReduce)?

        Well... take out your debugger, look at the source of the module and make it go. Simple.

        ⠤⠤ ⠙⠊⠕⠞⠁⠇⠑⠧⠊

        Fix the module as follows:

        • Line 197: Change Threads->new to Thread->new.

        • Line 292: Insert require Thread; before require Thread::Semaphore;.

        These fixes have not been tested, just like the module itself:

        Threads are unstested. Forks work; threads are still in progress. The code may work but I have not tested it yet.

        Update: Added spelling fix.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://579085]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (8)
As of 2014-11-27 03:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (178 votes), past polls