in reply to Google's MapReduce

I don't know in which programming language MapReduce was written
On page 13 of the PDF you linked to, I find:
#include "mapreduce/mapreduce.h"
So I'm guessing one of the C variants.

could you write the basic functions in Perl?

Well, yes. Nothing about Perl limits your ability to write MapReduce: you have robust I/O, complex computational ability, networking capabilities, and comprehensive tools to manipulate data structures.

I think the real question, given the low-level and high-performance nature of some of the MapReduce requirements, is "would a Perl version of this be fast enough?"

Perl's runtime compiler and VM are pretty damn good; but, they still can't (and shouldn't try to) compete with C for low-level system operations. It is those kinds of operations that make MapReduce efficient for the volumes of data Google deals with. Of course, if you needn't process terabytes of data in one go, Perl might perform acceptably, and you'd have all of Perl's advantages in exchange for waiting a bit.

I think it's a worthy project to implement something like this in Perl. At the very least, it would be an interesting benchmark opportunity. Perl is a great language for prototyping, so even if it's too slow for practical use, having an open Perl implementation of MapReduce provides a great reference for porting to "faster" languages.

radiantmatrix
require General::Disclaimer;
"Users are evil. All users are evil. Do not trust them. Perl specifically offers the -T switch because it knows users are evil." - japhy

Replies are listed 'Best First'.
Re^2: Google's MapReduce
by Fletch (Chancellor) on Oct 27, 2004 at 16:38 UTC

    Erm, read harder guys. :) Section 2.1 third paragraph.

    . . . The user's code is linked together with the MapReduce library (implemented in C++). . . .

      Read?! But that would mean I wasn't exercising the Noble Virtues of laziness (why read all of something?), impatience (I don't have time to read 13 pages of detail, just give me the code!), and hubris (besides, I already know everything.). ;-P.

      j/k, of course...

      radiantmatrix
      require General::Disclaimer;
      "Users are evil. All users are evil. Do not trust them. Perl specifically offers the -T switch because it knows users are evil." - japhy