The Benchmark.pm idea could be done if the code were submitted to run on a particular computer, if someone is crazy enough to risk something like that ;-)
Seriously, it could be done in the sense that everyone is able to confirm the results on his own machine with its specific setup. The format of submission would be a function that can be run using Benchmark so that the "competition" could be compared by just pasting the code and running the benchmarks.
I agree that you won't see a clear winner, but it would give those of us interested in raw performance a number of alternative implementations and hence an opportunity to learn. It's all about taking part, not about winning ;-)
Just my 2 cents, -gjb-
It would be possible to get a machine/OS/etc. independent comparison by using some "gauge code", ie. a standard piece of code. The benchmark for the Perl Polo code would then be calculated proportional to the time it takes the "gauge code" to execute.