Re: Unpacking and converting

Your code above never does anything with the modified $item--ie. it never pushes it into @newdata, which means that your loop does exactly nothing, but slowly. However, looking past that...

This is weird but shifting items one by one into new array seems to be a lot faster than iterating over the array with canonical for (@list).

Update: Ignore. The Benchmark is broken!

Frankly, I thought you were talking through the top of your hat when you said that, until I benchmarked it. And, despite my spending a while trying to see a flaw in the benchmark, you seem to be right. Not only am I surprised that it seems to be true, but I'm utterly staggered by the difference in performance. And at an utter loss to explain why it should be the case.

our  @a = @b = @c = '0001' .. '1000';;
cmpthese -1,{
    a => q[ $_ += 0 for @a; ], 
    b => q[ my @new; push @new, $_ + 0 while defined( $_ = shift @b ) 
+], 
    c => q[ $c[ $_ ] += 0 for 0 .. $#c; ]
};;
       Rate      c      a      b
c    6220/s     --   -37%  -100%
a    9893/s    59%     --  -100%
b 4562313/s 73247% 46016%     --
[download]

And the bigger the array, the more extraordinary the difference becomes:

our @a = @b = @c = @d = '00001' .. '10000';;

cmpthese -1,{
    a => q[ $_ += 0 for @a;], 
    b => q[ my @new; push @new, $_ + 0 while defined( $_ = shift @b ) 
+], 
    c => q[ $c[ $_ ] += 0 for 0 .. $#c; ], 
    d => q[ my @new = map $_ += 0, @d ] 
};;
       Rate        d        c        a        b
d     258/s       --     -58%     -72%    -100%
c     615/s     138%       --     -34%    -100%
a     932/s     261%      52%       --    -100%
b 4651085/s 1800042%  756579%  499189%       --
[download]

There is something amiss here, but if it is the benchhmark I cannot see it.

And if not, I'm loathed to explain why creating a new array by pushing them one at a time whilst destroying the old one, would be so much faster than iterating over the original in place.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

Comment on Re: Unpacking and converting Select or Download Code

Replies are listed 'Best First'.
Re^2: Unpacking and converting by andal (Hermit) on Feb 16, 2011 at 08:02 UTC
I recommend to try `cmpthese -1,{ c => q[ my @c = '0001' .. '1000'; $c[ $_ ] += 0 for 0 .. $#c; ], a => q[ my @a = '0001' .. '1000'; $_ += 0 for @a; ], b => q[ my @b = '0001' .. '1000'; my @new; push @new, $_ + 0 while + defined( $_ = shift @b ) ], };` [download] Then the results are appropriate `Rate b c a b 999/s -- -22% -28% c 1279/s 28% -- -7% a 1382/s 38% 8% --` [download]	[reply] [d/l] [select]
Re^3: Unpacking and converting by dwalin (Monk) on Feb 16, 2011 at 09:14 UTC
I think in this case the array creation and population makes significant influence on results. Consider this: `cmpthese -1,{ d => q[ my @d = '0001' .. '1000'; ], c => q[ my @c = '0001' .. '1000'; $c[ $_ ] += 0 for 0 .. $#c; ], a => q[ my @a = '0001' .. '1000'; $_ += 0 for @a; ], b => q[ my @b = '0001' .. '1000'; my @new; push @new, $_ + 0 while + defined( $_ = shift @b ) ], };` [download] Results speak for themselves: `Rate b c a d b 1267/s -- -29% -37% -63% c 1794/s 42% -- -10% -47% a 2000/s 58% 11% -- -41% d 3413/s 169% 90% 71% --` [download] Regards, Alex.	[reply] [d/l] [select]
Re^4: Unpacking and converting by andal (Hermit) on Feb 16, 2011 at 10:39 UTC
I think in this case the array creation and population makes significant influence on results. Consider this You missed the point. The shifting of elements from the array makes it empty after first iteration. But Benchmark makes thousands of iteration within 1 second. All of those iterations work with empty array, this explains why this approach appears to be "faster".	[reply]
Re^5: Unpacking and converting by BrowserUk (Patriarch) on Feb 16, 2011 at 15:51 UTC
Re^5: Unpacking and converting by dwalin (Monk) on Feb 16, 2011 at 14:01 UTC
Re^4: Unpacking and converting by Anonyrnous Monk (Hermit) on Feb 16, 2011 at 10:26 UTC
I think in this case the array creation and population makes significant influence on results. Sure, but in the original case the benchmark is flawed: the test code is being executed many times, and after the first execution, the array `@b` has been emptied... (unlike the arrays for the other cases) Code that modifies test input is notoriously tricky to benchmark.	[reply] [d/l]
Re^2: Unpacking and converting by dwalin (Monk) on Feb 16, 2011 at 08:47 UTC
Your code above never does anything Oopsie, thanks for pointing out this braino. I ran my benchmarks against working code, you can be sure of that. :) Considering array processing, well I'm not a perl guru so I can only guess; and my guess is that in perl, arrays are not really arrays internally but linked lists with external indices. That would at least explain why shifting and tossing is so blazing fast - there's very little expense in following a pointer, after all - but it cannot explain why index lookup is so shocking slow. And why for (@list) is 60% faster than index lookup but otherwise slow is beyond me. This was exactly the reason I asked my question, since my first impression was that I'm doing something wrong but couldn't pinpoint what exactly. Regards, Alex.	[reply]


No such thing as a small change
	PerlMonks