perl 5.36 and the for_list feature - a simple speed comparison

[swl]

Perl 5.36 has a new feature that iterates over multiple values at a time in a for loop (perldelta entry here). Any number of values can be specified, but the pairwise case is the focus here.

The for_list feature is a useful alternative to a while-each idiom over a hash when working with key-value pairs. It also allows one to natively work with arrays of key-value pairs without converting them to a hash (and thus losing values associated with duplicate keys).

Speed is one factor in the decision to updated code, so I figured I would see how fast the new feature is compared with while-each and some other approaches using List::Util functions. Some benchmark code is below (inside readmore tags), with results following.

Benchmark labels starting with a_ operate on array data while those with h_ operate on hash data.

I note that it is fairly well documented that the while-each idiom is best avoided because the hash iterator has global state and thus can cause action at a distance. In the benchmark code the hash is only accessed this way in one sub.

The summation of the hash values in the loops is merely something for the loop to do that would have relatively low overhead compared with the looping itself.

The hash_vals sub is done as a point of comparison. There is no reason why one would iterate over both keys and values when only the values are needed.

Code was run under Ubuntu 20.04 under WSL2 for Windows.

The main conclusion is that the new feature is faster than all the others when an array is used. It is faster than all of the hash approaches that use both keys and values. Profiling shows the cause of the slowdown when using List::Util::pairs is the dereference of the pair array, which is not surprising. Using the declared_refs feature does not help in this case.

Overall I quite like the new for_list feature. Aside from being faster than many of the alternatives, it is also much cleaner.

use 5.036;
use strict;
use warnings;
use List::Util qw /pairs pairvalues/;
use Benchmark qw /cmpthese/;

use experimental qw/for_list declared_refs/;
    
my @vals = (1..100000);
my %val_hash = @vals;

say lu_pairs();
say lu_pairs_refalias();
say lu_pairvals();
say for_list_a();
say for_list_h();
say while_each();
say hash_vals();
say hash_by_key();

cmpthese(-3, {
    a_pairs       => \&lu_pairs,
    a_pairs_alias => \&lu_pairs_refalias,
    a_pair_vals   => \&lu_pairvals,
    a_for_list    => \&for_list_a,
    h_for_list    => \&for_list_h,
    h_each        => \&while_each,
    h_vals        => \&hash_vals,
    h_by_key      => \&hash_by_key,
});

sub lu_pairs {
    my $i;
    for my $pair (pairs @vals) {
        $i += $pair->[1];
    }
    return $i;
}

sub lu_pairs_refalias {
    my $i;
    for my \@pair (pairs @vals) {
        $i += $pair[1];
    }
    return $i;
}

sub lu_pairvals {
    my $i;
    for my $val (pairvalues @vals) {
        $i += $val;
    }
    return $i;
}


sub for_list_a {
    my $i;
    for my ($key, $value) (@vals) {
        $i += $value;
    }
    return $i;
}

sub for_list_h {
    my $i;
    for my ($key, $value) (%val_hash) {
        $i += $value;
    }
    return $i;
}

sub while_each {
    my $i;
    while (my ($key, $value) = each %val_hash) {
        $i += $value;
    }
    return $i;
}

sub hash_vals {
    my $i;
    foreach my $value (values %val_hash) {
        $i += $value;
    }
    return $i;

}

sub hash_by_key {
    my $i;
    foreach my $key (keys %val_hash) {
        $i += $val_hash{$key};
    }
    return $i;

}

Results:

2500050000
2500050000
2500050000
2500050000
2500050000
2500050000
2500050000
2500050000
                Rate a_pairs_alias a_pairs h_by_key h_each h_for_list 
+h_vals a_pair_vals a_for_list
a_pairs_alias 87.9/s            --     -9%     -19%   -35%       -51% 
+  -82%        -83%       -89%
a_pairs       96.5/s           10%      --     -11%   -28%       -47% 
+  -80%        -81%       -88%
h_by_key       109/s           24%     13%       --   -19%       -40% 
+  -77%        -78%       -87%
h_each         135/s           53%     40%      24%     --       -25% 
+  -72%        -73%       -84%
h_for_list     181/s          106%     88%      66%    34%         -- 
+  -62%        -64%       -78%
h_vals         477/s          443%    394%     338%   254%       164% 
+    --         -6%       -42%
a_pair_vals    506/s          476%    424%     364%   275%       180% 
+    6%          --       -39%
a_for_list     827/s          841%    757%     659%   513%       357% 
+   73%         63%         --