Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Unexplained benchmark when using chop vs. chomp (or neither)

by Cristoforo (Curate)
on Mar 08, 2018 at 01:36 UTC ( #1210495=perlquestion: print w/replies, xml ) Need Help??
Cristoforo has asked for the wisdom of the Perl Monks concerning the following question:

Solved: See post Re: Unexplained benchmark when using chop vs. chomp (or neither). Thanks to everyone who helped trying to solve this question. And davido avoided the problem by making a copy of the array to operate upon.

I don't understand the different results I am getting when doing a benchmark on 4 different code snippets.

When using chop in the code, the benchmark runs extremely fast. But, when using chomp or neither chop or chomp, the the particular code runs the slowest in the benchmark.

Any ideas?

#!/usr/bin/perl use strict; use warnings; use List::Util qw/first reduce/; use Benchmark qw/ cmpthese /; my $file = 'bench_reduce.txt'; open my $fh, '<', $file or die $!; my @data = <$fh>; seek $fh, 0, 0; my @chunks; { local $/ = ""; @chunks = <$fh>; } my $cache = sub { my %c; # cache my $max = reduce {($c{$a}//=f_32($a)) > ($c{$b}//=f_32($b)) ? $a : + $b} @chunks; ($max) = $max =~ /\A(.+)/; #print $max; }; my $reduce = sub { my $max = reduce {f_32($a) > f_32($b) ? $a : $b} @chunks; ($max) = $max =~ /\A(.+)/; #print $max; }; sub f_32 { $_[0] =~ /^32 (\S+)/m; return $1; } my $tshiono = sub { my ($maxline, $firstline); my $max = 0; my $target = 32; for (@data) { next unless /\S/; chop; # also tested with chomp and with neither chop nor chomp if (/^\d+:/) { # treat the 1st line $firstline = $_; # keep the 1st line } elsif (/^(\d+) +([\d.]+)/) { # treat other lines in the blo +ck if ($1 == $target && $2 > $max) { $max = $2; # update the highest value $maxline = $firstline; # and the 1st line of the bloc +k } } } }; my $_7stud = sub { my %results; for my $block (@chunks) { my @lines = split /\n/, $block; my $line32 = first {/^32/} @lines; (undef, my $ranking_num) = split ' ', $line32; $results{$ranking_num} = $lines[0]; } my @sorted_nums = sort{$a <=> $b} keys %results; my $biggest_num = $sorted_nums[-1]; }; cmpthese (-1, { w_o_cache => $reduce, w_cache => $cache, tshiono => $tshiono, '7stud' => $_7stud }); __END__ C:\Old_Data\perlp>perl test2.pl (with chop tshiono) Rate 7stud w_o_cache w_cache tshiono 7stud 200/s -- -37% -45% -92% w_o_cache 320/s 60% -- -12% -87% w_cache 364/s 82% 14% -- -86% tshiono 2557/s 1180% 700% 602% -- C:\Old_Data\perlp>perl test2.pl (with chomp tshiono) Rate tshiono 7stud w_o_cache w_cache tshiono 160/s -- -19% -50% -57% 7stud 197/s 23% -- -38% -47% w_o_cache 318/s 98% 61% -- -15% w_cache 373/s 133% 90% 17% -- C:\Old_Data\perlp>perl test2.pl (w/o chop or chomp tshiono) Rate tshiono 7stud w_o_cache w_cache tshiono 177/s -- -13% -45% -52% 7stud 202/s 15% -- -37% -45% w_o_cache 321/s 82% 59% -- -13% w_cache 370/s 109% 83% 15% --
The file I ran it against was (1000 chunks - 5000 lines):
1: 1.0892 4.4110 33 17.46216 32 32.64771 34 112.64038 2: 9.6591 9.9243 33 116.30859 32 38.09204 34 117.57813 3: 1.6385 3.0847 33 45.08057 32 3.52173 34 145.28809 4: 7.0053 9.0564 33 24.13330 32 169.46411 34 92.74902 5: 9.9045 4.8688 33 130.81055 32 4.88892 34 54.73022 6: 0.1675 1.0004 33 11.64551 32 124.31641 34 186.35864 7: 5.5832 8.3316 33 113.65356 32 23.00415 34 85.02808 8: 5.2414 9.9789 33 37.54883 32 102.96021 34 30.73730 9: 9.4675 2.2897 33 105.28564 32 63.26904 34 11.12671 10: 5.4352 3.6444 33 68.45703 32 140.71655 34 77.05078 11: 5.9378 0.5676 33 70.95947 32 8.36182 34 162.20703 12: 7.9144 1.4954 33 37.33521 32 42.61475 34 160.79102 13: 2.7045 9.0518 33 28.79028 32 152.38647 34 23.10791 14: 3.0066 9.0158 33 89.53857 32 161.81030 34 17.35840 15: 5.8575 3.3990 33 16.30249 32 14.50806 34 53.58887 16: 2.2849 0.5002 33 189.22729 32 52.99072 34 103.76587 17: 9.3036 2.6373 33 159.46655 32 76.39160 34 42.39502 18: 0.4990 8.1360 33 81.29883 32 65.38086 34 183.02612 19: 2.6181 9.0866 33 68.85986 32 19.53735 34 117.96875 20: 5.5716 7.2162 33 67.04102 32 151.03760 34 93.67065 21: 6.4343 9.7278 33 105.16357 32 85.71167 34 195.94727 22: 2.1356 3.5342 33 79.92554 32 78.86353 34 183.61206 23: 6.0007 9.6381 33 127.00806 32 171.78955 34 79.93164 24: 8.6444 5.3683 33 167.31567 32 187.07275 34 59.71680 25: 7.6883 5.6271 33 82.87964 32 23.54736 34 171.41724 26: 7.1753 5.2127 33 118.62183 32 102.74658 34 17.41333 27: 6.3071 9.0173 33 160.80322 32 37.37183 34 112.35962 28: 9.8917 1.8271 33 55.63965 32 30.85327 34 193.15186 29: 9.3063 1.8320 33 89.63623 32 138.42163 34 4.58984 30: 8.0484 7.4863 33 142.09595 32 192.08984 34 151.72119 31: 3.5632 1.7795 33 118.94531 32 4.26025 34 49.07227 32: 9.5773 4.7897 33 0.84839 32 93.93921 34 105.06592 33: 7.7533 9.5587 33 115.83252 32 168.96362 34 50.00610 34: 1.0129 8.2678 33 174.52393 32 77.58789 34 84.80225 35: 4.4803 8.4412 33 31.43921 32 99.33472 34 158.77686 36: 9.9988 4.5566 33 53.35693 32 119.68384 34 181.53076 37: 7.7020 2.0383 33 32.53174 32 7.19604 34 16.04614 38: 7.1399 7.3541 33 147.31445 32 107.27539 34 45.70923 39: 8.7286 2.9053 33 189.21509 32 174.50562 34 11.57227 40: 1.7572 5.9695 33 23.92578 32 72.00928 34 42.18750 41: 4.4919 3.3966 33 83.71582 32 146.63696 34 45.71533 42: 6.4484 5.2805 33 92.90771 32 90.19775 34 188.25073 43: 1.0941 2.2470 33 181.93970 32 118.30444 34 69.34814 44: 1.4545 6.5723 33 119.01855 32 54.29077 34 108.66699 45: 0.1193 6.9247 33 14.95972 32 21.35010 34 124.39575 46: 2.9922 9.3030 33 110.78491 32 23.29712 34 135.34546 47: 1.0715 4.1751 33 75.29297 32 49.02344 34 131.18896 48: 1.1270 9.5557 33 89.06860 32 89.55688 34 188.93433 49: 6.8298 0.5603 33 169.64722 32 42.34009 34 195.75806 50: 3.7204 8.0731 33 12.04834 32 182.76367 34 30.04761 51: 6.4676 2.4835 33 108.92334 32 145.84961 34 65.14282 52: 4.7330 5.5518 33 45.87402 32 93.24951 34 84.01489 53: 7.3666 2.3865 33 183.98438 32 37.37793 34 67.19971 54: 8.9166 1.5652 33 48.73047 32 161.04126 34 149.40796 55: 2.5650 5.8023 33 196.39893 32 74.03564 34 157.56226 56: 1.9995 2.3358 33 5.43213 32 69.85474 34 123.12622 57: 5.9445 3.9627 33 12.15820 32 100.65918 34 85.90698 58: 7.0340 7.8214 33 34.12476 32 130.82886 34 37.51831 59: 3.4177 9.9609 33 8.33130 32 60.53467 34 111.27930 60: 4.9487 6.5036 33 112.33521 32 64.58130 34 33.66089 61: 9.0079 4.3008 33 112.75635 32 158.45947 34 66.91895 62: 1.7398 3.0731 33 60.40039 32 117.28516 34 13.95264 63: 6.8384 9.2804 33 119.94019 32 176.25122 34 23.84033 64: 7.4100 6.1813 33 2.13623 32 101.34888 34 52.11182 65: 9.8358 2.2983 33 127.81982 32 57.03125 34 54.39453 66: 3.9740 6.8088 33 119.72046 32 159.64966 34 13.79395 67: 9.6390 7.1008 33 4.36401 32 81.16455 34 177.09961 68: 2.8333 8.8361 33 57.78809 32 38.42163 34 82.33643 69: 6.0590 0.4404 33 85.68115 32 79.03442 34 197.29614 70: 6.8454 8.3401 33 35.11353 32 143.61572 34 15.61279 71: 4.6860 0.8606 33 152.88696 32 188.64746 34 169.35425 72: 0.7166 3.2431 33 16.52222 32 172.79053 34 7.27539 73: 4.1467 7.5842 33 58.91724 32 94.75098 34 120.82520 74: 6.1691 6.4380 33 7.37915 32 185.61401 34 13.57422 75: 6.3385 2.0398 33 103.37524 32 105.09644 34 21.33179 76: 6.9598 9.6793 33 83.34961 32 124.68262 34 160.47363 77: 9.9326 3.6383 33 108.58154 32 54.29077 34 74.77417 78: 4.4620 8.5956 33 184.26514 32 150.34180 34 189.88037 79: 5.8237 6.2778 33 58.19092 32 101.26953 34 187.26196 80: 8.4534 7.3975 33 7.44629 32 33.40454 34 63.59253 81: 9.2337 3.9478 33 46.09375 32 27.05078 34 40.09399 82: 1.2070 1.4856 33 158.13599 32 182.67822 34 43.94531 83: 3.1189 3.9801 33 72.77832 32 71.49048 34 102.23999 84: 7.2223 3.4537 33 77.67334 32 195.70313 34 36.21216 85: 7.6334 6.2009 33 43.85376 32 94.72656 34 74.91455 86: 7.2015 9.1669 33 198.40088 32 35.39429 34 134.37500 87: 6.8979 6.7264 33 96.20361 32 136.99341 34 197.21680 88: 4.4299 1.9403 33 93.53638 32 178.04565 34 184.14307 89: 3.4857 7.6654 33 26.01929 32 104.07104 34 81.26831 90: 5.7971 0.6119 33 125.03052 32 73.73047 34 190.13672 91: 9.1089 9.7906 33 44.94019 32 149.63989 34 12.77466 92: 9.5602 7.6926 33 67.71240 32 133.89893 34 172.46094 93: 3.2031 5.5722 33 70.59326 32 185.66284 34 73.77319 94: 8.8495 3.1110 33 29.21143 32 138.22632 34 47.12524 95: 5.7419 6.0880 33 191.05835 32 52.31323 34 143.01758 96: 5.0851 3.5330 33 116.52222 32 157.37305 34 89.64233 97: 2.8961 8.4531 33 154.04663 32 44.94019 34 131.50635 98: 0.1855 3.1180 33 147.65625 32 147.00928 34 166.31470 99: 0.3491 1.8594 33 46.12427 32 152.19116 34 40.74097 100: 1.9360 5.1099 33 94.50684 32 4.43726 34 8.89282 ... 980: 7.6590 0.1218 33 186.43188 32 180.46265 34 36.37085 981: 1.3202 3.3847 33 155.68237 32 18.16406 34 65.32593 982: 3.9047 5.1355 33 163.06152 32 110.54077 34 189.04419 983: 8.5547 6.5576 33 155.72510 32 197.72339 34 134.64966 984: 0.2481 2.0639 33 18.96362 32 40.72876 34 166.73584 985: 7.3749 1.5366 33 98.92578 32 57.36694 34 42.84668 986: 7.7145 8.2721 33 148.16895 32 93.54858 34 50.45166 987: 3.9731 7.8418 33 140.75928 32 113.00659 34 7.70874 988: 1.9385 2.5787 33 40.78369 32 53.44849 34 185.05249 989: 2.5055 3.4680 33 86.19995 32 158.31909 34 34.89380 990: 2.1933 2.4203 33 197.74780 32 188.86108 34 14.01978 991: 4.7644 1.2238 33 88.16528 32 132.70264 34 24.19434 992: 8.9307 0.8044 33 139.25781 32 149.59106 34 19.18945 993: 4.4110 0.4428 33 80.05981 32 18.20068 34 95.30640 994: 2.4542 7.6740 33 80.09033 32 69.81812 34 25.07324 995: 9.3329 5.8911 33 82.72095 32 194.39697 34 95.65430 996: 0.8450 3.8217 33 32.95288 32 43.08472 34 18.46924 997: 2.0261 6.6580 33 68.61572 32 52.85645 34 136.04736 998: 2.7808 2.9456 33 54.71191 32 75.15869 34 15.55786 999: 0.6631 9.5544 33 161.90186 32 185.33325 34 61.21216 1000: 1.6800 5.0317 33 94.12842 32 188.36060 34 6.04248

Replies are listed 'Best First'.
Re: Unexplained benchmark when using chop vs. chomp (or neither)
by davido (Archbishop) on Mar 08, 2018 at 06:00 UTC

    The following code strives to test only chop, chomp, and a noop for the same number of iterations, with the same starting data on each iteration. As far as is practical it attempts to maintain simplicity and to establish a control group (mynada) that performs as similar an algorithm as possible to the two other alternatives. Observe this code and the results:

    use strict; use warnings; use Benchmark qw(cmpthese); my @orig = ("\n") x 100000; sub mychomp { my $data = shift; my @copy = @$data; chomp for @copy; } sub mychop { my $data = shift; my @copy = @$data; chop for @copy; } sub mynada { my $data = shift; my @copy = @$data; 1 for @copy; } cmpthese -10, { chomp => sub {mychomp(\@orig)}, chop => sub {mychop(\@orig)}, nada => sub {mynada(\@orig)}, };

    This produces (on my system):

    Rate chomp chop nada chomp 87.8/s -- -4% -20% chop 91.6/s 4% -- -16% nada 109/s 25% 20% --

    This shows me that the noop is about 20%-25% faster than the other two alternatives, and that the other two are only 4% apart, which is really within the margin for error.

    The fact that you are getting more dramatically different results might indicate that the experiment is not as pristine as it should be, and that you are comparing non-equivalent alternatives.


    Dave

      Note that both chop and chomp can take an array as the argument. Interestingly,
      chop @copy;
      makes mychomp and mychop just as fast as nada.
      ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,

        Yes, I had considered setting up alternatives that pass an array to chomp and chop, but figured it wasn't going to be a very useful comparison for this specific situation. But absolutely, pushing the loop into internals is going to be a lot faster. In fact it wouldn't have surprised me if you had said that the array version was even faster than the 'nada' version, since the 'nada' version still has a loop, unless it's been optimized away, which I'm not sure on.


        Dave

      A better way of reading your results:

      Each chomp took 1/(87.8/s)/100000 = 114 ns (including overhead) Each chop took 1/(91.6/s)/100000 = 109 ns (including overhead) Each nada took 1/(109/s)/100000 = 92 ns

      so

      Each chomp took 22 ns Each chop took 17 ns

      It's not surprising that chop is faster than chomp (since what it does is far, far simpler), but they are both seriously fast! What this means is that trying to optimize this is a waste of time; there is far more "fat" to trim elsewhere.

Re: Unexplained benchmark when using chop vs. chomp (or neither)
by Eily (Prior) on Mar 08, 2018 at 09:56 UTC

    chop will remove a character from the string each time, unless there is none. Since for aliases the variable to the list elements, the strings in @data get chopped until they are empty. As a matter of fact, since the lines are pretty short, the strings are empty before the 20th call to the function, and even without chopping, the function gets called more than 150 times

    Edit: reformulated and shortened my confusing post :)

      Thank you Eily. You have shown the problem that caused the results of the benchmark to be what they were.
Re: Unexplained benchmark when using chop vs. chomp (or neither)
by sundialsvc4 (Abbot) on Mar 08, 2018 at 03:44 UTC

    Although you appear to have made (thank you!) a diligent effort to present data to explain your point, I’m afraid that I see the various alternatives to be “apples and oranges.”   As I superficially review your four alternatives – and without expanding your post – I see what appears to be a “side-by-side comparison” of four very different algorithms.   Also, you neglect to place the three “Perl alternatives” (chomp, chop, neither ...) into the benchmark.   Could you please do that, and post the results as an additional comment to this thread?   Please show us 4 + 2 = 6 lines of benchmark-data.

    (And if I “just missed it,” I apologize in advance.)

      four very different algorithms
      Well, benchmarking several times the same code and using the result to simulate a horse race is a legitimate use of Benchmark. But some people also use Benchmark to compare different algorithms that give the same result.

      Also, you neglect to place the three “Perl alternatives”
      As a matter of fact, since the issue is that the input data is modified by the chopping function, comparing them in the same run may have shown that all the variants gets faster when chop is called.

      Will OP posting the "three Perl alternatives" help you write code for them?

      Why don't you present a solution; that is what this place is about.

      I admit that I comprehend your attention-getting now, but don't you get sick of being shunned, spit and shit on all the time?

      Seriously; my brother just called me the other day. He's been a troll for many years across mostly sports forums. Nonetheless, he told me "Steve, I'm angry all the time. I just interject in ways that attempt to antagonize people. I need to stop."

      Can you stop, or at least start to "pretend" to speak in a way where your rhetoric appears like you're not just a political advisor?

      If you do get off on "trolling", have the balls to say it. If not, please let us know what 1992 product you still service, along with how you "consult" and "advise" software companies.

      Please show us 4 to 6 lines of Perl code that compiles, runs and does something useful. Go on. Surprise me.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". The enemy of (IT) success is complexity.
      In the absence of evidence, opinion is indistinguishable from prejudice. Suck that fhit
        "...4 to 6 lines of Perl code that compiles..."

        Mostly it is just a question of the his comments:

        perl -E 'say qq(I “just missed it,” I apologize in advance); // svc'

        Best regards, Karl

        «The Crux of the Biscuit is the Apostrophe»

        perl -MCrypt::CBC -E 'say Crypt::CBC->new(-key=>'kgb',-cipher=>"Blowfish")->decrypt_hex($ENV{KARL});'Help

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1210495]
Front-paged by stevieb
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (7)
As of 2018-06-20 07:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Should cpanminus be part of the standard Perl release?



    Results (116 votes). Check out past polls.

    Notices?