Heh, I did the same thing chromatic did and created another benchmark, then figured I should really get work done instead on futzing around on Perlmonks. If I think the speed difference between quoting operators is the limiting part of my program, I really have too much time on my hands or a program that doesn't do anything interesting (or is something Perl shouldn't be doing). Any result from Benchmark that is less than about 7% shouldn't be trusted (see links later in post).

Another problem with this particular benchmark is that the interesting part of the code (the quote thing) is tied up in the I/O and other bits. The numbers don't show the difference between the quotes, they show the difference between the various other things combined with it. You have to pull that stuff apart to figure out what's going on. One way to do that is just what chromatic did: can you get the same answer twice?

I went through a lot of this discussion in Wasting time thinking about wasted time, in my Benchmarking Perl" talk at Uniforum last year, and in my Benchmarking chapter for Mastering Perl. Benchmarks are for the people who constantly ask themselves "How can this be wrong" and do the thinking and the work to find every hole in what they are doing. You can't stop when you have a number that you think is right. You have to do more benchmarks in other ways to see if you can use what you know to make predictions that hold up to reality. :)

--
brian d foy <brian@stonehenge.com>
Subscribe to The Perl Review

In reply to Re: No More Meaningless Benchmarks! by brian_d_foy
in thread No More Meaningless Benchmarks! by chromatic

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.