Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re^3: Interpreting Benchmark 'cmpthese' and 'timethese' (time)

by tye (Sage)
on Aug 28, 2006 at 11:53 UTC ( #569968=note: print w/replies, xml ) Need Help??


in reply to Re^2: Interpreting Benchmark 'cmpthese' and 'timethese'
in thread Interpreting Benchmark's "cmpthese" and "timethese"

So what is the best way to show the speed comparison in terms of it's time and not iteration/time?

Since you have

Rate enum list enum 502/s -- -70% list 1668/s 232% --

The times are 1/502 of a second vs. 1/1668 of a second. Or about 0.002s vs. 0.0006s so a difference of 0.0014s.

The block of output that I quoted compares the results while the prior output logs some details about how the computations were done. Those details aren't particularly useful for making a comparison.

You could also multiply by 1000 and say that, if you needed to call the compared bits of code 1000 times each, then one would take about 2s plus overhead while the other would take about 0.6s plus overhead. So the time difference for 1000 iterations would be about 1.4s. Note that the time for "overhead" will usually be quite a bit more than the time required to run just the compared code so that real code doing something 1000 times might have run times like 22s vs 20.6s.

Update:

from 'cmpthese' results it is clear that "list" is more than twice faster than "enum"

Actually, the "232%" means that one is more than three times faster than the other. This is a pet peave of one of my coworkers. There are several contradicting ways to report "percentage difference" and most of them lead to people making mistakes when they try to combine them. He strongly believes that it is best to avoid "percentage difference" and only ever deal in terms of simple multipliers. So he'd report the above results like:

Rate enum list enum 502/s -- 0.3 list 1668/s 3.32 --

"50% faster" means "can do it as fast as the other plus 50%" or "150% as fast" or "1.5 times as fast". So "200% faster" means "3 times as fast".

To see how confusing this can be, consider these two ways of describing the same situation:

If "A is 50% faster than B and B is 50% faster than C", how much faster is A than C? 'Obviously', "A is 125% faster than C".

If "A is 1.5 times faster than B and B is 1.5 times faster than C", how much faster is A than C? Well, 1.5*1.5 is 2.25 so, obviously, "A is 2.25 times faster than C".

- tye        

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://569968]
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (4)
As of 2018-07-17 06:13 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    It has been suggested to rename Perl 6 in order to boost its marketing potential. Which name would you prefer?















    Results (355 votes). Check out past polls.

    Notices?