Re: use File::Slurp for (!"speed");

[revdiablo]

reading a file smaller than a disk block (because, hell: we're not using slurping for any kind of sizable file... who would be doing that and still be concerned about performance?)

I think there is some area between a file "smaller than a disk block" and "any kind of sizable file." You only test one end of the spectrum, and nowhere in the middle. Let's see how File::Slurp holds up with a bit more testing.

First, the benchmark code:

use File::Slurp;
use Benchmark qw(cmpthese);

cmpthese(-2, { fs => sub { $x = read_file "foo" },
               is => sub { $x = is("foo"); }
             });

sub is { local (@ARGV, $/) = $_[0]; <> };

Note, I put the idiomatic slurp into a subroutine. Much of File::Slurp's compared performance hit was due to Perl's slow subroutine calling. I thought it would be a more apples-to-apples comparison to add that hit for the idiomatic slurp. Feel free to post results with the idiomatic slurp inlined, if you want, but all they will do is change the point where File::Slurp overtakes the IS.

Now, some test runs:

$ perl -e 'print "x"x500' > foo
$ perl benchmark
      Rate   fs   is
fs 30210/s   -- -33%
is 44886/s  49%   --

$ perl -e 'print "x"x5_000' > foo
$ perl benchmark
      Rate   fs   is
fs 27499/s   -- -26%
is 37057/s  35%   --

$ perl -e 'print "x"x50_000' > foo
$ perl benchmark
      Rate   is   fs
is 11275/s   -- -14%
fs 13094/s  16%   --

$ perl -e 'print "x"x500_000' > foo
$ perl benchmark
    Rate   is   fs
is 277/s   -- -15%
fs 325/s  17%   --

$ perl -e 'print "x"x5_000_000' > foo
$ perl benchmark
     Rate   is   fs
is 29.5/s   -- -17%
fs 35.6/s  21%   --

As we can see, the idiomatic slurp is faster for the 500 and 5,000 byte files, but once we get into the 50,000 range, File::Slurp takes the lead. You may consider 50k to be too big to slurp, but I certainly do not. In fact, I can even imagine circumstances where slurping the 5mb file would be reasonable.

Don't get me wrong. I'm not trying to disagree with your main point. In your particular case, switching to File::Slurp was probably not the right idea. But I certainly can see the case for slurping files that are 50k or more, and in this case, File::Slurp is faster.

[etcshadow]

OK, let's not arbitrarily turn something into a function call that doesn't have to be, and, more importantly, let's recognize the overhead of loading a module:

use Benchmark qw(cmpthese);

cmpthese(-2, {
        fs => sub { delete $INC{'File/Slurp.pm'}; require File::Slurp;
+ File::Slurp->import(); $x = read_file("foo") },
        is => sub { $x = do { local (@ARGV, $/) = "foo"; <> }; }
});

---

$ perl5.6.0 test.pl
Benchmark: running fs, is, each for at least 2 CPU seconds...
        fs:  3 wallclock secs ( 2.08 usr +  0.11 sys =  2.19 CPU) @ 26
+6.67/s (n=584)
        is:  2 wallclock secs ( 1.16 usr +  0.86 sys =  2.02 CPU) @ 47
+311.39/s (n=95569)
      Rate     fs     is
fs   267/s     --   -99%
is 47311/s 17642%

And, yes, I am being a little tongue in cheek with that... but the point is: benchmarks are what you make of them.

[revdiablo]

let's not arbitrarily turn something into a function call that doesn't have to be

I didn't turn it into a function call arbitrarily. In fact, I explained my reasoning for doing so. I was attempting to compare the speed of File::Slurp to the idiomatic slurp. I was not attempting to measure the speed of Perl's subroutine calls, much less loading of modules. It's already well-established that these things are slow.

This is how I see things so far: you stated that File::Slurp is slower than the idiomatic slurp. I demonstrated that it is, but only under very constrained circumstances. Then, you constrained the circumstances even further. I don't get the point you're going for here...

[rcaputo]

I didn't turn it into a function call arbitrarily. In fact, I explained my reasoning for doing so. I was attempting to compare the speed of File::Slurp to the idiomatic slurp. I was not attempting to measure the speed of Perl's subroutine calls, much less loading of modules. It's already well-established that these things are slow.

A fairer benchmark would compare the slurp methods as they'll be used. Since idiomatic slurp doesn't require a function call, it seems sensible not to add the extra overhead into the benchmark. Unfortunately the modular solution can't escape Perl's function calls, so the overhead must be factored into its performance.

I'm sure there's a point where File::Slurp's efficiency overwhelms the function call penalty. It would be interesting to see where that point actually is, even though it probably varies from one system to another.

The original poster's argument is that their system slurps files that fall below the break-even point for File::Slurp. It's therefore more efficient (in runtime speed) to use inline idiomatic slurp.

-- Rocco Caputo - http://poe.perl.org/