perl printf vs C fprintf performance

[decnartne]

I'm probably missing something obvious... Why do many C fprintf statements take around 1 second (wallclock) and the equivalent perl printf statements take 4 seconds? Consider (ignoring for now some of my perhaps idiomatic failings):

#!/usr/bin/perl -w

use strict;

my $outf = "/tmp/outf.p.out";

open OUTF, ">$outf" or die "ERROR: could not open file $outf - $!\n";

foreach my $i ( 1..1000 )
{
    foreach my $j ( 0..255 )
    {
        printf( OUTF "host DAAA%02X { blah blah XX:XX:XX:%02X; }\n", $
+j, $j );
        printf( OUTF "host DAAA%02X { blah blah XX:XX:XX:%02X; }\n", $
+j, $j );
        printf( OUTF "host DAAA%02X { blah blah XX:XX:XX:%02X; }\n", $
+j, $j );
        printf( OUTF "host DAAA%02X { blah blah XX:XX:XX:%02X; }\n", $
+j, $j );
    }
}

close OUTF;

exit 0;

__DATA__

/*  C (somewhat?) equivalent of the perl script above  */
#include <stdio.h>

int main ()
{
    char *outf = "/tmp/outf.c.out";
    FILE *fd;
    int i,j;

    if ( !( fd = fopen( outf, "w" ) ) )
    {
        printf( "ERROR: could not open file %s\n", outf );
        exit(1);
    }

    for ( i=1;i<=1000;i++ )
    {
        for ( j=0;j<=255;j++ )
        {
            fprintf( fd, "   host DAAA%02X { blah blah XX:XX:XX:%02X; 
+}\n", j, j );
            fprintf( fd, "   host DAAA%02X { blah blah XX:XX:XX:%02X; 
+}\n", j, j );
            fprintf( fd, "   host DAAA%02X { blah blah XX:XX:XX:%02X; 
+}\n", j, j );
            fprintf( fd, "   host DAAA%02X { blah blah XX:XX:XX:%02X; 
+}\n", j, j );
        }
    }

    fclose( fd );

    exit(0);
}

My configuration is as follows:

hostname% uname -a
SunOS hostname 5.9 Generic_117171-07 sun4u sparc SUNW,Sun-Fire-V440
hostname% perl -v

This is perl, v5.6.1 built for sun4-solaris-64int

Timed results are:

hostname% time ../bin/ftest
1.0u 0.0s 0:01 85% 0+0k 0+0io 0pf+0w
hostname% time ./ftest.pl
4.0u 0.0s 0:04 86% 0+0k 0+0io 0pf+0w

(... for the curious, I have ported a C program which generates our DHCP table to perl, thus the pseudo formatting of the string...)

Taking out the hex formatting results in a faster run time (duh!), but still slower than equivalent C fprintf().

Thanks in advance for any info...

[moot]

The unix 'time' command is a very unreliable indicator of program performance. Included in those 4 wallclock seconds, for instance, are the time taken to start the perl interpreter; the time taken to read, parse, and compile your script; and the time taken to unload everything at the end, to name but a few.

You might want to look into Benchmark to see how your perl script is really performing. There are also several other great references available with a few moments googling, with more general benchmark advice.

[decnartne]

agreed - Benchmark probably would have been more accurate for times... I did actually use it in the original script to debug where my performance hit was. I found that the actual writing (the 8 or so printf statements) took ~8 seconds whereas the C app took ~2

All that aside - I'm still somewhat at a loss (surprise!) as to why writing ~ 1M lines in a file w/ C fprintf is so much faster...

[brian_d_foy]

You're not simply timing the print statements, but also the perl overhead (which gets less and less significant the larger your program gets) as well as the compilation time (which can continue to be significant).

To be fair to perl, you'd have to also add in the time to compile your C program and whatever else you have to do to make it an executable.

Still, you should never be surprised that C is faster than perl. :)

[decnartne]

...understood. I guess, I'm just surprised that C was *that* much faster in this instance.

The replacement script fairs pretty well processing all of the records from the DB, etc. However, I am just a little curious how printf vs fprintf differ enough for a ~6 second difference...

[hv]

I used the following code to benchmark, in an attempt to reduce the comparison solely to the actual printf calls:

use warnings;
use Benchmark qw(cmpthese);
open OUTF, '>', '/dev/null' or die "/dev/null: $!\n";
cmpthese(0, {
  perlprint => q{
    for my $j (0 .. 255) {
      printf OUTF "host DAAA%02X { blah blah XX:XX:XX:%02X; }\n", $j, 
+$j;
    }   
  },
  Cprint => q{
    for my $j (0 .. 255) {
      C_print(*OUTF, $j);
    }
  },
});

use Inline 'C' => <<INLINE;
void C_print(FILE *fp, int j) {
  fprintf(fp, "host DAAA%02X { blah blah XX:XX:XX:%02X; }\n", j, j);
}
INLINE

This consistently reported the C code as being about twice as fast as the perl code (on my perl-5.8.5 installation), which seems about reasonable to me given all the extra layers of indirection the perl code has to wade through.

One of the differences between perl-5.6.x and perl-5.8.x is that the 'perlio' abstraction is now the default, and with perlio enabled we get to take advantage of some of the lower-level I/O mechanisms available to the C library fprintf. It is possible that switching to a perl built with 'perlio' will get you a factor closer to this - check for "perlio=define" in the perl -V output.

Hugo

[decnartne]

...thx for the explanation

[Leotrillo]

Hi, my name is Leo an i have the same problem, is very slow to open some document to write in it. you comment "It is possible that switching to a perl built with 'perlio' will get you a factor closer to this - check for "perlio=define" in the perl -V output." Can you put some examples to see how i can to accelerate this r/w docs in perl? thank you.

[Anonymous Monk]

useperlio is an option you set when compiling/installing perl http://search.cpan.org/dist/perl/INSTALL

$ perl -V |grep -i perlio
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=und
+ef
                        USE_LARGE_FILES USE_PERLIO USE_SITECUSTOMIZE

$ perl -V:useperlio
useperlio='define';

[sh1tn]

One strict criteria:

%strace -c ./ftime    
%strace -c ./ftime.pl

Thus one can see which spends what time and where
(as far as system calls are concerned).