comment on

It is slow probably because you seem to be doing an awful lot of reading and writing. An example log entry would have helped, but the following (with the addition of the appropriate checking) will create the log file on sorted order without a temp file and without readin the whole file into memory.

my $MAXBUFLEN = 1024 * 1024; # 1M

sub sort_logfile {
  my $file = shift;
  my @off = (0);
  my @lines;
  open(INFILE, "<$file");
  while(<INFILE>) {
    push @off, tell;
    if (/^\d.*?\[([^\]]+)\]/) {
      push @lines, [$1, $.-1];
    }
    else {
      # assume any non-matching line
      # is a continuation of the previous
      $off[-2] = $off[-1];
      next;
    }
  }

  @lines = sort { $a->[0] cmp $b->[0] } @lines;

  open(INFILE,"<$file");

  my $line = 0;
  my $buf;
  while (my $i = shift @lines) {
    my ($start, $end) = ($i->[1],$i->[1]+1);
    while( @lines and $lines[0]->[1] == $end) {
      shift @lines;
      $end++;
    }
    ($start,$end) = @off[$start,$end];
    my $len = $end - $start;
    while ($len > 0) {
      seek(INFILE, $start, 0) or die "seek: $!";
      read(INFILE, $buf='', $len > $MAXBUFLEN ? $MAXBUFLEN : $len)
        or die "read: $!";
      print $buf;
      $len -= length $buf;
    }
  }
}
[download]

In reply to Re: Slow but it works by gbarr
in thread Slow but it works by scottstef

Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
Want more info? How to link or How to display code and escape characters are good places to start.


Perl: the Markov chain saw
	PerlMonks