Re: Recommendations for perl modules to work on data sets ?

[tybalt89]

Fun little problem. When you think patterns, think regex :)

#!/usr/bin/perl -l

# http://perlmonks.org/?node_id=1174464

use strict;
use warnings;

my @data;
my $increasing = '';

my $previous;

while(<DATA>)
  {
  my $n = push @data, $_;
  my ($this) = (split)[1];
  $previous //= $this;
  $n > 1 and $increasing .= ($this <=> $previous) + 2;
  $previous = $this;
  }

#print "$increasing\n";

my $pos = 0;

$1 && print( @data[$pos..$pos+4]), $pos += length $&
  while $increasing =~ / ([13])\1{3}(?!\1)(?:.|$) | (.)\2* /gx;


__DATA__
10/01/2016   99.71
10/02/2016   99.53
10/03/2016   100.10
10/04/2016   100.96
10/05/2016   100.99
10/06/2016   101.38
10/07/2016   100.74
10/08/2016   100.70
10/09/2016   99.88
10/10/2016   97.62
10/11/2016   97.55
10/12/2016   99.12

[raghuprasad241]

@tybalt89, thank you for showing me the cool code.


I got a question though, isn't it true that REGEX's are slow when you are dealing with large amounts of data ?

Thanks!

[Laurent_R]

Premature optimization is the root of all evil (or at least most of it) in programming. (Donald Knuth)

In general, if a regex is the right tool to solve your problem, then use a regex. Try to change it so something else afterwards, only if you've found that it is too slow. But don't make it more complicated than it needs to be if you don't have to.

Well, having said that, I should add that there are some exceptions: sometimes it is good to think about a fast algorithm or a more efficient data structure before you start coding, but that usually does not apply to micro-optimizations and small efficiencies such as regexes versus other solutions (such as substr, index or unpack).

[tybalt89]

Only benchmarking can answer that question. See Benchmark.pm