Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling

Searching for Patterns in CSV

by Fischer (Initiate)
on Jun 28, 2015 at 13:58 UTC ( #1132375=perlquestion: print w/replies, xml ) Need Help??

Fischer has asked for the wisdom of the Perl Monks concerning the following question:

I'm a Perl newbie. I have this perl command line
perl -i.bak -F, -ane ' if ($F[0]+1 == $F[1] and $F[1]+1 == $F[2]) {warn "row $.: $_"} els +e {print} ' file.csv 2>file.log
I want to make it a perl script So the program should work like so, but I'm not sure how to write it properly in perl
for line in file: for i in range(1,4): if ($F[0]+ i == $F[1] and $F[1]+i == $F[2]): remove line
so you see it's pretty basic, I want use perl, not python for reasons outside the scope of my question

Replies are listed 'Best First'.
Re: Searching for Patterns in CSV
by davido (Cardinal) on Jun 28, 2015 at 14:20 UTC

    Start by reading perlintro, and perlsyn. You could use the <> operator (or if you have Perl 5.22, the <<>> operator) to read from the file. You may want to chomp your input. Then split into an array, do your test, warn if there's a problem, and finally print your output.

    If you prefer to eliminate shell redirection, then you will also need to open an output file and print to it instead.

    If your CSV could ever become non-trivial, you would probably want to switch to using Text::CSV instead of simple split to parse the file.


Re: Searching for Patterns in CSV
by roboticus (Chancellor) on Jun 28, 2015 at 14:18 UTC


    Assuming that 'file' is a file handle and you're iterating over a file, then you'd use while (my $line=<$file>) { ... }. The range(1,4) function call translates to 1..3. So the first couple lines translates to:

    while (my $line=<$file>) { for my $i (1..3) { ... stuff you want to do ... } }

    That ought to get you started. Read some basic docs like perlintro to start coming up to speed.

    Note: You *could* translate the first line to for my $line (<$file>) { ... } but you don't want to do that--for the for loop version, perl would read the entire file into memory before giving your code the first line. The while loop version reads a line at a time from the file, which is nearly always what you want to do.


    When your only tool is a hammer, all problems look like your thumb.

Re: Searching for Patterns in CSV
by Laurent_R (Canon) on Jun 28, 2015 at 15:01 UTC
    Since you provided some code (although not Perl code), I am willing to bring you a bit further:
    while (my $line = <$file>) { chomp $line; # remove trailing end of line character my @F = split /,/, $line; for my $i (1..3) { if ($F[0]+ $i == $F[1] and $F[1]+$i == $F[2]) { warn "row $.: $_\n"; } else { print $line, "\n"; } } }
    Not sure though that this implements the logic that you want. Your Python program does not do the same thing as the Perl one-liner, so I don't really know what you want. If you want to test for each value in the (1..3) range and print if none of the conditions matched then it should rather something like this:
    LINE: while (my $line = <$file>) { chomp $line; my @F = split /,/, $line; for my $i (1..3) { if ($F[0]+ $i == $F[1] and $F[1]+$i == $F[2]) { warn "row $.: $_\n"; next LINE; # the condition matched, go immediately to the +next input line, do not print } } print $line, "\n"; }
Re: Searching for Patterns in CSV
by 1nickt (Abbot) on Jun 28, 2015 at 14:41 UTC

    Please do spend some minutes reading one or two basic Perl tutorials, just because at the least it will help you formulate questions better and also allow you to make an attempt in Perl at what you want to do, and then post what you have tried, here. You'll get better help, faster, that way :)

    The above post by Roboticus gives some great info on reading through the lines in a file. You'll also have to consider how to open the file for reading, how to handle an error if that fails, etc.

    One of Perl's strengths is the vast public library of modules that already do most of what you need to do. There are lots of ways to open and read from a file, but like everything in Perl, you quickly reach the point at which it is more efficient to use a pre-existing module.

    You might search on meta::cpan for "file", or "path", or "file open", or "file read", or something like that, and see what you get.

    It is nice to be able to do in your code only this, as one example, to get to where your pseudo-code example begins:

    #!/usr/bin/env perl use strict; use warnings; use File::Slurp::Tiny 'read_lines'; my $file = '/path/to/file'; for ( read_lines($file) ) { # do something }
Re: Searching for Patterns in CSV
by Tux (Canon) on Jun 29, 2015 at 05:57 UTC

    If using a module like Text::CSV_XS is an option, use its csv function with the filter attribute:

    $ cat use 5.20.0; use warnings; use List::Util qw(first); use Text::CSV_XS qw(csv); csv (in => *DATA, filter => { 2 => sub { (first { $_[1][0] + $_ == $_[1][1] && $_[1][1] + $_ == $_[1][2] } 1 .. 3) ? 0 : 1 }}); __END__ 0,1,3,4,5,# +1 +2 keep 1,2,3,4,5,# +1 +1 skip 1,3,1,4,5,# +2 -2 keep 1,3,5,4,5,# +2 +2 skip 2,2,3,3,4,# 0 0 keep 0,3,6,4,5,# +3 +3 skip 1,0,0,4,5,# -1 0 keep 0,4,8,4,5,# +4 +4 keep weird python 0,5,10,,5,# +5 +5 keep $
    $ perl 0,1,3,4,5,"# +1 +2 keep" 1,3,1,4,5,"# +2 -2 keep" 2,2,3,3,4,"# 0 0 keep" 1,0,0,4,5,"# -1 0 keep" 0,4,8,4,5,"# +4 +4 keep weird pyhton" 0,5,10,,5,"# +5 +5 keep" $

    update: changed 1..4 to 1..3, as I misinterpreted the stupid counterintuitive python range function

    Enjoy, Have FUN! H.Merijn
Re: Searching for Patterns in CSV
by stevieb (Canon) on Jun 28, 2015 at 15:09 UTC

    It would help if you could post the exact Python code you're using instead of mixing Perl and Python. Also include a few lines of your input, preferably at least one that matches your if() and another one that doesn't.


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1132375]
Approved by hippo
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (9)
As of 2021-06-15 09:32 GMT
Find Nodes?
    Voting Booth?
    What does the "s" stand for in "perls"? (Whence perls)

    Results (69 votes). Check out past polls.