Stripping a seemingly complicated comma-delimited file

[jamesSA]

I have a comma-delimited file of a portfolio management system, i understand that by using a ($var1,$var2) = split(/,/) i can get the fields i need out, the trouble is my while <FILE> handle isnt working. By this i mean ,i cannot get this while loop to loop to the next line. i keep getting the same line being stripped and broken up over and over again. ive tried WHILE $var =~ /\n/ and the \r parameters. but it still wont loop. any ideas guys ? thx (only worked with perl for a month now,logic is fine,syntax is fine,just find one or two things bugging hehe) The file looks like this (01655 etc indicates the start of a new line)

0165593,P,SHF     ,AWJ   ,001,001,01,METBOX,00000044869,00000311839.55
+,00000006.9500000,25/01/01,ZAR,000000779.60
0165594,P,SHF     ,AWJ   ,001,001,01,LINCWD,00000023602,00000164033.90
+,00000006.9500000,25/01/01,ZAR,000000410.08

[the_slycer]

Perhaps CSV would do the trick for you. This module is designed to work with comma seperated files.

For your problem I'm really not sure what you are asking. To split up a file like this you could use something like:

while (<FILE>){
   @split_vals=split /,/,$_;
   #process line in @split_vals
}

Or to get the whole file in one big array:

while (<FILE){
   push @split_vals=split /,/,$_;
}
#process whole split file.

[jamesSA]

yeh i can get it into an array but its really messy. you're talking about pushing each value seperated by a comma into an array variable which ive done ... but it wont help. effectively im trying to break each line into 14 variables. those variables will ultimately be shoved into a db table. So i need to 14 $variables that in the loop will keep generating new values and populate the db table. i'd prefer not to use the CSV module,actually trying to learn the hard way :>

[the_slycer]

Well in that case instead of using @array=split/,/,$_ just use the ($var1,$var2...) syntax, no real difference (Though personally I'd rather assign to the array and call the values by $array[5] instead of $var5 :-).

Not using CSV is your perogative, though if any of your fields have any chance of having an embedded comma in them, it gets tricky to split the values.

[xphase_work]

From the information you give, I'd imagine that you have a file handle
open, and are trying to read each line from it.

# FH is our example file handle
while(<FH>){
    #action on each line
}

Or...

my @file = <FH>;
foreach my $line (@file){
    # action on each line
}

The first example loops through the file opened on FH, while
the second example stores the file opened on FH in the array
@file, one line per entry. I hope this is what you wanted.

-xPhase

[Big Willy]

This is a bit tough without seeing the code, but I am guessing this will probably do what you are asking...

while(<DATA>) {
     ($var1, $var2, ...) = split(/,/);
     ...
}

The loop will read each line of the file into $_, which is split. Then you can go to town on it however you like.

[iakobski]

You need to post the code to show us how you are storing the data before you split it. If you are reading each line from a file and splitting it straight away the while loop will be something like

while (<FILE_HANDLE>){
  @items = split /,/;
  # do something with @items
}

Or if you have the lines in an array use a foreach loop:

foreach ( @lines ){
  # split in here
}

The while statement you have shown stays true for as long as there is a new line, it does not move through the data.

-- iakobski

[jamesSA]

 #!/usr/local/bin/perl -w

open(HIPORT,"eq010125.txt");

$file = <HIPORT>;

$i = 0;

while ($file =~ /\n$/) {
    chop $file;
($var1,$var2,$var3,$var4,$var5,$var6,$var7,$var8,$var9,$var10,$var11,$
+var12,$var13,$var14) =
split(/[,]|[,]|\n$/,$file);
    print "$i : $var1 $var2 $var3 $var4 $var5 $var6 $var7 $var8 $var9 
+$var10 $var11 $var12 $var13 $var14 \n";
    $i++;
}

[virtualsue]

#!/usr/local/bin/perl -w     # GOOD!

 open(HIPORT,"eq010125.txt");

Please check the error from your file open. What if it fails? open(HIPORT,"eq010125.txt") or die "Couldn't open file, $!"; However, your main problem is that you are assigning the contents of your file to a scalar - $file.
$file = <HIPORT>; You definitely don't want to do that. @file = <HIPORT>; will suck the contents of the file into an array for you. I strongly suggest that you start off with code that reads your file line by line. You are trying to do far too much work. Use the program I have posted or any one of the similar programs that others have put up. :)

[the_slycer]

The problem that you are running into here is with this line

$file = <HIPORT>;

What you have done there is not what you think. When you assing $file to the <> operator it grabs one line - not every line. Either assign it to an array (@file) which will put each line as an array entry, or even better loop through the file with

while my $file(<HIPORT>){
   #do stuff
}

[virtualsue]

Let's start with an example that you can take and adapt to fit your comma-separated datafile. Download (use link at bottom of this node) or cut & paste this code & adapt it. Here's an example of parsing a file containing 3 comma-delimited fields:

#!perl -w
use strict;

open(FILE, "< csv_filename") or die "Couldn't open file, $!";

while(<FILE>) {
   chomp;  # Remove the newline
   my ($var1, $var2, $var3) = split /,/;
   print $var1," ",$var2," ",$var3,"\n";
}
exit;

We can move on to what you want to do next after you get past reading the file & parsing each line successfully.

[jamesSA]

thanks i was an idiot ... i forgot chomp :)

[AidanLee]

could you show us your actual while loop?

[jamesSA]

heres the code :

open(HIPORT,"eq010125.txt");

$file = <HIPORT>;

$i = 0;

while ($file =~ /\n$/) {
    chop $file;
($var1,$var2,$var3,$var4,$var5,$var6,$var7,$var8,$var9,$var10,$var11,$
+var12,$var13,$var14) =
split(/[,]|[,]|\n$/,$file);
    print "$i : $var1 $var2 $var3 $var4 $var5 $var6 $var7 $var8 $var9 
+$var10 $var11 $var12 $var13 $var14 \n";
    $i++;
}

Edit: chipmunk 2001-06-11

[boo_radley]

you're never actually reading from the file according to this code. try

while ($file=<HIPORT>)

instead of

while ($file=~/\n$/)

[AidanLee]

you need to wrap your code in <code> tags so it is more readable. You're also currently missing out on your brackets in your regex. This will be solved when using code tags.

[andreychek]

Well, if you enjoy that code you are writing, that is fine.. however, there is already a module that can do most of the dirty CSV work for you.

I went to dig around at CPAN to figure out what all was available, but apparently somebody tampered with it, so CPAN is down for now.

In the meantime, modules that may help you are Text::CSV_XS and, if you are interested in using SQL syntax, perhaps DBD::CSV might be neat.

One good reason to use one of these is that if you're data gets any more complicated then what you mentioned, you might have to use some regexes which are a bit more complex, and somebody already did that work for us :-)
-Eric

Update: Good golly, apparently I type too slow.. there were only 2 comments when I clicked on the comment button :-)