Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Processing CSV File

by Perl3r (Initiate)
on Oct 03, 2012 at 05:28 UTC ( #996981=perlquestion: print w/ replies, xml ) Need Help??
Perl3r has asked for the wisdom of the Perl Monks concerning the following question:

Hello, I have the following code that I am trying to get to work.. There are a couple of components; 1. The .pl script (see contents below) 2. A template file rtr-template.txt (which contains the variables listed below - ie ##location##, ##rtrname## etc 3. A CSV file that contains all the variables.. What is suppose to happen, is that when you run the script, it processes the CSV, line by line, and creates an individual file called ("rtrname.txt") for each line.. What is currently happening, is that when I run it, it creates the first file fine.. but then stops after that, and doesn't process the subsequent lines..

#!/usr/bin/perl # use warnings; $template_file_name="rtr-template.txt"; while(<>) { ($location, $name, $lo0ip, $frameip, $framedlci, $eth0ip) = split (/,/); open(TFILE, "< $template_file_name") || die "config template file $ +template_file_name: $!\n"; $ofile_name = $name . ".txt"; open(OFILE, "> $ofile_name") || die "output config file $ofile_name +: $!\n"; while (<TFILE>) { s/##location##/$location/; s/##rtrname##/$name/; s/##eth0-ip##/$eth0ip/; s/##loop0-ip##/$lo0ip/; s/##frame-ip##/$frameip/; s/##frame-DLCI##/$framedlci/; printf OFILE $_; } }

Would any one be able to shed some light on what might be my issue? I am running; Perl Version 5, version 12, subversion 4 (v5.12.4), on a MAC OS X 10.8.2

Comment on Processing CSV File
Download Code
Re: Processing CSV File
by Athanasius (Monsignor) on Oct 03, 2012 at 06:11 UTC

    Hello Perl3r, and welcome to the Monastery!

    You are opening the output file $ofile_name each time through the outer while loop. This truncates the file each time before writing to it! Move the statement open(OFILE, ...) ... to before the first while loop, so that the file is opened only once.

    Update 1: Sorry, I didn’t see that the name of the output file is changing on each pass of the outer loop.  :-(

    It is also good practice to begin your scripts with

    use strict;

    and to declare lexical variables using my.

    Hope that helps,

    Update 2: I created two files:

    (1) “data.csv”

    here,fred,42,43,44,45 there,wilma,2,5,7,9

    (2) “rtr-template.txt”

    this is the ##location## to be in! my name is ##rtrname##. the frame is '##frame-DLCI##'.

    Then I invoked your original script as:

    perl test_script.pl data.csv

    And the result was two output files:

    (a) “fred.txt“

    this is the here to be in! my name is fred. the frame is '44'.

    (b) “wilma.txt”

    this is the there to be in! my name is wilma. the frame is '7'.

    — which is exactly as expected. So, I cannot reproduce the problem you are seeing. Are you sure you are running the code as described?

    Athanasius <°(((><contra mundum

      Thanks for the response.. I did the following;

      #!/usr/bin/perl # $template_file_name="rtr-template.txt"; open(OFILE, "> test") || die "output config file $ofile_name: $!\n"; while(<>) { ($location, $name, $lo0ip, $frameip, $framedlci, $eth0ip, $x) = split (/,/); open(TFILE, "< $template_file_name") || die "config template file $ +template_file_name: $!\n"; $ofile_name = $name . ".txt"; while (<TFILE>) { s/##location##/$location/; s/##rtrname##/$name/; s/##eth0-ip##/$eth0ip/; s/##loop0-ip##/$lo0ip/; s/##frame-ip##/$frameip/; s/##frame-DLCI##/$framedlci/; printf OFILE $_; } }

      I tried just moving the line as is, but since $ofile_name hadn't been defined yet, that would't work.. so i replaced the variable with 'Test' This creates a file called Test with the information required.. however still only processes the first line of the CSV - as i guess is expected, as it is emulating the same problem as before.. If i move the line "$ofile_name = $name . ".txt";" to the top aswell, so it looks like:

      #!/usr/bin/perl # $template_file_name="rtr-template.txt"; $ofile_name = $name . ".txt"; open(OFILE, "> test") || die "output config file $ofile_name: $!\n"; while(<>) { ($location, $name, $lo0ip, $frameip, $framedlci, $eth0ip, $x) = split (/,/); open(TFILE, "< $template_file_name") || die "config template file $ +template_file_name: $!\n"; while (<TFILE>) { s/##location##/$location/; s/##rtrname##/$name/; s/##eth0-ip##/$eth0ip/; s/##loop0-ip##/$lo0ip/; s/##frame-ip##/$frameip/; s/##frame-DLCI##/$framedlci/; printf OFILE $_; } }

      then when i run the code, I don't get any files outputted.. Do you have any ideas, what is going on... Thanks again

        If I may suggest considering a slightly different approach to your task...

        It appears that you're opening the template file each time a csv line is read, so that you can fill in the template with the entries. Consider opening the template file only once, and grabbing its contents into a scalar. Then, you can do the substitutions on a copy of that scalar, and write that copy to a file. Doing that may look something like this:

        #!/usr/bin/perl # use strict; use warnings; my ( $templateText, %hash ); my $template_file_name = 'rtr-template.txt'; my @vars = split "\n", <<END; ##location## ##rtrname## ##loop0-ip## ##frame-ip## ##frame-DLCI## ##eth0-ip## END { local $/; open my $fh, '<', $template_file_name or die "$template_file_name: + $!"; $templateText = <$fh>; close $fh; } while (<>) { chomp; @hash{@vars} = split /,/; my $templateTextCopy = $templateText; $templateTextCopy =~ s/$_/$hash{$_}/g for keys %hash; my $ofile_name = $hash{'##rtrname##'} . '.txt'; open my $fh, '>', $ofile_name or die "$ofile_name: $!"; print $fh $templateTextCopy; close $fh; }

        You'll notice this notation: @hash{@vars} = split /,/; It pairs the elements of @var as keys with the split entries of the csv line as values of %hash. This creates the hash that's used in the substitution line, taking the place of the six substitution lines. Also, note that the substitution is global, because your script was going through all the lines of the template for the substitutions.

        Hope this helps!

Re: Processing CSV File
by james2vegas (Chaplain) on Oct 03, 2012 at 06:43 UTC
    When you have CSV to process, reach for Text::CSV
Re: Processing CSV File
by choroba (Abbot) on Oct 03, 2012 at 07:55 UTC
    This works for me:
    #!/usr/bin/perl use warnings; use strict; open my $TEMPLATE, '<', 'rtr-template.txt' or die "Template: $!"; my @template = <$TEMPLATE>; while (<>) { chomp; my ($location, $name, $lo0ip, $frameip, $framedlci, $eth0ip) = split (/,/); my $ofile_name = $name . ".txt"; open my $OUT, '>', $ofile_name or die "Output: $!"; for my $line (@template) { $_ = $line; s/##location##/$location/; s/##rtrname##/$name/; s/##eth0-ip##/$eth0ip/; s/##loop0-ip##/$lo0ip/; s/##frame-ip##/$frameip/; s/##frame-DLCI##/$framedlci/; print {$OUT} $_; } }
    The important changes I made to your code:
    • 3-argument version of open is used.
    • Lexical filehandles are used ($FH instead of FH).
    • I am using strict.
    • The template file is not changing, so it is opened and read just once.
    • chomp is used to remove the newline from $eth0ip.
    • print is used instead of printf: using printf without a format string is useless.
    لսႽ ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ

      Thank you all, you have been extremely helpful!! I have found another problem, and that is with the CSV file.. The CSV will need to be a spreadsheet maintained in MS Excel.. The problem with that is, that when the CSV is saved in excel, it adds a hidden CR at the end of each line, so when the script runs, it fails after reading the first line.. Is it possible to amend the script above to ignore that CR?, or I am better off doing some pre processing to the CSV file to remove the CR? Thanks again!!

        Instead of chomp, use s/\r\n//.
        لսႽ ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://996981]
Approved by Athanasius
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (6)
As of 2014-07-29 07:59 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (211 votes), past polls