Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight

Reading and writing CSV

by Concept99 (Acolyte)
on Jun 04, 2003 at 14:38 UTC ( #263006=perlquestion: print w/replies, xml ) Need Help??
Concept99 has asked for the wisdom of the Perl Monks concerning the following question:

Hey Monks, I'm a bit of a newbie to Perl, so please bear with me. I have developed/pieced together a script that will calculate the total file size of a directory (including its subdirectories). What I really need to be able to do is have my script: (1) read a list of server names and their shares(ie \\server\share) FROM a csv file (2) scan those shares and report the size of each (3) output that information to a new csv file Here is my code so far:
use File::Find; use strict; my ($dir); open(OUT, "> output.csv") or die("Couldn't open output.csv\n"); #Displays total size of specified path (total includes subfolders) foreach my $dir (@ARGV) { my $total; print "\n\nWalking $dir\n\n"; find(sub { $total += -s }, $dir); $total = ($total / 1024) / 1024; $total = sprintf("%0.2f", $total); print OUT "$dir, $total,"; print "\tOutput created.\n"; } close(OUT); #Displays program syntax on screen if ($ARGV[0] eq ""){&syntax; exit 1;} $dir = $ARGV[0]; sub syntax { print " SYNTAX \tperl {dirname}\n\n Examples\n \tperl D:\MYFILES\n\t \tperl \\\\SERVER1\\PROFILES\n\n"; }
I understand that I probably need to use DBI to parse the CSV, but I am not quite sure how to go about it. I would appreciate any type of help on this. Thanks in advance!

Replies are listed 'Best First'.
Re: Reading and writing CSV
by Aristotle (Chancellor) on Jun 04, 2003 at 14:44 UTC
    DBI would be overkill here. Have a look at Text::xSV - that'll do the trick. There's also Text::CSV_XS if you prefer it. Tie::CSV might be worth a look as well.

    Makeshifts last the longest.

Re: Reading and writing CSV
by DrHyde (Prior) on Jun 04, 2003 at 14:47 UTC
    Whilst you *can* use DBI to look at CSV files with the DBD::CSV driver, you're probably better off just using the Text::CSV (pure-perl) or Text::CSV_XS (perl and C) module. Try the C version first, cos it's faster, but if you're programming on Windows you might have to use the pure perl version.

      Text::CSV_XS is available via PPM compiled for ActivePerl. Run

      ppm install Text-CSV_XS
      and off you go :-)

      Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.
         -- Rick Osborne

      Edit by castaway: Closed small tag in signature

        Ah, I thought PPM still relied on having a C compiler, but just put modules into Windows' database so you could easily uninstall them or sumfink. That's useful to know, thanks.
Re: Reading and writing CSV
by hangmanto (Monk) on Jun 04, 2003 at 15:59 UTC
    I agree with the comments that DBI would be overkill. DBI is a great module, but get some more experience with more basic perl before trying to tackle it.

    The Perl Cookbook has a recipe for parsing comma-separated data. This recipe (1.15) is taken from Mastering Regular Expressions. (My apologies if posting this is inappropriate).


    Use the procedure in Mastering Regular Expressions.

    sub parse_csv { my $text = shift; # record containing comma-separated values my @new = (); push(@new, $+) while $text =~ m{ # the first part groups the phrase inside the quotes. # see explanation of this pattern in MRE "([^\"\\]*(?:\\.[^\"\\]*)*)",? | ([^,]+),? | , }gx; push(@new, undef) if substr($text, -1,1) eq ','; return @new; # list of values that were comma-separated }
    Or use the standard Text::ParseWords module.
    use Text::ParseWords; sub parse_csv { return quotewords(",",0, $_[0]); }

    The Perl Cookbook is a must-have book for any perl programmer. If you don't have a copy, please buy one. It has helped me immensely.

Re: Reading and writing CSV
by AcidHawk (Vicar) on Jun 04, 2003 at 15:30 UTC

    As you have been told in earlier posts you probably don't really need the DBI::CSV stuff. But if you are interested look at DBD::CSV Memory Leak which has some code in it. The memory issue was sorted in a newer version of the module.

    Of all the things I've lost in my life, its my mind I miss the most.
Re: Reading and writing CSV
by wufnik (Friar) on Jun 05, 2003 at 11:43 UTC
    hola dudes, dudettes, others;

    i know this is late, but a wee scripty follows which actually does what Concept99 wants. and a little more; filecounts, directory counts for the paths to be examined. remove to taste.

    i use Adam Rich's Win32::DirSize because i gather from the Concept99's posting's paths that we are on a Win32 system and also, incidentally, it is > 50% faster, does the formatting etc and more.

    Concept99 need not read & reread mastering RegeXes, admirable tho the book is, to parse a csv. is anything simpler to parse than a csv of directory paths? well, yes, i hope that none of the paths have commas in them etc. but do i need a module to write a csv file? no sirree. perl is practical enough by itself. here goes:

    use Win32::DirSize; # >= 50% faster than file::find chomp(my @lines = (<DATA>)); my @pathsets = map { [ split ",", $_ ] } @lines; my @dstats; push @dstats,join ",",("Directory", "Size", "FileCount", "DirCount"); foreach my $pathset (@pathsets){ foreach my $path (@{$pathset}){ if (dir_size($path, my $dstat) == DS_RESULT_OK){ my $size = best_convert(my $unit, $dstat->{HighSize}, $dstat->{LowSize}); my $fcnt = $dstat->{FileCount}; my $dcnt = $dstat->{DirCount}; push @dstats, join ",",($path, sprintf("%7.4f %s", $size, $uni +t),$fcnt, $dcnt); } push @dstats, undef; } } map { print } grep { defined } @dstats; __DATA__ \\ooby\dooby$\stodge,\\ooby\ccc$\etc\lib \\foo\bar$\wufnik,\\foo\bar$\id
    which will spit out the results for each dir encountered. hope it helps,


    -- in the world of the mules there are no rules --
      As I said earlier, I'm new to Perl, so I am not quite sure how to install "Win32-DirSize-1.00.tar.gz" file I downloaded. I am running ActivePerl 5.6 on a Win2k Box. When I use nmake and try to install this, I get the following response:
      cp blib\lib\Win32\ cp blib\lib\Win32\ cp blib\lib\Win32\ cp blib\lib\Win32\ cp blib\lib\Win32\ AutoSplitting blib\lib\Win32\ (blib\lib\auto\Win32\DirSize) cp blib\lib\Win32\ C:\Perl\bin\perl.exe -IC:\Perl\lib -IC:\Perl\lib C:\Perl\lib\E +xtUtils/xs ubpp -typemap C:\Perl\lib\ExtUtils\typemap DirSize.xs > DirSize.xsc & +& C:\Perl\ bin\perl.exe -IC:\Perl\lib -IC:\Perl\lib -MExtUtils::Command -e mv Dir +Size.xsc D irSize.c Please specify prototyping behavior for DirSize.xs (see perlxs manual) cl -c -nologo -O1 -MD -Zi -DNDEBUG -DWIN32 -D_CONSOLE -DNO_ST +RICT -DHAV E_DES_FCRYPT -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DPERL_MSVCRT +_READFIX - O1 -MD -Zi -DNDEBUG -DVERSION=\"1.00\" -DXS_VERSION=\"1.00\" -IC: +\Perl\lib\ CORE DirSize.c 'cl' is not recognized as an internal or external command, operable program or batch file. NMAKE : fatal error U1077: 'C:\WINNT\system32\cmd.exe' : return code ' +0x1' Stop.
      I'd appreciate any suggestions if you have em.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://263006]
Approved by broquaint
Front-paged by broquaint
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (4)
As of 2018-05-24 16:11 GMT
Find Nodes?
    Voting Booth?