Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot

Reading and writing CSV

by Concept99 (Acolyte)
on Jun 04, 2003 at 14:38 UTC ( #263006=perlquestion: print w/replies, xml ) Need Help??
Concept99 has asked for the wisdom of the Perl Monks concerning the following question:

Hey Monks, I'm a bit of a newbie to Perl, so please bear with me. I have developed/pieced together a script that will calculate the total file size of a directory (including its subdirectories). What I really need to be able to do is have my script: (1) read a list of server names and their shares(ie \\server\share) FROM a csv file (2) scan those shares and report the size of each (3) output that information to a new csv file Here is my code so far:
use File::Find; use strict; my ($dir); open(OUT, "> output.csv") or die("Couldn't open output.csv\n"); #Displays total size of specified path (total includes subfolders) foreach my $dir (@ARGV) { my $total; print "\n\nWalking $dir\n\n"; find(sub { $total += -s }, $dir); $total = ($total / 1024) / 1024; $total = sprintf("%0.2f", $total); print OUT "$dir, $total,"; print "\tOutput created.\n"; } close(OUT); #Displays program syntax on screen if ($ARGV[0] eq ""){&syntax; exit 1;} $dir = $ARGV[0]; sub syntax { print " SYNTAX \tperl {dirname}\n\n Examples\n \tperl D:\MYFILES\n\t \tperl \\\\SERVER1\\PROFILES\n\n"; }
I understand that I probably need to use DBI to parse the CSV, but I am not quite sure how to go about it. I would appreciate any type of help on this. Thanks in advance!

Replies are listed 'Best First'.
Re: Reading and writing CSV
by Aristotle (Chancellor) on Jun 04, 2003 at 14:44 UTC
    DBI would be overkill here. Have a look at Text::xSV - that'll do the trick. There's also Text::CSV_XS if you prefer it. Tie::CSV might be worth a look as well.

    Makeshifts last the longest.

Re: Reading and writing CSV
by DrHyde (Prior) on Jun 04, 2003 at 14:47 UTC
    Whilst you *can* use DBI to look at CSV files with the DBD::CSV driver, you're probably better off just using the Text::CSV (pure-perl) or Text::CSV_XS (perl and C) module. Try the C version first, cos it's faster, but if you're programming on Windows you might have to use the pure perl version.

      Text::CSV_XS is available via PPM compiled for ActivePerl. Run

      ppm install Text-CSV_XS
      and off you go :-)

      Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.
         -- Rick Osborne

      Edit by castaway: Closed small tag in signature

        Ah, I thought PPM still relied on having a C compiler, but just put modules into Windows' database so you could easily uninstall them or sumfink. That's useful to know, thanks.
Re: Reading and writing CSV
by hangmanto (Monk) on Jun 04, 2003 at 15:59 UTC
    I agree with the comments that DBI would be overkill. DBI is a great module, but get some more experience with more basic perl before trying to tackle it.

    The Perl Cookbook has a recipe for parsing comma-separated data. This recipe (1.15) is taken from Mastering Regular Expressions. (My apologies if posting this is inappropriate).


    Use the procedure in Mastering Regular Expressions.

    sub parse_csv { my $text = shift; # record containing comma-separated values my @new = (); push(@new, $+) while $text =~ m{ # the first part groups the phrase inside the quotes. # see explanation of this pattern in MRE "([^\"\\]*(?:\\.[^\"\\]*)*)",? | ([^,]+),? | , }gx; push(@new, undef) if substr($text, -1,1) eq ','; return @new; # list of values that were comma-separated }
    Or use the standard Text::ParseWords module.
    use Text::ParseWords; sub parse_csv { return quotewords(",",0, $_[0]); }

    The Perl Cookbook is a must-have book for any perl programmer. If you don't have a copy, please buy one. It has helped me immensely.

Re: Reading and writing CSV
by AcidHawk (Vicar) on Jun 04, 2003 at 15:30 UTC

    As you have been told in earlier posts you probably don't really need the DBI::CSV stuff. But if you are interested look at DBD::CSV Memory Leak which has some code in it. The memory issue was sorted in a newer version of the module.

    Of all the things I've lost in my life, its my mind I miss the most.
Re: Reading and writing CSV
by wufnik (Friar) on Jun 05, 2003 at 11:43 UTC
    hola dudes, dudettes, others;

    i know this is late, but a wee scripty follows which actually does what Concept99 wants. and a little more; filecounts, directory counts for the paths to be examined. remove to taste.

    i use Adam Rich's Win32::DirSize because i gather from the Concept99's posting's paths that we are on a Win32 system and also, incidentally, it is > 50% faster, does the formatting etc and more.

    Concept99 need not read & reread mastering RegeXes, admirable tho the book is, to parse a csv. is anything simpler to parse than a csv of directory paths? well, yes, i hope that none of the paths have commas in them etc. but do i need a module to write a csv file? no sirree. perl is practical enough by itself. here goes:

    use Win32::DirSize; # >= 50% faster than file::find chomp(my @lines = (<DATA>)); my @pathsets = map { [ split ",", $_ ] } @lines; my @dstats; push @dstats,join ",",("Directory", "Size", "FileCount", "DirCount"); foreach my $pathset (@pathsets){ foreach my $path (@{$pathset}){ if (dir_size($path, my $dstat) == DS_RESULT_OK){ my $size = best_convert(my $unit, $dstat->{HighSize}, $dstat->{LowSize}); my $fcnt = $dstat->{FileCount}; my $dcnt = $dstat->{DirCount}; push @dstats, join ",",($path, sprintf("%7.4f %s", $size, $uni +t),$fcnt, $dcnt); } push @dstats, undef; } } map { print } grep { defined } @dstats; __DATA__ \\ooby\dooby$\stodge,\\ooby\ccc$\etc\lib \\foo\bar$\wufnik,\\foo\bar$\id
    which will spit out the results for each dir encountered. hope it helps,


    -- in the world of the mules there are no rules --
      As I said earlier, I'm new to Perl, so I am not quite sure how to install "Win32-DirSize-1.00.tar.gz" file I downloaded. I am running ActivePerl 5.6 on a Win2k Box. When I use nmake and try to install this, I get the following response:
      cp blib\lib\Win32\ cp blib\lib\Win32\ cp blib\lib\Win32\ cp blib\lib\Win32\ cp blib\lib\Win32\ AutoSplitting blib\lib\Win32\ (blib\lib\auto\Win32\DirSize) cp blib\lib\Win32\ C:\Perl\bin\perl.exe -IC:\Perl\lib -IC:\Perl\lib C:\Perl\lib\E +xtUtils/xs ubpp -typemap C:\Perl\lib\ExtUtils\typemap DirSize.xs > DirSize.xsc & +& C:\Perl\ bin\perl.exe -IC:\Perl\lib -IC:\Perl\lib -MExtUtils::Command -e mv Dir +Size.xsc D irSize.c Please specify prototyping behavior for DirSize.xs (see perlxs manual) cl -c -nologo -O1 -MD -Zi -DNDEBUG -DWIN32 -D_CONSOLE -DNO_ST +RICT -DHAV E_DES_FCRYPT -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DPERL_MSVCRT +_READFIX - O1 -MD -Zi -DNDEBUG -DVERSION=\"1.00\" -DXS_VERSION=\"1.00\" -IC: +\Perl\lib\ CORE DirSize.c 'cl' is not recognized as an internal or external command, operable program or batch file. NMAKE : fatal error U1077: 'C:\WINNT\system32\cmd.exe' : return code ' +0x1' Stop.
      I'd appreciate any suggestions if you have em.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://263006]
Approved by broquaint
Front-paged by broquaint
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (5)
As of 2017-01-22 23:04 GMT
Find Nodes?
    Voting Booth?
    Do you watch meteor showers?

    Results (190 votes). Check out past polls.