Syntactic Confectionery Delight PerlMonks

by Concept99 (Acolyte)
 on Jun 04, 2003 at 14:38 UTC Need Help??
Concept99 has asked for the wisdom of the Perl Monks concerning the following question:

Hey Monks, I'm a bit of a newbie to Perl, so please bear with me. I have developed/pieced together a script that will calculate the total file size of a directory (including its subdirectories). What I really need to be able to do is have my script: (1) read a list of server names and their shares(ie \\server\share) FROM a csv file (2) scan those shares and report the size of each (3) output that information to a new csv file Here is my code so far:
use File::Find;
use strict;

my ($dir); open(OUT, "> output.csv") or die("Couldn't open output.csv\n"); #Displays total size of specified path (total includes subfolders) foreach my$dir (@ARGV) {
my $total; print "\n\nWalking$dir\n\n";
find(sub { $total += -s },$dir);
$total = ($total / 1024) / 1024;
$total = sprintf("%0.2f",$total);
print OUT "$dir,$total,";
print "\tOutput created.\n";
}
close(OUT);

#Displays program syntax on screen
if ($ARGV[0] eq ""){&syntax; exit 1;}$dir = $ARGV[0]; sub syntax { print " SYNTAX \tperl dirsize.pl {dirname}\n\n Examples\n \tperl dirsize.pl D:\MYFILES\n\t \tperl dirsize.pl \\\\SERVER1\\PROFILES\n\n"; } [download] I understand that I probably need to use DBI to parse the CSV, but I am not quite sure how to go about it. I would appreciate any type of help on this. Thanks in advance! Replies are listed 'Best First'. Re: Reading and writing CSV by Aristotle (Chancellor) on Jun 04, 2003 at 14:44 UTC DBI would be overkill here. Have a look at Text::xSV - that'll do the trick. There's also Text::CSV_XS if you prefer it. Tie::CSV might be worth a look as well. Makeshifts last the longest. Re: Reading and writing CSV by DrHyde (Prior) on Jun 04, 2003 at 14:47 UTC Whilst you *can* use DBI to look at CSV files with the DBD::CSV driver, you're probably better off just using the Text::CSV (pure-perl) or Text::CSV_XS (perl and C) module. Try the C version first, cos it's faster, but if you're programming on Windows you might have to use the pure perl version. Text::CSV_XS is available via PPM compiled for ActivePerl. Run ppm install Text-CSV_XS [download] and off you go :-) Jenda Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live. -- Rick Osborne Edit by castaway: Closed small tag in signature Ah, I thought PPM still relied on having a C compiler, but just put modules into Windows' database so you could easily uninstall them or sumfink. That's useful to know, thanks. Re: Reading and writing CSV by hangmanto (Monk) on Jun 04, 2003 at 15:59 UTC I agree with the comments that DBI would be overkill. DBI is a great module, but get some more experience with more basic perl before trying to tackle it. The Perl Cookbook has a recipe for parsing comma-separated data. This recipe (1.15) is taken from Mastering Regular Expressions. (My apologies if posting this is inappropriate). BEGIN QUOTED MATERIAL: Use the procedure in Mastering Regular Expressions. sub parse_csv { my$text = shift;      # record containing comma-separated values
my @new  = ();
push(@new, $+) while$text =~ m{
# the first part groups the phrase inside the quotes.
# see explanation of this pattern in MRE
"([^\"\\]*(?:\\.[^\"\\]*)*)",?
|  ([^,]+),?
| ,
}gx;
push(@new, undef) if substr($text, -1,1) eq ','; return @new; # list of values that were comma-separated } [download] Or use the standard Text::ParseWords module. use Text::ParseWords; sub parse_csv { return quotewords(",",0,$_[0]);
}
[download]
END QUOTED MATERIAL

The Perl Cookbook is a must-have book for any perl programmer. If you don't have a copy, please buy one. It has helped me immensely.

by AcidHawk (Vicar) on Jun 04, 2003 at 15:30 UTC

As you have been told in earlier posts you probably don't really need the DBI::CSV stuff. But if you are interested look at DBD::CSV Memory Leak which has some code in it. The memory issue was sorted in a newer version of the module.

-----
Of all the things I've lost in my life, its my mind I miss the most.
by wufnik (Friar) on Jun 05, 2003 at 11:43 UTC
hola dudes, dudettes, others;

i know this is late, but a wee scripty follows which actually does what Concept99 wants. and a little more; filecounts, directory counts for the paths to be examined. remove to taste.

i use Adam Rich's Win32::DirSize because i gather from the Concept99's posting's paths that we are on a Win32 system and also, incidentally, it is > 50% faster, does the formatting etc and more.

Concept99 need not read & reread mastering RegeXes, admirable tho the book is, to parse a csv. is anything simpler to parse than a csv of directory paths? well, yes, i hope that none of the paths have commas in them etc. but do i need a module to write a csv file? no sirree. perl is practical enough by itself. here goes:

use Win32::DirSize; # >= 50% faster than file::find
chomp(my @lines = (<DATA>));
my @pathsets = map { [ split ",", $_ ] } @lines; my @dstats; push @dstats,join ",",("Directory", "Size", "FileCount", "DirCount"); foreach my$pathset (@pathsets){
foreach my $path (@{$pathset}){
if (dir_size($path, my$dstat) == DS_RESULT_OK){
my $size = best_convert(my$unit,
$dstat->{HighSize},$dstat->{LowSize});
my $fcnt =$dstat->{FileCount};
my $dcnt =$dstat->{DirCount};
push @dstats, join ",",($path, sprintf("%7.4f %s",$size, $uni +t),$fcnt, $dcnt); } push @dstats, undef; } } map { print } grep { defined } @dstats; __DATA__ \\ooby\dooby$\stodge,\\ooby\ccc$\etc\lib \\foo\bar$\wufnik,\\foo\bar\$\id

[download]
which will spit out the results for each dir encountered. hope it helps,

...wufnik

-- in the world of the mules there are no rules --
As I said earlier, I'm new to Perl, so I am not quite sure how to install "Win32-DirSize-1.00.tar.gz" file I downloaded. I am running ActivePerl 5.6 on a Win2k Box. When I use nmake and try to install this, I get the following response:
cp csvtest.pl blib\lib\Win32\csvtest.pl
cp dirsize.pl blib\lib\Win32\dirsize.pl
cp csvtest2.pl blib\lib\Win32\csvtest2.pl
cp test2.pl blib\lib\Win32\test2.pl
cp DirSize.pm blib\lib\Win32\DirSize.pm
AutoSplitting blib\lib\Win32\DirSize.pm (blib\lib\auto\Win32\DirSize)
cp try.pl blib\lib\Win32\try.pl
C:\Perl\bin\perl.exe -IC:\Perl\lib -IC:\Perl\lib C:\Perl\lib\E
+xtUtils/xs
ubpp  -typemap C:\Perl\lib\ExtUtils\typemap DirSize.xs > DirSize.xsc &
+& C:\Perl\
bin\perl.exe -IC:\Perl\lib -IC:\Perl\lib -MExtUtils::Command -e mv Dir
+Size.xsc D
irSize.c
Please specify prototyping behavior for DirSize.xs (see perlxs manual)
cl -c  -nologo -O1 -MD -Zi -DNDEBUG -DWIN32 -D_CONSOLE -DNO_ST
+RICT -DHAV
E_DES_FCRYPT -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DPERL_MSVCRT
O1 -MD -Zi -DNDEBUG    -DVERSION=\"1.00\"  -DXS_VERSION=\"1.00\"  -IC:
+\Perl\lib\
CORE  DirSize.c
'cl' is not recognized as an internal or external command,
operable program or batch file.
NMAKE : fatal error U1077: 'C:\WINNT\system32\cmd.exe' : return code '
+0x1'
Stop.
[download]
I'd appreciate any suggestions if you have em.

Create A New User
Node Status?
node history
Node Type: perlquestion [id://263006]
Approved by broquaint
Front-paged by broquaint
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (4)
As of 2018-05-24 16:11 GMT
Sections?
Information?
Find Nodes?
Leftovers?
Voting Booth?
World peace can best be achieved by:

Results (179 votes). Check out past polls.

Notices?