Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

extracting data from CSV files and making hash of hashes

by reez (Novice)
on Jul 20, 2010 at 17:11 UTC ( #850466=perlquestion: print w/replies, xml ) Need Help??
reez has asked for the wisdom of the Perl Monks concerning the following question:

Hello Perl Monks,

What am i trying to do is extract data from directory of CSV files, in which columns are separated with ";"

eg.Table;M Data;O ......
The ouput file must be made as this structure:
%hMFD = ( Name_of_the_file_without_ext => { Fieldname_1(column A) => 'value M or O(column B)', Fieldname_2(column A) => 'value M or O(column B)', . . . Fieldname_x(column A) => 'value M or O(column B)', }, Name_of_the_next_file_without_ext => { Fieldname_1(column A) => 'value M or O(column B)', Fieldname_2(column A) => 'value M or O(column B)', . . . Fieldname_x(column A) => 'value M or O(column B)', }, );
here what i did:
#!c:\perl\ -w use Text::CSV; opendir(KAT, "$ARGV[0]"); @fls=grep {/\.csv$/} readdir KAT; closedir KAT; foreach $fl(@fls) { $fl=~s/\..*//; print "$fl\n"; } @files=<C:\\experl\\csv\\*.csv>; foreach $file (@files){ print "$file\n"; } foreach $file (@files){ my $csv = Text::CSV->new(); open (CSV, "<", $file) or die $!; while (<CSV>) { if ($csv->parse($_)) { my @columns = $csv->fields(); print "@columns\n"; } else { my $err = $csv->error_input; print "Failed to parse line: $err"; } } close CSV; }

still it is only making list of files as an array, and reads all data without separating them, i have no clue how to combine this into hash of hashes, i would be grateful for some advice, this task is to hard for me at this point

Replies are listed 'Best First'.
Re: extracting data from CSV files and making hash of hashes
by toolic (Bishop) on Jul 20, 2010 at 18:54 UTC
    One problem is that you need to change the separator character for Text::CSV to ";". Another problem is that you are not populating your HASHES OF HASHES data structure. I created 2 tiny CSV files for myself, named 1.csv and 2.csv.
    use strict; use warnings; use Data::Dumper; use Text::CSV; use File::Basename; my @files = glob '*.csv'; my %hMFD; foreach my $file (@files){ my $file_base = fileparse($file, qr/\.csv/); my $csv = Text::CSV->new({sep_char => ';'}); open (CSV, "<", $file) or die $!; while (<CSV>) { if ($csv->parse($_)) { my @columns = $csv->fields(); $hMFD{$file_base}{$columns[0]} = $columns[1]; } else { my $err = $csv->error_input; print "Failed to parse line: $err"; } } close CSV; } print Dumper(\%hMFD); __END__ $VAR1 = { '1' => { 'Table' => 'M', 'Data' => 'O' }, '2' => { 'Table' => 'X', 'Data' => 'Y' } };

      This works just fine, I`ve added lines to print dumper into a file, still is there easy way to get this hash sorted in a order the files appear in directory?Thanks a lot.

Re: extracting data from CSV files and making hash of hashes
by intel (Beadle) on Jul 20, 2010 at 19:03 UTC
    First of all, it would be helpful to see your input data.

    I'm not very familiar with the CSV module, because parsing CSVs is something that Perl does very well natively.

    Everything is fine up until the  my $csv = Text::CSV->new(); At that point you should probably be doing:

    open (CSV, "<", $file) or die $!; my %subhash; for my $line ( split "\n" ) { my( $key, $value ) = split /\s+;\s+/, $line; $subhash{ $key } = $value; } $hash{ $subhash{ $file } } = \%subhash; }
    This is nearly identical to a question answered by BrowserUK in this node: creating hash of hashes from input file

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://850466]
Approved by Corion
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (2)
As of 2018-04-22 09:22 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Notices?