extracting data from CSV files and making hash of hashes

[reez]

Hello Perl Monks,

What am i trying to do is extract data from directory of CSV files, in which columns are separated with ";"

eg.Table;M
     Data;O
     ......

The ouput file must be made as this structure:

%hMFD = (
         Name_of_the_file_without_ext => {
                 Fieldname_1(column A) => 'value M or O(column B)',
                 Fieldname_2(column A) => 'value M or O(column B)',
    .
    .
    .
                 Fieldname_x(column A) => 'value M or O(column B)',
                },
         Name_of_the_next_file_without_ext => {
                 Fieldname_1(column A) => 'value M or O(column B)',
                 Fieldname_2(column A) => 'value M or O(column B)',
    .
    .
    .
                 Fieldname_x(column A) => 'value M or O(column B)',
                },
     );

here what i did:

#!c:\perl\ -w
use Text::CSV;
opendir(KAT, "$ARGV[0]"); 
@fls=grep {/\.csv$/} readdir KAT;
closedir KAT;
foreach $fl(@fls) {
    $fl=~s/\..*//;
    print "$fl\n";
    }    
@files=<C:\\experl\\csv\\*.csv>;
    foreach $file (@files){
        print "$file\n";
        }
foreach $file (@files){
        my $csv = Text::CSV->new();
        open (CSV, "<", $file) or die $!;
            while (<CSV>) {
            if ($csv->parse($_)) {
            my @columns = $csv->fields();
            print "@columns\n";
            } else {
            my $err = $csv->error_input;
            print "Failed to parse line: $err";
            }
            }
        close CSV;
        }

still it is only making list of files as an array, and reads all data without separating them, i have no clue how to combine this into hash of hashes, i would be grateful for some advice, this task is to hard for me at this point

[toolic]

One problem is that you need to change the separator character for Text::CSV to ";". Another problem is that you are not populating your HASHES OF HASHES data structure. I created 2 tiny CSV files for myself, named 1.csv and 2.csv.

use strict;
use warnings;
use Data::Dumper;
use Text::CSV;
use File::Basename;

my @files = glob '*.csv';

my %hMFD;
foreach my $file (@files){
        my $file_base = fileparse($file, qr/\.csv/);
        my $csv = Text::CSV->new({sep_char => ';'});
        open (CSV, "<", $file) or die $!;
        while (<CSV>) {
                if ($csv->parse($_)) {
                    my @columns = $csv->fields();
                    $hMFD{$file_base}{$columns[0]} = $columns[1];
                }
                else {
                    my $err = $csv->error_input;
                    print "Failed to parse line: $err";
                }
        }
        close CSV;
}

print Dumper(\%hMFD);

__END__

$VAR1 = {
          '1' => {
                   'Table' => 'M',
                   'Data' => 'O'
                 },
          '2' => {
                   'Table' => 'X',
                   'Data' => 'Y'
                 }
        };

[reez]

This works just fine, I`ve added lines to print dumper into a file, still is there easy way to get this hash sorted in a order the files appear in directory?Thanks a lot.

[toolic]

Two ways I can think of:
• Load the data into a tied hash using Tie::IxHash
• Instead of a hash, use ARRAYS OF HASHES

[intel]

First of all, it would be helpful to see your input data.

I'm not very familiar with the CSV module, because parsing CSVs is something that Perl does very well natively.

Everything is fine up until the  my $csv = Text::CSV->new(); At that point you should probably be doing:

open (CSV, "<", $file) or die $!;
  my %subhash;
    for my $line ( split "\n" ) {
        my( $key, $value ) = split /\s+;\s+/, $line;
        $subhash{ $key } = $value;
    }
    $hash{ $subhash{ $file } } = \%subhash;
}

This is nearly identical to a question answered by BrowserUK in this node: creating hash of hashes from input file