Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight

Re: Open multiple file handles?

by GrandFather (Sage)
on May 07, 2011 at 06:53 UTC ( #903505=note: print w/replies, xml ) Need Help??

in reply to Open multiple file handles?

As suggested in the preceding replies, you've grabbed the wrong end of the stick and run with it. A better solution (for modest size data sets anyway) than the one you are struggling to implement is to read the files one at a time and merge the results in memory. Consider:

#!/usr/bin/perl use warnings; use strict; # First set up the sample files my @fileContents = ('a 1 b 2 c 3', 'a 5 d 3', 'a 1 x 4'); @ARGV = (); for (my $fileNum = 1; @fileContents; ++$fileNum) { my $fileName = "file$fileNum.txt"; open my $fileOut, '>', $fileName or die "Failed to create $fileNam +e: $!\n"; push @ARGV, $fileName; print {$fileOut} shift @fileContents; } # Now for the "real" code my %data; my $maxFile = @ARGV - 1; while (<>) { my %newData = split; $data{$_}[$maxFile - @ARGV] = $newData{$_} for keys %newData; } for my $key (sort keys %data) { $data{$key}[$maxFile] ||= 0; $_ ||= 0 for @{$data{$key}}; print "$key @{$data{$key}}\n"; }


a 1 5 1 b 2 0 0 c 3 0 0 d 0 3 0 x 0 0 4

Note that most of the "tricky" code is to deal with getting the output data in the required format accounting for "missing" elements.

True laziness is hard work

Replies are listed 'Best First'.
Re^2: Open multiple file handles?
by Anonymous Monk on May 07, 2011 at 20:34 UTC
    Dear all,

    Thanks to you for all your suggestions. I've started teaching myself PERL since 3 weeks, and had never come across the need to process multiple input files. Based on your responses, am I right in understanding the following?

    1. I do NOT have to open as many file handles as the number of ARGV elements from command line?

    2. I can build a merge of the lists in each input file into one held in memory and not an actual file

    3. It is possible to compare this merge list in memory to the individual file lists from input? <\p>

    Except I do not still understand how to do this comparison when opening multiple files with ONLY 1 file handle! I am a little lost here...My understanding is that the final merge list can be compiled only after reading the contents of the last input file, at which time the common file handle is pointing to the last file. So how can a comparison of the merge list to ANYTHING but the last file be made?

    I know I am being naive, but I am sure Your Holiness' the Monks will be kind to a new initiate :)

    Thanks to all those who wrote down partial and even FULL scripts, it is very sweet and kind of you. Since I am learning and very much a newbie, I would appreciate and benefit much more if you could point out an outline of logic and refer me to the operations/syntax that I should teach mysef to implement the most suitable algo for my work

    Also, my example might have been misleading, my input files are not numbered, they have unrelated alphabetical names. And each of the files is not an array, it is more in the format of a 2 column Excel sheet, but as txt file. Entries in each line separated by tab, lines themselves separated by newline

    Thanks again, I aspire to be as helpful and patient as all of you, one day! :)

      I know I am being naive

      Naivety we are fine with. Foremost this is a place of learning and naivety is to be expected. What makes us grumpy or inclined to ignore supplicants is when we offer advice that seems to be ignored. In this case you have had a number of replies that each address points 1 - 3. Go back and re-read them, and remember that what we tell you three times is true!

      If you have specific questions about some code you have been offered ask them in a reply to the node the code was provided to. That makes it easier for people who come afterwards to follow the different threads of "conversation".

      Oh, and if you simplify your problem to the point of irrelevance, the answers you get will be irrelevant too!

      True laziness is hard work

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://903505]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (9)
As of 2019-10-18 13:55 GMT
Find Nodes?
    Voting Booth?