my $SIZE = 1000;
sub create_matrix {
my ($filename) = @_;
open my $OUT, '>', $filename or die $!;
for my $i (1 .. $SIZE) {
for my $j ( 1 .. $SIZE ) {
print {$OUT} $i <= $j ? $i * $j : 'NA';
print {$OUT} ' ' unless $SIZE == $j;
}
print {$OUT} "\n";
}
}
($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord
}map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
| [reply] [Watch: Dir/Any] [d/l] [select] |
You and others in this thread silently assume "matrix" meaning a somehow delimited data file. What if the file looks like this:
0001 1202 3030 ...
8491 9382 9381 ...
In such a fixed lenght case you don't need any memory (well, kinda) and can just do the task by seek()ing the appropriate positions on disk.
We won't know unless the OP tells us.
holli
You can lead your users to water, but alas, you cannot drown them.
| [reply] [Watch: Dir/Any] [d/l] |
I originally started with
sub fill_matrix {
my ($in) = @_;
open my $IN, '<', $in or die $!;
my @index = (0);
push @index, tell $IN while <$IN>;
pop @index;
for my $line_no (0 .. $#index) {
print STDERR "$line_no\r";
for my $idx (0 .. $line_no - 1) {
seek $IN, $index[$idx], 0;
my $line = <$IN>;
print +(split ' ', $line, $line_no + 2)[$line_no], ' ';
}
seek $IN, $index[$line_no], 0;
my $line = <$IN>;
print +(split ' ', $line, $line_no + 1)[-1];
}
}
but it was much slower: 28s for SIZE 1000, 280s for SIZE 2000.
($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord
}map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
| [reply] [Watch: Dir/Any] [d/l] [select] |
It helps if you combine several columns into each of the tempfiles. But yeah, slinging around this much data is... challenging.
| [reply] [Watch: Dir/Any] |