Re: efficient perl code to count, rank

[roboticus]

Perl_Noob2021:

I took a brief look at your code and can't immediately identify the cause of why the code is so slow. However, your code is difficult to look at.

I'd suggest first using strict and warnings to ensure that your code is clean. (I identified one minor problem that way.)

I'd next suggest that you clean up the indentation so that the overall code structure is easy enough to look at so people can identify the loops and nesting level of various statements without having to count braces. You might consider using perltidy to clean up the indentation.

Since you're running on a largish file (62gig), you might be thrashing (swapping memory in and out of disk) which can cause quite a slowdown in performance. If you're on a windows box, then using the performance view of Task manager would show your memory at 100% full and a high rate of Disk I/O if you're thrashing. If so, you'll want to look at the problem and see if you can handle it in multiple passes, such as processing a subset of the columns each time through.

Having said all that, I can't make sense of what you're trying to do in your code, so you might try making a tiny dataset as an example, and process it, and verify that it's doing what you're expecting.

...roboticus

When your only tool is a hammer, all problems look like your thumb.

[Perl_Noob2021]

The memory is about 80% using 6gig. it does work on small subset, but when i add "use strict; use warnings", it shows error and was aborted due to compilation errors. will try to provide clear codes