a217 has asked for the wisdom of the Perl Monks concerning the following question:
Hello, I need help speeding up my perl code and was hoping that someone could give me a few pointers.
My goal of this code is to print out lines from chr1.txt (see below) only when the ID (first column) is shared by chr1.coverage.txt as well.
Hence, I decided to save each unique ID from the "coverage" file as a hash key so that I could then sort through them and print out the corresponding lines from chr1.txt
The problem with this method I currently use is the speed of the program. I am using a nested foreach loop within a while loop, and when there are thousands of unique keys for the "coverage" file the process can take a long time.
Given that my data is well-sorted (there is only 1 chr for each file, in this case chr1, and the numbers are numerically sorted) I know that there should be a faster way to achieve the same results, but I have been struggling to find a working solution. Any help you can provide would be greatly appreciated.
Below is my current perl code
#!/usr/bin/perl -w # perl.ID.match.pl use strict; use warnings; open(KEY, "<chr1.coverage.txt") or die "error reading file"; my %Chr; my (@Key,@line); while (my $key = <KEY>) { chomp $key; @Key=split("\t",$key); $Chr{$Key[0]} = undef; } open(FULL, "<chr1.txt") or die "error reading file"; open(OUT, ">chr1.match.txt") or die "error reading file"; while (my $line=<FULL>) { chomp $line; @line=split("\t",$line); foreach my $key (sort keys %Chr) { if ($line[0] eq $key) { print OUT "$line\n"; } } } close KEY; close FULL; close OUT; exit;
This is chr1.coverage.txt sample data
chr1:496-588 44 0 chr1:496-588 44 0 chr1:496-588 44 0 chr1:496-588 44 0 chr1:496-588 44 0 chr1:496-588 44 0 chr1:496-588 43 0 chr1:496-588 43 0 chr1:496-588 39 0 chr1:496-588 39 0 chr1:496-588 38 0 chr1:496-588 37 0 chr1:126982-127030 56 0 chr1:126982-127030 56 0 chr1:126982-127030 56 0 chr1:126982-127030 56 0.0892857 chr1:126982-127030 56 0 chr1:126982-127030 56 0 chr1:126982-127030 56 0 chr1:126982-127030 56 0
This is chr1.txt sample data
chr1:496-588 0 0 chr1:496-588 0 0 chr1:496-588 0 0 chr1:496-588 0 0 chr1:496-588 0 0 chr1:496-588 0 0 chr1:496-588 43 0 chr1:496-588 43 0 chr1:496-588 39 0 chr1:496-588 39 0 chr1:496-588 38 0 chr1:496-588 0 0 chr1:496-588 0 0 chr1:496-588 39 0 chr1:496-588 0 0 chr1:648-719 0 0 chr1:648-719 0 0 chr1:648-719 0 0 chr1:648-719 0 0 chr1:648-719 0 0 chr1:648-719 0 0 chr1:648-719 0 0 chr1:648-719 0 0 chr1:648-719 0 0 chr1:720-795 0 0 chr1:720-795 0 0 chr1:720-795 0 0 chr1:720-795 0 0 chr1:720-795 0 0 chr1:720-795 0 0 chr1:720-795 0 0 chr1:720-795 0 0 chr1:720-795 0 0 chr1:720-795 0 0 chr1:720-795 0 0 chr1:126982-127030 0 0 chr1:126982-127030 56 0 chr1:126982-127030 56 0 chr1:126982-127030 56 0.0892857 chr1:126982-127030 56 0 chr1:126982-127030 56 0 chr1:126982-127030 56 0 chr1:126982-127030 56 0 chr1:126982-127030 0 0 chr1:126982-127030 0 0 chr1:126982-127030 0 0 chr1:126982-127030 0 0 chr1:126982-127030 0 0 chr1:126982-127030 0 0
This is the sample output chr1.match.txt
chr1:496-588 0 0 chr1:496-588 0 0 chr1:496-588 0 0 chr1:496-588 0 0 chr1:496-588 0 0 chr1:496-588 0 0 chr1:496-588 43 0 chr1:496-588 43 0 chr1:496-588 39 0 chr1:496-588 39 0 chr1:496-588 38 0 chr1:496-588 0 0 chr1:496-588 0 0 chr1:496-588 39 0 chr1:496-588 0 0 chr1:126982-127030 0 0 chr1:126982-127030 56 0 chr1:126982-127030 56 0 chr1:126982-127030 56 0.0892857 chr1:126982-127030 56 0 chr1:126982-127030 56 0 chr1:126982-127030 56 0 chr1:126982-127030 56 0 chr1:126982-127030 0 0 chr1:126982-127030 0 0 chr1:126982-127030 0 0 chr1:126982-127030 0 0 chr1:126982-127030 0 0 chr1:126982-127030 0 0
Also note that although the data may look unnecessarily repetitious, it is actually a cropped version of another file with more columns so that each specific line represents a different data point.
|
---|
Replies are listed 'Best First'. | |
---|---|
Re: help speeding up my perl code
by BrowserUk (Patriarch) on Aug 31, 2011 at 20:12 UTC | |
Re: help speeding up my perl code
by MidLifeXis (Monsignor) on Aug 31, 2011 at 20:12 UTC | |
by SuicideJunkie (Vicar) on Sep 01, 2011 at 13:10 UTC | |
Re: help speeding up my perl code
by jwkrahn (Abbot) on Aug 31, 2011 at 20:27 UTC | |
Re: help speeding up my perl code
by thezip (Vicar) on Aug 31, 2011 at 20:10 UTC | |
by a217 (Novice) on Aug 31, 2011 at 20:18 UTC | |
by BrowserUk (Patriarch) on Aug 31, 2011 at 20:13 UTC | |
by thezip (Vicar) on Aug 31, 2011 at 20:21 UTC |