Re^2: Find overlap

[linseyr]

Hi, No there are three columns per line: chr, start and end. There are 4 different files which contain these three columns. In total there are 24 chromosomes, and every files contains a different number of lines per chromosome. Every file has aprox 240,000 lines. so about 10,000 per chromosome.

[Anonymous Monk]

hi,

i'm not sure if i get the problem right but this script will find the min of all values in column 2 of all given files and the max of all values in column 3.

you just call it with the file names as arguments.

#!/usr/bin/perl -w

use strict;

my $min = undef;
my $max = undef;
foreach (@ARGV) {
    open(F, "< $_");
    while(my $line = <F>) {
        chop($line);
        $line =~ s/[\ \t]+/:/g;
        my @p = split(/:/,$line);
        if(!defined($min)) {
            $min = $p[1];
        }
        if(!defined($max)) {
            $max = $p[2];
        }
        if($p[1] < $min) {
            $min = $p[1];
        }
        if($p[2] > $max) {
            $max = $p[2];
        }
    }
    close(F);
}
print "$min $max\n";

[Anonymous Monk]

but if you need the first column as well, and if the columns are tab separated, this script would give you the minima of chr1..chrN of column2 of all given files and the maxima of chr1..chrN of column3 of all given files. do you need the file names as well? and i hope i understood the problem right

#!/usr/bin/perl -w

use strict;

my %minmax;


foreach (@ARGV) {
    open(F, "< $_");
    while(my $line = <F>) {
        chop($line);
        my @p = split(/\t/,$line);

        if(!exists( $minmax{ $p[0] }{ min } )) {
            $minmax{ $p[0] }{ min } = $p[1];
        }
        if($p[1] < $minmax{ $p[0] }{ min }) {
            $minmax{ $p[0] }{ min } = $p[1];
        }

        if(!exists( $minmax{ $p[0] } { max } )) {
            $minmax{ $p[0] }{ max } = $p[2];
        }
        if($p[2] > $minmax{ $p[0] }{ max }) {
            $minmax{ $p[0] }{ max } = $p[2];
        }

    }
    close(F);
}
foreach (keys %minmax) {
    print "$_: min: $minmax{$_}{min} max: $minmax{$_}{max}\n";
}

for the given input:

file1:

chr1    4   60
chr2    2   40
chr3    4   90
chr1    5   40

file2:

chr2    1   30
chr1    6   20
chr4    9   100

file3:

chr1    2   20
chr2    2   90
chr1    6   20
chr4    4   30

file4:

chr2    4   90
chr3    3   90
chr2    4   90
chr4    3   90
chr2    4   30

it would output:

chr1: min: 2 max: 60
chr2: min: 1 max: 90
chr3: min: 3 max: 90
chr4: min: 3 max: 100

[Kenosis]

Your solution is an excellent use of a HoH! Here are a few items to consider within your foreach:

foreach (@ARGV) {
    open my $fh, '<', $_ or die $!;
    while ( my $line = <$fh> ) {
        next unless my @p = $line =~ /^(chr.+?)\t+(\d+)\t+(\d+)/;

        $minmax{ $p[0] }{min} //= $p[1];
        $minmax{ $p[0] }{max} //= $p[2];

        if ( $p[1] < $minmax{ $p[0] }{min} ) {
            $minmax{ $p[0] }{min} = $p[1];
        }

        if ( $p[2] > $minmax{ $p[0] }{max} ) {
            $minmax{ $p[0] }{max} = $p[2];
        }
    }
    close $fh;
}

• A three-argument open is used
• A lexically-scoped variable is used for the file handle
• or die $! checks for open errors
• The regex line:
• Skips possible comment and track definition lines in the BED files
• And then makes using chomp and split unnecessary
• Uses defined-or-equals (//=) to replace the !exists

Hope this is helpful.

[Anonymous Monk]

Hi, Thank you for your answer, but this wasn't the problem. I dont want to find just the minima and maxima of a chromosome. I want to find the overlapping regions and find the minima and maxima of that. So when there are lines in the 4 files like this:

file1:
chr1    100    500
chr1    25      50
file2:
chr1    10       50
chr1    60       80
file3:
chr1    12       40
chr1    41       45
file4:
chr1     20      45
chr1     48      80

The line of the first file does not overlap with the other 3 files, so i dont want to find the minima and maxima. I want to find only the minima and maxima of the regions in which the lines of all four files overlap each other. So the minima and maxima of the other regions in the files. Minima = 10 maxima = 80

[Cristoforo]

Here is a solution that won't include non-overlapping regions. It uses the Sort::Naturally and List::Util modules.

It also prints out the number of overlapping records, (merged 3 says 3 records overlapped this range). You can change the print statement to not print this if you (probably) want.

For each chromosome, the records are sorted in order from the smallest beginning position to the largest beginning position. This simplifies the logic and makes a solution possible. The code that does this is:

my ($first, @in_order) =  sort {$a->[0] <=> $b->[0]} @{ $data{$chr} };

And, this is the program.

#!/usr/bin/perl
use strict;
use warnings;
use Sort::Naturally qw/ nsort /;
use List::Util qw / max /;

@ARGV = qw/ 148N.txt 162N.txt 174N.txt 175N.txt /;
my %data;

while (<>) {
    my ($chr, @start_stop) = split;    
    push @{ $data{$chr} }, \@start_stop;
}

for my $chr (nsort keys %data) {
    my ($first, @in_order) =  sort {$a->[0] <=> $b->[0]} @{ $data{$chr
+} };
    
    my ($lo, $hi) = @$first;
    
    my $merged;
    
    for my $aref (@in_order) { # array reference
        my ($start, $stop) = @$aref;
        if ($start <= $hi) {
            $hi = max $hi, $stop;
            $merged++;
        }
        else {
            printf "%s %s %s merged: %s\n", $chr, $lo, $hi, $merged + 
+1 if $merged;
            #print "$chr $lo $hi\n" if $merged;
            ($lo, $hi) = ($start, $stop);
            $merged = 0;    
        }    
    }
    printf "%s %s %s merged: %s\n", $chr, $lo, $hi, $merged + 1 if $me
+rged;
    #print "$chr $lo $hi\n" if $merged;
}

Update: Just a note - you don't need the 'nsort' function from the Sort::Naturally module for the program to process correctly. This would merely order your chromosomes in your output.

chr1 xx xx
chr2 xx xx
chr4 xx xx
...

And, if you can't use List::Util for the max function, you could easily define it yourself. I used the module just to save some additional code.