Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister

Re^2: Find overlap

by linseyr (Acolyte)
on Oct 14, 2012 at 01:06 UTC ( #998894=note: print w/ replies, xml ) Need Help??

in reply to Re: Find overlap
in thread Find overlap

Hi, No there are three columns per line: chr, start and end. There are 4 different files which contain these three columns. In total there are 24 chromosomes, and every files contains a different number of lines per chromosome. Every file has aprox 240,000 lines. so about 10,000 per chromosome.

Comment on Re^2: Find overlap
Replies are listed 'Best First'.
Re^3: Find overlap
by Anonymous Monk on Oct 14, 2012 at 02:47 UTC

    i'm not sure if i get the problem right but this script will find the min of all values in column 2 of all given files and the max of all values in column 3.

    you just call it with the file names as arguments.

    #!/usr/bin/perl -w use strict; my $min = undef; my $max = undef; foreach (@ARGV) { open(F, "< $_"); while(my $line = <F>) { chop($line); $line =~ s/[\ \t]+/:/g; my @p = split(/:/,$line); if(!defined($min)) { $min = $p[1]; } if(!defined($max)) { $max = $p[2]; } if($p[1] < $min) { $min = $p[1]; } if($p[2] > $max) { $max = $p[2]; } } close(F); } print "$min $max\n";
Re^3: Find overlap
by Anonymous Monk on Oct 14, 2012 at 03:37 UTC
    but if you need the first column as well, and if the columns are tab separated, this script would give you the minima of chr1..chrN of column2 of all given files and the maxima of chr1..chrN of column3 of all given files. do you need the file names as well? and i hope i understood the problem right
    #!/usr/bin/perl -w use strict; my %minmax; foreach (@ARGV) { open(F, "< $_"); while(my $line = <F>) { chop($line); my @p = split(/\t/,$line); if(!exists( $minmax{ $p[0] }{ min } )) { $minmax{ $p[0] }{ min } = $p[1]; } if($p[1] < $minmax{ $p[0] }{ min }) { $minmax{ $p[0] }{ min } = $p[1]; } if(!exists( $minmax{ $p[0] } { max } )) { $minmax{ $p[0] }{ max } = $p[2]; } if($p[2] > $minmax{ $p[0] }{ max }) { $minmax{ $p[0] }{ max } = $p[2]; } } close(F); } foreach (keys %minmax) { print "$_: min: $minmax{$_}{min} max: $minmax{$_}{max}\n"; }
    for the given input:


    chr1 4 60 chr2 2 40 chr3 4 90 chr1 5 40
    chr2 1 30 chr1 6 20 chr4 9 100
    chr1 2 20 chr2 2 90 chr1 6 20 chr4 4 30
    chr2 4 90 chr3 3 90 chr2 4 90 chr4 3 90 chr2 4 30
    it would output:
    chr1: min: 2 max: 60 chr2: min: 1 max: 90 chr3: min: 3 max: 90 chr4: min: 3 max: 100

      Your solution is an excellent use of a HoH! Here are a few items to consider within your foreach:

      foreach (@ARGV) { open my $fh, '<', $_ or die $!; while ( my $line = <$fh> ) { next unless my @p = $line =~ /^(chr.+?)\t+(\d+)\t+(\d+)/; $minmax{ $p[0] }{min} //= $p[1]; $minmax{ $p[0] }{max} //= $p[2]; if ( $p[1] < $minmax{ $p[0] }{min} ) { $minmax{ $p[0] }{min} = $p[1]; } if ( $p[2] > $minmax{ $p[0] }{max} ) { $minmax{ $p[0] }{max} = $p[2]; } } close $fh; }
      • A three-argument open is used
      • A lexically-scoped variable is used for the file handle
      • or die $! checks for open errors
      • The regex line:
        • Skips possible comment and track definition lines in the BED files
        • And then makes using chomp and split unnecessary
      • Uses defined-or-equals (//=) to replace the !exists

      Hope this is helpful.

      Hi, Thank you for your answer, but this wasn't the problem. I dont want to find just the minima and maxima of a chromosome. I want to find the overlapping regions and find the minima and maxima of that. So when there are lines in the 4 files like this:
      file1: chr1 100 500 chr1 25 50 file2: chr1 10 50 chr1 60 80 file3: chr1 12 40 chr1 41 45 file4: chr1 20 45 chr1 48 80
      The line of the first file does not overlap with the other 3 files, so i dont want to find the minima and maxima. I want to find only the minima and maxima of the regions in which the lines of all four files overlap each other. So the minima and maxima of the other regions in the files. Minima = 10 maxima = 80
        Here is a solution that won't include non-overlapping regions. It uses the Sort::Naturally and List::Util modules.

        It also prints out the number of overlapping records, (merged 3 says 3 records overlapped this range). You can change the print statement to not print this if you (probably) want.

        For each chromosome, the records are sorted in order from the smallest beginning position to the largest beginning position. This simplifies the logic and makes a solution possible. The code that does this is:

        my ($first, @in_order) =  sort {$a->[0] <=> $b->[0]} @{ $data{$chr} };

        And, this is the program.

        #!/usr/bin/perl use strict; use warnings; use Sort::Naturally qw/ nsort /; use List::Util qw / max /; @ARGV = qw/ 148N.txt 162N.txt 174N.txt 175N.txt /; my %data; while (<>) { my ($chr, @start_stop) = split; push @{ $data{$chr} }, \@start_stop; } for my $chr (nsort keys %data) { my ($first, @in_order) = sort {$a->[0] <=> $b->[0]} @{ $data{$chr +} }; my ($lo, $hi) = @$first; my $merged; for my $aref (@in_order) { # array reference my ($start, $stop) = @$aref; if ($start <= $hi) { $hi = max $hi, $stop; $merged++; } else { printf "%s %s %s merged: %s\n", $chr, $lo, $hi, $merged + +1 if $merged; #print "$chr $lo $hi\n" if $merged; ($lo, $hi) = ($start, $stop); $merged = 0; } } printf "%s %s %s merged: %s\n", $chr, $lo, $hi, $merged + 1 if $me +rged; #print "$chr $lo $hi\n" if $merged; }
        Update: Just a note - you don't need the 'nsort' function from the Sort::Naturally module for the program to process correctly. This would merely order your chromosomes in your output.
        chr1 xx xx chr2 xx xx chr4 xx xx ...
        And, if you can't use List::Util for the max function, you could easily define it yourself. I used the module just to save some additional code.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://998894]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (4)
As of 2016-02-06 16:45 GMT
Find Nodes?
    Voting Booth?

    How many photographs, souvenirs, artworks, trophies or other decorative objects are displayed in your home?

    Results (233 votes), past polls