Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

Re^3: average a column in tab-delimited file

by Anonymous Monk
on Jan 31, 2012 at 14:51 UTC ( [id://950995]=note: print w/replies, xml ) Need Help??


in reply to Re^2: average a column in tab-delimited file
in thread average a column in tab-delimited file

Please explain in words, how you would do it, using paper and pencil

Replies are listed 'Best First'.
Re^4: average a column in tab-delimited file
by garyboyd (Acolyte) on Jan 31, 2012 at 15:36 UTC

    well I've written something that almost does the job, except it goes wrong on the very last line of the file. I read the file into an array and calculate the averages which I then put into a hash and then iterate through the array and lookup values from the hashes. I'm sure there's a more elegant way to do it though....

    #!/usr/bin/perl # Usage: perl average_table.pl -i <input file> use strict; use warnings; use Getopt::Long; my $infile; my @fields; my @array; my @values; my %hash; my $add_fst; my @out; my @count; my $i=1; my @final; my %pval; my %fst; my $prev_loc; my $prev_pval; my $prev_fst; my $prev_join; GetOptions ( "infile=s" => \$infile, ); open INFILE, "<$infile" or die $!; @fields = <INFILE>; foreach (@fields){ @array = split(/\t+/, $_); chomp (@array); my ($name, $chr, $location, $gen, $dom, $pval, $fst) = @array[ +0..6]; my $join = "$chr-$location"; push (@values, $join, $pval); if ($location == $prev_loc){ $pval = $pval + $prev_pval; $fst = $fst + $prev_fst; $i++; } elsif ($location != $prev_loc) { my $pval_average = ($prev_pval / $i); my $fst_average = ($prev_fst / $i); my $join2 = "$prev_join\t$pval_average\t$fst_average"; $pval{$prev_join} = $pval_average; $fst{$prev_join} = $fst_average; push (@final, $join2); $i = 1; } $prev_loc = $location; $prev_pval = $pval; $prev_fst = $fst; $prev_join = $join; } foreach (@fields){ @out = split(/\t+/, $_); chomp (@out); my ($name, $chr, $location, $gen, $dom, $pval, $fst) = @ou +t[0..6]; my $join = "$chr-$location"; my $new_pval = $pval{$join}; my $new_fst = $fst{$join}; print $name."\t".$chr."\t".$location."\t".$gen."\t".$dom."\t". +$new_pval."\t".$new_fst."\n"; }
      it goes wrong on the very last line of the file

      Between the 2 foreach blocks, add these 2 lines of code.

      $pval{$prev_join} = $prev_pval/$i; $fst{$prev_join} = $prev_fst/$i;

      There were alot of warnings about some of your variables not being initialized. To get rid of the warnings, I initialized them with values from the first line in the file. Posting some of that code here.

      open INFILE, "<$infile" or die $!; chomp(my @lines = <INFILE>); close INFILE or die $!; my ($chr, $prev_loc, $prev_pval, $prev_fst) = (split /\t/, $lines[0])[ +1,2,5,6]; my $prev_join = "$chr-$prev_loc"; my $i=1; my %pval; my %fst; foreach (@lines[1 .. $#lines]){ my ($name, $chr, $location, $gen, $dom, $pval, $fst) = split /\t/; my $join = "$chr-$location"; if ($location == $prev_loc){ $pval = $pval + $prev_pval; $fst = $fst + $prev_fst; $i++; } else { $pval{$prev_join} = $prev_pval / $i; $fst{$prev_join} = $prev_fst / $i; $i = 1; } $prev_loc = $location; $prev_pval = $pval; $prev_fst = $fst; $prev_join = $join; } $pval{$prev_join} = $prev_pval/$i; $fst{$prev_join} = $prev_fst/$i; foreach (@lines){ my ($name, $chr, $location, $gen, $dom, $pval, $fst) = split /\t+/ +; my $join = "$chr-$location"; my $new_pval = $pval{$join}; my $new_fst = $fst{$join}; print $name."\t".$chr."\t".$location."\t".$gen."\t".$dom."\t".$new +_pval."\t".$new_fst."\n"; }

      There are 2 different criteria about what you use to create a group to average. In your code, a group to average is determined by the code, if ($location == $prev_loc) .... But you are also joining the $chr and $location keys for the 2 different hashes here. I believe this leaves the door open for difficult to trace behaviors. It works without error with the sample data provided, but it could break down with an actual file.

      Would you want a line like if ($join eq $prev_join)...?

      Why join the chr location columns. Can the same location have different 'chr' columns? If not, there is no advantage to combining them like that. They could be more simply using the location for the 2 hashes here and the keys are simple to sort, the keys being 'location' alone - a floating point number.

      Here is a solution I came up with. The same general idea - but uses the module List::Util, (part of perl core since v5.7.3), to get the sum of the 2 columns.

      I compared the join of (chr, location), to determine the end of a run instead of just location by itself.

      #!/usr/bin/perl use strict; use warnings; use List::Util qw/ sum /; chomp(my $line = <DATA>); my $prev_chr_loc = join "-", (split /\t/, $line)[1,2]; my @lines = $line; while (my $line = <DATA>) { chomp $line; my $chr_loc = join "-", (split /\t/, $line)[1,2]; if ($chr_loc eq $prev_chr_loc) { push @lines, $line; } else { print "$_\n" for compute_avg( @lines ); @lines = $line; } $prev_chr_loc = $chr_loc; } print "$_\n" for compute_avg( @lines ); sub compute_avg { my @lines = @_; my $avg_col5 = sprintf "%.9f", sum(map {(split /\t/)[5]} @lines) / @lines; my $avg_col6 = sprintf "%.9f", sum(map {(split /\t/)[6]} @lines) / @lines; for (@lines) { my ($c5,$c6) = (split /\t/)[5,6]; s/$c5\t$c6$/$avg_col5\t$avg_col6/; } return @lines; }

      Update: Redid substitution for closer spec.

      well I've written something that almost does the job, except it goes wrong on the very last line of the file.

      Did you at any point sit down with pencil and paper, and write out steps to solve this problem as human computer with pen and paper?

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://950995]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others studying the Monastery: (5)
As of 2024-03-29 10:50 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found