Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Computing Covariance Matrices with PDL

by lin0 (Curate)
on Jul 08, 2007 at 20:45 UTC ( #625532=snippet: print w/replies, xml ) Need Help??
Description:

A Covariance Matrix is a matrix of covariances (the measure of how much two random variables vary together) between elements of a vector.

In this snippet, I present how to compute a covariance matrix using the Perl Data Language. The input is a piddle (see comment below for a definition) in which each row represents an input vector and each column represents a dimension of the input vector. The output is a piddle that holds the covariance matrix.

What are Piddles?

They are a new data structure defined in the Perl Data Language. As indicated in RFC: Getting Started with PDL (the Perl Data Language):

Piddles are numerical arrays stored in column major order (meaning that the fastest varying dimension represent the columns following computational convention rather than the rows as mathematicians prefer). Even though, piddles look like Perl arrays, they are not. Unlike Perl arrays, piddles are stored in consecutive memory locations facilitating the passing of piddles to the C and FORTRAN code that handles the element by element arithmetic. One more thing to note about piddles is that they are referenced with a leading $

Cheers,

lin0

#!/usr/bin/perl
use warnings;
use strict;
use PDL;

# ================================
# covariance: 
#
#   $Sigma = covariance( $X )
#
#   computes the Sample Covariance Matrix of
#   a sample X1...Xn of p-dimensional vectors
# ================================
sub covariance {
    my ( $X ) = @_;
    
    my $Diff = $X - average( $X->xchg(0,1) );
    
    my $Sigma = ( 1 / ( $X->getdim(1) - 1 ) )
                * transpose( $Diff ) x $Diff;
    
    return $Sigma;
}
Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: snippet [id://625532]
help
Chatterbox?
[Corion]: Oh yay. I wonder why a very simple change in a program doesn't elicit a diff, and now I see that my diff program seems to have a bug ;)
[1nickt]: marto thanks for asking, so far so good. A pretty modern stack and decent procedures, although rather too much home-built stuff (e.g. a logging role that should tries to duplicate Log::Any).
[Corion]: No. It's just that I'm comparing the same output file twice, instead of comparing the output files of the two runs %-)
[Corion]: Lo and behold, running a program with the correct input files yields the correct (and expected) output. Yay me.
[1nickt]: Got a MacBook and am expected to develop directly on it, ironic given the recent thread about that.
[marto]: Corion, some not too serious issues with the kids, hopefully, other than that just dealing with commuting by car again in the winter, not much fun so far, and there's no real 'bad' weather yet :)
[marto]: 1nickt, glad to hear it, I'm sure you'll bring improvements to their stack in due course ;)
[1nickt]: Just trying to adjust to a new culture. Better doc than I am used to , but quite a bit more Agiley meta-bullshit; at least half a dozen meetings a week, strict "sprint" schedule etc. I'll adjust. And I hope to just switch off after the "scrum" each day at
[1nickt]: ... the end of the afternoon. marto How are you, snowed in? Mum and Dad expecting snow today in Surrey. (Have 5" on the ground here in NJ. And my son's high school in SOCal closed until after Xmas due to fires.)
[Corion]: 1nickt: In a sense the meta-bullshit might even be beneficial to you as you get to hear the cow-orkers talk (or not talk) about the issues you perceive. I think Agile is mostly about the meta ;)

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (5)
As of 2017-12-11 11:45 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    What programming language do you hate the most?




















    Results (289 votes). Check out past polls.

    Notices?