Re: Data Normalization with PDL

[lin0]

One typical practice is to use the code above to normalize the training data and to use the resulting mean and standard deviation to normalize the testing data as illustrated in the snippet below:

#!/usr/bin/perl
use warnings;
use strict;

use PDL;
use PDL::NiceSlice;

# ================================
# normalize_testing_patterns
# $output_data  = 
#   $normalized_testing_patterns(   $input_data,
#                                   $mean_of_input,
#                                   $stdev_of_input) )
#
# processess $input_data so that $output_data
# has 0 mean and 1 stdev
#
# $output_data = ( $input_data - $mean_of_input ) / $stdev_of_input
# ================================
sub normalize_testing_patterns {
    my ( $input_data, $mean, $stdev ) = @_;
    
    my ( $number_of_dimensions, $number_of_patterns ) 
        = $input_data->dims();
    my $output_data 
        = ( $input_data - $mean->dummy(1, $number_of_patterns) ) 
        / $stdev->dummy(1, $number_of_patterns);
    
    return $output_data;
}

Cheers,

lin0