Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re: Data Normalization with PDL

by lin0 (Curate)
on Jul 07, 2007 at 16:22 UTC ( #625425=note: print w/replies, xml ) Need Help??


in reply to Data Normalization with PDL

One typical practice is to use the code above to normalize the training data and to use the resulting mean and standard deviation to normalize the testing data as illustrated in the snippet below:

#!/usr/bin/perl use warnings; use strict; use PDL; use PDL::NiceSlice; # ================================ # normalize_testing_patterns # $output_data = # $normalized_testing_patterns( $input_data, # $mean_of_input, # $stdev_of_input) ) # # processess $input_data so that $output_data # has 0 mean and 1 stdev # # $output_data = ( $input_data - $mean_of_input ) / $stdev_of_input # ================================ sub normalize_testing_patterns { my ( $input_data, $mean, $stdev ) = @_; my ( $number_of_dimensions, $number_of_patterns ) = $input_data->dims(); my $output_data = ( $input_data - $mean->dummy(1, $number_of_patterns) ) / $stdev->dummy(1, $number_of_patterns); return $output_data; }

Cheers,

lin0

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://625425]
help
Chatterbox?
[Corion]: ... or so I think. As I said, I'm somewhat vague on how to make AnyEvent cooperate with a callback-driven IO event loop that gives me callbacks when data is available or can be written
[ambrus]: what push_write thing? I don't think you need that. that's implemented generically by AnyEvent::Handle
[Corion]: ambrus: Yeah, that's what I think as well. But you give me an idea, maybe I should start with implementing the timer, as that should be far simpler and with fewer edge-cases/nasty interaction than the file watcher
[ambrus]: You only provide the watcher part that tells when the handle is readable or writable, not the actual writing and reading.
[Corion]: ambrus: Hmmm. It makes sense that AnyEvent would implement the push_write itself, but I think I don't have a good idea of where the boundary between AnyEvent and the underlying event system lies... Implementing the timer should give me a better idea
[ambrus]: Corion: push_write is in the higher level abstraction of AnyEvent::Handle, not in the watcher
[Corion]: ambrus: Hmm - rereading Prima::File, that merrily coincides with what Prima does - it tells you "you can read", and you're supposed to read from the fh yourself. I thought it called you with the data already read, which would've been harder to integrate
[ambrus]: you just need an io watcher, created by &AnyEvent::Impl:: Whatever::io(...)
[Corion]: So after talking it through with you even while I'm still not entirely clear on where AE ends and my implementation begins, I think I understand that I only need to implement some smaller parts for each functionality I want to support.
[Corion]: Yeah... and you might even be able to mix and match additional functionality if you have additional async suppliers, like from a separate thread

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (8)
As of 2016-12-08 12:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    On a regular basis, I'm most likely to spy upon:













    Results (141 votes). Check out past polls.