Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery

Re^5: Comparing two arrays

by educated_foo (Vicar)
on Dec 18, 2013 at 00:57 UTC ( #1067565=note: print w/replies, xml ) Need Help??

in reply to Re^4: Comparing two arrays
in thread Comparing two arrays

I just re-read your original post, and was reminded of this:
there are about 500 times less 1's then 0's
So for the average item, only 30 features are true, which changes things. You still can't do an all-to-all pairwise comparison in any reasonable amount of time, but you may be able to e.g. recursively partition your data by the feature closest to a 50/50 distribution, then find closest matches once you cut the comparison set down to something reasonable. If something simple like that won't work, you should look into dimensionality reduction, e.g. via random projection or PCA.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1067565]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (9)
As of 2016-10-24 18:48 GMT
Find Nodes?
    Voting Booth?
    How many different varieties (color, size, etc) of socks do you have in your sock drawer?

    Results (309 votes). Check out past polls.