Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re^3: Compact and sparse bit vector

by diotalevi (Canon)
on Jan 04, 2009 at 09:30 UTC ( #733999=note: print w/ replies, xml ) Need Help??


in reply to Re^2: Compact and sparse bit vector
in thread Compact and sparse bit vector

I've never really found myself doing vector operations of bitmaps against each other. I didn't notice that being a feature of the linked Netflix node. I do occasionally find it convenient to have a sparse bitmask. Usually I just use a hash of the stringified integer.

⠤⠤ ⠙⠊⠕⠞⠁⠇⠑⠧⠊


Comment on Re^3: Compact and sparse bit vector
Re^4: Compact and sparse bit vector
by BrowserUk (Pope) on Jan 04, 2009 at 10:34 UTC

    There's no implied criticism of Judy arrays, or your module, for the uses for which they are designed.

    And indeed, there is no requirement in the NetFlix challenge to use bitvectors at all. There is a need however, to perform some more or less complex relational operations across the three datasets to arrive at the goal of the challenge: To estimate a user rating for given user for a given film (that they haven't yet watched or rated) based upon how others that have watched and rated the given film have rated it.

    Whilst this kind of query is relatively trivial to code using SQL, the volumes of raw data, and size of the cross-product of the film & user bases is such that it is time&memory intensive to produce the results. Using bitwise storage for this type of join query is extremely fast. Some literature.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://733999]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (6)
As of 2014-09-23 07:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (211 votes), past polls