Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re^3: DBI::SQLite slowness

by Anonymous Monk
on Sep 20, 2013 at 04:59 UTC ( #1054957=note: print w/ replies, xml ) Need Help??


in reply to Re^2: DBI::SQLite slowness
in thread DBI::SQLite slowness

I imagine BrowserUk is using vec

See Re: vec overflow?, Re: searching a vector array, Re^2: Array vs. Hash for sparsely integer-indexed data (bit vectors)

This takes about 33MB of ram,

perl -e " my $f='' ; vec($f, 2**28,1)=1; "

I think that's marking as seen (as duplicate) the id 268435456 or 268_435_456

I think BrowserUk is probably assuming a minimum starting id, 268_435_456 is a lot of IDs :) 256MiB


Comment on Re^3: DBI::SQLite slowness
Download Code
Re^4: DBI::SQLite slowness ( mini vec tutorial )
by Anonymous Monk on Sep 20, 2013 at 13:10 UTC

    Following up on Re: vec overflow? an adjustable example (mini vec tutorial)

    The @ in the output it produces is used for meaning of "@at", its not an actual array :) for example  0@[ 0][ 0] means the number zero is stored in the first(zero-th,0-th) seen_vecs string, and its the first bit of the string (0-th bit) ; neat how that works, id-zero is zero-th bit, is offset-th-ed-bit :)

    This can help with the vec syntax :) Bit::Vector::Minimal - Object-oriented vec wrapper

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1054957]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (7)
As of 2014-07-13 23:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    When choosing user names for websites, I prefer to use:








    Results (252 votes), past polls