in reply to Re: RAM: It isn't free . . .
in thread RAM: It isn't free . . .

I've just run a (not yet thoroughly tested; nor cleaned up; nor even fully optimised) solution to my problem, and the results are promising:

C:\test>BiMap.pl -S=5 -F=0.6 -I=33554432 Fetch by Addr & Fetch by Name took 18.535743 seconds for 11881376 item +s in a 33554432 sized BiMap Mem: 965,028 K

For context, here is the single Perl hash solution from the other thread on the same dataset:

C:\test>1112165-hash.pl Start: mem: 7,892 K 23762752 Hash built: mem: 4,382,916 K Fetch by Name took 7.166012 seconds Fetch by Addr took 7.895133 seconds Hash queried: check mem: 4,382,940 K

That equates to just 7% longer to run the same dataset using just over 1/5th the memory. Not bad for a fairly crude POC with still considerable potential for further time reduction.

So, I've expended 3 days -- call it ~50 hrs of my time -- to optimise this.

No matter how you look at it, that is a far better trade-off than your criminally inept suggestion -- to move to a disk-based DB -- which at (a minimum) of 5000x slower, would cost my customer $2800 * 5000 = $14 million for every hour of processing he intends to do.

And given the proposal is to run 100 hours of simulations; the net cost of your suggestion would be $1.4 billion - $280,000 = $1.12 $1.39972 *BILLION* of your customer's money that you would be flushing down the drain; and all because you've eschewed learning anything new for the last 30 years; seemingly preferring to remain more an archaeologist, than a programmer.

I just hope that people will learn from this and finally realise that your malodorously constructed, maliciously provocative, willfully ignorant, hopelessly outdated -- not just non-useful, but perilously inept -- completely worthless, merit-less fluff; is intended to be neither technically useful nor intellectually stimulating; its only purpose being shameful self-promotion of the vilest, any-hits-are-good-hits, form.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^3: RAM: It isn't free . . .(Mike Robinson:criminally inept.)
by thezip (Vicar) on Jan 15, 2015 at 23:26 UTC

    BrowserUK, you really crack me up...

    Whenever you type something, I envision Major Charles Emerson Winchester III (from M*A*S*H) bellowing out loud invectives to anyone who dares to confront him.

    Not a bad thing, it just amuses me... :-)


    *My* tenacity goes to eleven...