in reply to Re: RAM: It isn't free . . .
in thread RAM: It isn't free . . .
I've just run a (not yet thoroughly tested; nor cleaned up; nor even fully optimised) solution to my problem, and the results are promising:
C:\test>BiMap.pl -S=5 -F=0.6 -I=33554432 Fetch by Addr & Fetch by Name took 18.535743 seconds for 11881376 item +s in a 33554432 sized BiMap Mem: 965,028 K
For context, here is the single Perl hash solution from the other thread on the same dataset:
C:\test>1112165-hash.pl Start: mem: 7,892 K 23762752 Hash built: mem: 4,382,916 K Fetch by Name took 7.166012 seconds Fetch by Addr took 7.895133 seconds Hash queried: check mem: 4,382,940 K
That equates to just 7% longer to run the same dataset using just over 1/5th the memory. Not bad for a fairly crude POC with still considerable potential for further time reduction.
So, I've expended 3 days -- call it ~50 hrs of my time -- to optimise this.
No matter how you look at it, that is a far better trade-off than your criminally inept suggestion -- to move to a disk-based DB -- which at (a minimum) of 5000x slower, would cost my customer $2800 * 5000 = $14 million for every hour of processing he intends to do.
And given the proposal is to run 100 hours of simulations; the net cost of your suggestion would be $1.4 billion - $280,000 = $1.12 $1.39972 *BILLION* of your customer's money that you would be flushing down the drain; and all because you've eschewed learning anything new for the last 30 years;
seemingly preferring to remain more an archaeologist, than a programmer.
I just hope that people will learn from this and finally realise that your malodorously constructed, maliciously provocative, willfully ignorant, hopelessly outdated -- not just non-useful, but perilously inept -- completely worthless, merit-less fluff; is intended to be neither technically useful nor intellectually stimulating; its only purpose being shameful self-promotion of the vilest, any-hits-are-good-hits, form.