Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re^2: Bidirectional lookup algorithm? (no redis )

by Anonymous Monk
on Jan 05, 2015 at 23:15 UTC ( [id://1112255]=note: print w/replies, xml ) Need Help??


in reply to Re: Bidirectional lookup algorithm? (Updated: further info.)
in thread Bidirectional lookup algorithm? (Updated: further info.)

See http://redis.io/topics/benchmarks (esp Pitfalls and misconceptions ), its too slow for this problem ( its ipc )
$ redis-benchmark -t set -r 100000 -n 1000000 ====== SET ====== 1000000 requests completed in 13.86 seconds 50 parallel clients 3 bytes payload keep alive: 1

A simple perl program adds 1mil key/value pairs in a hash in in 2.6875 seconds on a laptop from 2006

OP doesn't need concurrency , so redis not needed

Replies are listed 'Best First'.
Re^3: Bidirectional lookup algorithm? (no redis )
by polettix (Vicar) on Jan 10, 2015 at 22:15 UTC
    Although in the OP there were no specific quality parameters about space/time constraints, my understanding is that space efficiency improvements could come at the expense of (some) speed, hence the suggestion.

    Any benchmark should probably consider both aspects.

    perl -ple'$_=reverse' <<<ti.xittelop@oivalf

    Io ho capito... ma tu che hai detto?
      my understanding is that space efficiency improvements could come at the expense of (some) speed,

      You are correct in that. The ultimate desire is to retain (as much of as is possible) the speed of Perl's hashes; whilst reducing the space requirement necessitated by the need for bidi lookup.

      My initial brief suggested that a move from O(1) to O(logN) (with a very small constant) lookup time would be acceptable; if a 4:1 (75%) reduction in space/1:4 (400%) increase in capacity (or greater) was achieved.

      However, anonymonk is also correct, in that external storage -- whether disk/SSD or remote ram via sockets -- is too slow. A simple test will show that the minimum turnaround time of sending a request to a 'remote' (even where 'remote' equates to the callback address) and retrieving the reply requires 3 times as long as a perl hash loopkup. Even when you completely exclude the actual lookup and have the request and response consist of a single byte each way.

      No matter how fast the actual lookup at the remote, adding that constant to it crushes the overall goal.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
        I see. So, more or less you're left with the thing you were after in the first place - the algorithm. Good luck with that!

        perl -ple'$_=reverse' <<<ti.xittelop@oivalf

        Io ho capito... ma tu che hai detto?

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1112255]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others romping around the Monastery: (5)
As of 2024-06-18 09:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?
    erzuuli‥ 🛈The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.