Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re^2: Finding Nearly Identical Sets (Updated:4200/sec)

by BrowserUk (Pope)
on Sep 30, 2016 at 04:31 UTC ( #1172971=note: print w/replies, xml ) Need Help??


in reply to Re: Finding Nearly Identical Sets (Updated:4200/sec)
in thread Finding Nearly Identical Sets

FWIW: my fears that the transforms were not being detected were unfounded. There are 10 times as many possibilities for 1-insertion matches than there are for the deletion & substitution varieties, so it starts detecting insertions almost straight away, but the other two not so. However, once the number of known instances gets high enough, they start to show in roughly the expected proportions:

C:\test>\Perl22\bin\perl.exe 1172842.pl -N=1e6 -SAVE -LOAD > null 1000000 From 1000000 received there were: 767637 new; 16713734 + known; 0 deletions; 2899 insertions; 19788 transformations. Numbers processed at a rate of 4366/second C:\test>\Perl22\bin\perl.exe 1172842.pl -N=5e7 -SAVE -LOAD > null 50000000 From 50000000 received there were: 36232459 new; 21068 +158 known; 0 deletions; 353102 insertions; 1173728 transformations. Numbers processed at a rate of 4592/second C:\test>\Perl22\bin\perl.exe 1172842.pl -N=5e7 -SAVE -LOAD > null 50000000 From 50000000 received there were: 33583555 new; 52246 +313 known; 0 deletions; 650237 insertions; 1281578 transformations. Numbers processed at a rate of 4767/second C:\test>\Perl22\bin\perl.exe 1172842.pl -N=5e7 -SAVE -LOAD > null 50000000 From 50000000 received there were: 30372628 new; 86174 +891 known; 0 deletions; 899052 insertions; 1715170 transformations. Numbers processed at a rate of 5344/second C:\test>\Perl22\bin\perl.exe 1172842.pl -N=5e7 -SAVE -LOAD > null 50000000 From 50000000 received there were: 27148533 new; 11871 +9247 known; 93783 deletions; 1103187 insertions; 2202499 transformati +ons. Numbers processed at a rate of 5900/second

Note also that this code really loves having the system to itself -- the last couple of runs were overnight when the machine was otherwise idle -- and the performance jumps up from 4200/s to 5900/s; +40%.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1172971]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (6)
As of 2021-03-01 23:14 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My favorite kind of desktop background is:











    Results (28 votes). Check out past polls.

    Notices?