Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical

Re: Counter - number of tags per interval

by LanX (Chancellor)
on May 04, 2013 at 22:08 UTC ( #1032090=note: print w/replies, xml ) Need Help??

in reply to Counter - number of tags per interval

Just another idea based on tuning your current algorithm which is quite clever! (Well after being understood - you know my opinion about clear naming ;-)

After analyzing the numbers you provided, I suppose that intersections of different ranges are comparatively rare and if they appear they stretch over long distances.

So it doesn't really make sense to go into the count loop for each individual point, just go on counting the "1" till you reach the next "boundary" and add them collectively to your counter instead incrementing them one by one with++.

The boundaries are nothing else as the sorted list of range starts-1 and stops. In your example for ID "b"

ID start stop ... b 10 15 b 12 15

this means [9,11,15].

and results in the following flow

0-9 => ignore 10-11 => count 1 x "1" => add 1 to range [10-15] 12-15 => count 4 x "1" => add 4 to range [10-15] and [12-15] 16- => ignore

As you see, you can also stop counting between ranges and even try to jump forward in the file with seek.

HTH! =)

Update: Approximation

3e6 Intervals/50 IDs *2 Borders = 120000 Borders

6e9 Points / ID

=> in average 6e9/12e4=50000 points between two adjacent borders, where you can avoid the whole count loop.

This should lead to a considerable boost!

Cheers Rolf

( addicted to the Perl Programming Language)

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1032090]
[haukex]: Corion: Yes exactly, in the author tests I don't worry about portability as much, I also don't list the author tests' dependencies in Makefile.PL
[haukex]: I figure someone who wants to contribute will know how to install the missing modules ;-) Not the nicest way to go but I don't think many people are using my modules yet
[ambrus]: Corion: some of these stupid syntax highlighters assume that too. just look at the table in http://perldoc. functions/pack. html for example.
[haukex]: ..."yet" ;-) I haven't had to deal with Dist::Zilla yet but I've heard about how it's a big setup
[ambrus]: I really don't like automagic stuff. I'm happy when computers do exactly what I tell them, even if that means they sometimes do the wrong thing.

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (14)
As of 2017-02-27 12:41 GMT
Find Nodes?
    Voting Booth?
    Before electricity was invented, what was the Electric Eel called?

    Results (385 votes). Check out past polls.