Beefy Boxes and Bandwidth Generously Provided by pair Networks DiBona
Welcome to the Monastery
 
PerlMonks  

Re^2: Daily Counters

by docbrown25 (Initiate)
on May 06, 2013 at 17:53 UTC ( #1032356=note: print w/ replies, xml ) Need Help??


in reply to Re: Daily Counters
in thread Daily Counters

Interesting. Thanks for the reply. I'm going to look into implementing like this.

My user ids will be all numerics. Should I break each user_id up by each digit of the id?

For example: $user_id = 5989358

/pathtocountdir/$date/5/9/8/9/3/5/8/5989358/$count/

this will also allow me to just clear out the whole /pathtocountdir/$date/ dir for previous days

thoughts?


Comment on Re^2: Daily Counters
Re^3: Daily Counters
by BrowserUk (Pope) on May 06, 2013 at 19:04 UTC
    Should I break each user_id up by each digit of the id?

    No. It just creates extra levels for the filesystem to lookup, which slows things down, for no benefit.

    Adding the date into the path however is a brilliant idea.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re^3: Daily Counters
by BrowserUk (Pope) on May 06, 2013 at 19:27 UTC

    By the way, as you only have 10 characters in your alphabet, you might want to consider using the first 4 digits split into two groups of 2:

    /pathtocountdir/date/11/22/1122333/

    Or perhaps two groups of 3:<code>/pathtocountdir/date/111/222/1112223/


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      OK thanks. Just curious if that will give me enough directories to avoid hitting any filesystem limits?

        Depends upon your filing system, but 2 x 2-digits means a max of 100 in the lower levels and 10,000 in the top. 2 x 3-digit, gives 1000 in teh lower levels and 100 in the top. Every filesystem I am aware of will handle those numbers with ease.

        The latter probably works better for performance, but you'd have to do a few tests to be sure.

        On my system it is much of a muchness at around 3 milliseconds per lookup/increment either way:

        #! perl -slw use strict; use Time::HiRes qw[ time ]; mkdir 'myroot'; for my $l1 ( '00' .. '09' ) { mkdir "myroot/$l1"; for my $l2 ( '00' .. '09' ) { mkdir "myroot/$l1/$l2"; mkdir "myroot/$l1/$l2/$l1$l2$_" for '000' .. '999'; mkdir "myroot/$l1/$l2/$l1$l2$_/1" for '000' .. '999'; } } my $start = time; for( 1 .. 1000 ) { my $id = sprintf "%02u%02u%03u", int( rand 10 ), int( rand 10 ), in +t( rand 1000 ); my( $l1, $l2 ) = unpack 'a2a2', $id; opendir D, "myroot/$l1/$l2/$id/"; readdir D; readdir D; ## get rid of '.' & '..' my $count = readdir D; rename "myroot/$l1/$l2/$id/" . $count, "myroot/$l1/$l2/$id/" . ++$c +ount or warn "$! : $id : $count"; closedir D; } printf "2x2x7 took %f secs/lookup&increment\n", ( time() - $start ) / +1000; system "rd /q /s myroot"; mkdir 'myroot'; for my $l1 ( '000' .. '009' ) { mkdir "myroot/$l1"; for my $l2 ( '000' .. '009' ) { mkdir "myroot/$l1/$l2"; mkdir "myroot/$l1/$l2/$l1$l2$_" for '0' .. '9'; mkdir "myroot/$l1/$l2/$l1$l2$_/1" for '0' .. '9'; } } $start = time; for( 1 .. 1000 ) { my $id = sprintf "%03u%03u%01u", int( rand 10 ), int( rand 10 ), in +t( rand 10 ); my( $l1, $l2 ) = unpack 'a3a3', $id; opendir D, "myroot/$l1/$l2/$id/" or warn "$! : myroot/$l1/$l2/$id/" +; readdir D; readdir D; ## get rid of '.' & '..' my $count = readdir D; rename "myroot/$l1/$l2/$id/" . $count, "myroot/$l1/$l2/$id/" . ++$c +ount or warn "$! : $id : $count"; closedir D; } printf "3x3x7 took %f secs/lookup&increment\n", ( time() - $start ) / +1000; system "rd /q /s myroot"; __END__ C:\test>1032393 2x2x7 took 0.002671 secs/lookup&increment 3x3x7 took 0.002761 secs/lookup&increment C:\test>1032393 2x2x7 took 0.002628 secs/lookup&increment 3x3x7 took 0.003111 secs/lookup&increment

        BTW: Please note the additional readdir D; readdir D; ## get rid of '.' & '..' which I forgot above.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
        .

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1032356]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (9)
As of 2014-04-24 11:27 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (565 votes), past polls