http://www.perlmonks.org?node_id=221175


in reply to count sort & output II

You're running this as a CGI script, are you not? If so, then this line:
unless (++$counter % 10) { print "Press Enter"; <STDIN> }
..won't work. Whoever suggested using this must have assumed you were running this script from the command line. If you want to get back only 10 results at a time from your CGI script, you need to either: Your CGI script can't store your hash in between HTTP requests, since it is unloaded from memory after every request1. I'll assume you don't want to wait 2 minutes for each 10 records, so I'd recommend writing to a cache file somewhere, storing all of the referrer => num records in your preferred sorted order. On each request to your script, just get the appropriate 10 lines from the file, if it exists, or else regenerate it if it doesn't exist. There should also be a mechanism to force a regeneration if the contents of the cache get stale.

You may want to do some debugging to see where the major slowdown is.. If it's the while loop, there's probably not a lot you can do, but if it's in the sorting and copying of the hash keys (BTW, how big does this hash end up?), you may want to consider a non-hash-based solution. Your for-loop has to make a copy of all the hash keys in memory, which may take a long time, considering your HTTP-referer strings are all probably fairly long. Other monks might have some good ideas about improving this portion of the code, but I'm at a loss at the moment.

Good luck,

blokhead

1: Of course, this is not true if your script is running under mod_perl, but it doesn't look like it