Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

Re^3: Optimize my code with Hashes

by jbert (Priest)
on Aug 27, 2008 at 12:00 UTC ( #707140=note: print w/replies, xml ) Need Help??

in reply to Re^2: Optimize my code with Hashes
in thread Optimize my code with Hashes

the code took:67358 wallclock secs (29935.47 usr 60.38 sys + 0.00 cusr 0.00 csys = 29995.85 CPU) in considering each entry from PeopleFirst extract ..

That's interesing. So 29935/67358=44% of your time was spent on user CPU. That is significant, and you might want to look into profiling the app's CPU usage (using Devel::Profile and Devel::DProf).

Of course, it also means that 56% of your time is spent doing other things. If that is net latency then you'd do well to look at bulk import/export instead.

Your timestamp logging appeared to show ~6secs for one request, is that right? That can't be representative, since as noted elsewhere in this thread, you'd never manage to do 50k updates in 18 hours if each takes 6s.

Lastly, if you do profile the app, then it will probably benefit you to produce a cut-down version which runs more quickly. This is useful because the profilers slow things down and generate large amounts of data - they'll probably break on such a big run.

Also, having a more quickly repeatable test case (e.g. ~10mins) will greatly accelerate your ability to test ideas on code and algorithm changes.

However, the hard part is knowing if your cut-down test case has the same performance profile as your main job run.

Another thought: if the 'missing' 56% of your time is overnight you might be sharing a network with a backup job, or something else which saturates the net and makes your network response times go very slowly.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://707140]
[james28909]: there are all kinds of things that had to happen to let life come to be. but at the same time, life may not be the end goal IF there is any kind of end goal lol
[james28909]: well who is the person who gets to decide which behaviour is worthy of a downvote? a person with their own beliefs? xD
[erix]: teleology -- I've never understood why that was thunk up
[erix]: ( and when teleology was brought up, during my biology-study., I couldn't get an answer either )
[1nickt]: FTR I agree with you about "design." Just not sure about the trajectory of evolution. THere are a lot of dead-ends in the evolutionary paths. I suspect homo sapiens is just one of them.
[james28909]: thanks for the vid holli, i do like it. thanks for sharing :)
[holli]: gives me goosebumps every time I watch it
holli gotta grab some groceries before dawn. l8ters

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (12)
As of 2017-12-15 14:39 GMT
Find Nodes?
    Voting Booth?
    What programming language do you hate the most?

    Results (433 votes). Check out past polls.