Think about Loose Coupling | |
PerlMonks |
Re: quickness is not so obviousby roboticus (Chancellor) |
on Jan 23, 2015 at 13:14 UTC ( [id://1114277]=note: print w/replies, xml ) | Need Help?? |
If I wrote some code and found that it ran much slower than I expected, I first ask myself "What the heck did I do wrong?". I'll then look at the code to see if I did something dumb. If the code and algorithm look OK, I'll normally then check my assumptions and write a simple program to "look at" the data, but not do any processing, just to see what the fastest possible time could be. (So in this case, I'd have it simply open and read the 10,000 files, but not really look at the contents or do any writing.) Perhaps getting the data really does take that long. (It may be trawling through a remote filesystem with significant network latency, perhaps some of the data is archived on a tape system and it takes time for the robot to swap tapes to make the data available, etc. So if there's little time difference in simply reading the data and doing the processing, I'll ask the operations team about improving performance. If the difference is *huge*, then it's time to profile your code and find out where it's spending all its time. Sometimes you'll find yourself doing something silly (oops! I'm opening and scanning the contents of file X inside of a loop where I'm processing file Y line-by-line). Sometimes you might find a regex that's performing poorly for some odd data. Or you may find that the algorithm you used is even less performant than expected, and you need to use/invent a better one. As an example of the latter, I was surprised a few years ago when I found that the recursive method for computing fibonacci numbers was as slow as it is:
I knew that this method wasn't terribly efficient, but I was still amazed at how slow it actually was:
Yeah, like I'm going to sit through it computing the first 100 numbers that way. That's when I looked up Memoize and started using it. Which, by the way, is a very nice way to get a performance boost for some programs with little modification. I simply added the following two lines to the top of the program (just after the other 'use' statements):
To get much better performance:
Problem solved--Yeah, when you trawl through a deep binary tree *twice*for*each*level*, it's gonna get slow....*very*very*slow*. Lesson learned, moving on to next thing.... ...roboticus When your only tool is a hammer, all problems look like your thumb.
In Section
Meditations
|
|