|Perl: the Markov chain saw|
Make java code fasterby fizbin (Chaplain)
|on Aug 10, 2004 at 15:57 UTC||Need Help??|
One module that has saved me a few times when a script just wasn't performing fast enough is Devel::SmallProf. I'm sure that there's a way to get good use out of profilers that don't drill down to a line-by-line analysis, but for "so that's where all the time goes" surprise moments, I've never found anything else that works as well on perl scripts as Devel::SmallProf. (for example, letting me know that I was throwing a huge amount of time into one mktime() call, and that by memoizing it could double my script's performance)
Anyway, sadly most of my job does not involve the camel. Instead, I often find myself trying to make java code go faster. Usually, this involves running java with the included profiler and looking at this output in various viewers, like ProphIt or HPjmeter. However, although these tools are often nice for demonstrating to management that there's not one single big timesink we can point to and say "that's it", I often find them a bit short on where in a given method all the time is being spent - these tools stop at the method level, and I miss my line-by-line information.
Well, the raw output of the java profiler includes this line-by-line information (assuming you compiled your code with debugging on), and the output is text. Enter perl.
The tool below lets you focus on which lines are actually eating up your cpu time; usage is pretty simple:
perl wherehot.pl [-l limitingExpression] methodExpr profile1 [profile2 ...]For example, if you know that it's code inside org.perlmonks.fizbin.Foo.doStrangeThings() that you want to look at closely, you can do:
perl wherehot.pl org.perlmonks.fizbin.Foo.doStrangeThings todaysProfile.hprof.txtOr even (if you don't care about other classes with that same method name, or want to see them too):
perl wherehot.pl doStrangeThings todaysProfile.hprof.txtOr if you want to look at everything inside a given class, use:
perl wherehot.pl 'Foo\..*' yesterdaysProfile.hprof.txtThe expression is anchored so that you won't get any class named SuperFoo, but without the ugly \. you would select classes named FooBar. Maybe in a future version.
Another thing often is that there are certain usage patterns that are slow because they eventually cause some slow code to be invoked - for example, using a StringBuffer and consistently not allocating enough room. In this case, what you want to see is everything in your packages that eventually calls the slow code, so that you can try to avoid causing performance problems down the line.
perl wherehot.pl -l StringBuffer.expandCapacity com.perlmonks.fizbin.* *.hprof.txtThe -l limiting expression also accepts something like StringBuffer.java:145 in case the limit can't be expressed by a method name but can be expressed with a particular line.
When I use this script, I often pipe it through head, because I rarely care about more than the top few offenders. The top line always gives you the total number of samples in the entire profile, to keep you from losing perspective in your optimizing hunts. And now the script:
I have a similar script for analyzing the heap dumps that the java profiler can produce, but I've not found that nearly as useful. (For our work, HPjmeter is adequate for determining allocation sites)