|Problems? Is your data what you think it is?|
Re: Speed up perl codeby fluffyvoidwarrior (Monk)
|on Sep 21, 2012 at 07:55 UTC||Need Help??|
Lots of code.
Just a few generic suggestions that you've probably thought of anyway
Some jobs are actually big and do take a long time. However it's easy to make a job big that shouldn't be. I would suggest going back to basics for a few hours and pondering the underlying design of your program. Have you intrinsically designed it wrong? Is there a better approach to the same goal? Are you not processing in a smart way, perhaps multiplying the work as your program runs rather than reducing it. Its easy to tend towards infinite with something that's OK on a small dataset but multiplies up exponentially when you feed it large data.
Assuming you can only go down the route you have already taken:-
Have you specifically located the bottlenecks? Maybe when you do you should then benchmark chunks of code and run repeatedly with modifications.
You may find that you need to write your own optimised code for the cutting edge to your particular problem, rather than use a generic cpan module.
My guess is that you'll get a lot more help on here if you locate your problem areas, present much smaller code snippets and ask "How can I speed this up?"
Either way, there probably isn't much point in speculating 'till you've located the bottleneck(s). Possible quick fix : if it's thrashing your hard drive move it to a ramdisk.
Check this out: