Lots of code.

Just a few generic suggestions that you've probably thought of anyway
Some jobs are actually big and do take a long time. However it's easy to make a job big that shouldn't be. I would suggest going back to basics for a few hours and pondering the underlying design of your program. Have you intrinsically designed it wrong? Is there a better approach to the same goal? Are you not processing in a smart way, perhaps multiplying the work as your program runs rather than reducing it. Its easy to tend towards infinite with something that's OK on a small dataset but multiplies up exponentially when you feed it large data.

Assuming you can only go down the route you have already taken:-
Have you specifically located the bottlenecks? Maybe when you do you should then benchmark chunks of code and run repeatedly with modifications.

You may find that you need to write your own optimised code for the cutting edge to your particular problem, rather than use a generic cpan module.

My guess is that you'll get a lot more help on here if you locate your problem areas, present much smaller code snippets and ask "How can I speed this up?"

Either way, there probably isn't much point in speculating 'till you've located the bottleneck(s). Possible quick fix : if it's thrashing your hard drive move it to a ramdisk.

Check this out:
http://makepp.sourceforge.net/1.50/perl_performance.html

---

---

• Are you posting in the right place? Check out Where do I post X? to know for sure.
• Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:

  <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>

• Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
• Want more info? How to link or How to display code and escape characters are good places to start.