My usual procedure for optimizing something is:
- Profile it, to find out which part of the problem is taking the most time. (Sometimes the problem's a single algorithm, in which case this isn't really necessary.)
- Estimate the asymptotic bound of that bit of the problem. (Yet another excellent real-world use for those icky awful theory courses you were forced to take in university. :-)
- Try to make the algorithm faster. Sometimes this just isn't possible; IIRC, you can't get a general sort on a single-processor machine faster than O(n log n). But you can often simplify the problem: put a few restrictions on what you sort, and you can get down to O(n). Nobody knows how to solve the travelling salesman problem in less than exponential time, but if you don't need a perfect solution, just one no worse than twice as bad as optimal, you can do it in polynomial (cubic, IIRC) time.
- Repeat until you run out of relevant sub-problems.
- If it still isn't fast enough, profile it again; this time, look for lines and functions that get executed most often, and try to slim those down. If your program's data set is particularly small, this might be more important than step 2; then again, if n is small, your program's probably "fast enough".