in reply to Optimize my code with Hashes
Your process takes 18 hours to run. You want it to be less.
You need to work out what the computer spends most of it's time doing in those 18 hours, so that you can try and reduce the most costly elements first (and get the biggest improvements).
Prime possibilities are:
- Waiting for a network response (latency). A lot of network protocols are request/response. If you need to do 50k requests and a single request takes 1s to process remotely, you'll end up waiting 50k secs == 14hours.
- Waiting to swap stuff in+out. If your app working set is bigger than the RAM available, performance will plummet as it waits for the disk to access virtual memory.
- CPU. If your app isn't waiting for anything else, it's trying to crunch through code. This is the case where profiling etc can help the most.
- Other disk I/O. If your process is reading and writing to storage, that can be slow. Particularly if you are on a slow device and/or are doing a lot of flushing after small writes (but sometimes that is that you need to do).
So you need to run some monitoring tools ('top' is a good first start, but solaris has many others - check out vmstat and iostat too). These will tell you which of the above issues is the problem. (Well, if it's not CPU, swapping or other disk I/O, then it's probably network latency.
And after all that about measuring first?
My best guess is that you're doing 50K LDAP operations and network latency is killing you (i.e. your box isn't busy when you're doing this, but perhaps your LDAP server (or network) is).
Look for a bulk import/export tool for your LDAP server and use that instead.
|
---|
Replies are listed 'Best First'. | |
---|---|
Re^2: Optimize my code with Hashes
by sukhicool (Initiate) on Aug 27, 2008 at 11:05 UTC | |
by jbert (Priest) on Aug 27, 2008 at 12:00 UTC | |
by jethro (Monsignor) on Aug 27, 2008 at 11:22 UTC | |
by sukhicool (Initiate) on Aug 27, 2008 at 12:13 UTC | |
by jethro (Monsignor) on Aug 27, 2008 at 14:30 UTC |