Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

Your process takes 18 hours to run. You want it to be less.

You need to work out what the computer spends most of it's time doing in those 18 hours, so that you can try and reduce the most costly elements first (and get the biggest improvements).

Prime possibilities are:

  • Waiting for a network response (latency). A lot of network protocols are request/response. If you need to do 50k requests and a single request takes 1s to process remotely, you'll end up waiting 50k secs == 14hours.
  • Waiting to swap stuff in+out. If your app working set is bigger than the RAM available, performance will plummet as it waits for the disk to access virtual memory.
  • CPU. If your app isn't waiting for anything else, it's trying to crunch through code. This is the case where profiling etc can help the most.
  • Other disk I/O. If your process is reading and writing to storage, that can be slow. Particularly if you are on a slow device and/or are doing a lot of flushing after small writes (but sometimes that is that you need to do).

So you need to run some monitoring tools ('top' is a good first start, but solaris has many others - check out vmstat and iostat too). These will tell you which of the above issues is the problem. (Well, if it's not CPU, swapping or other disk I/O, then it's probably network latency.

And after all that about measuring first?

My best guess is that you're doing 50K LDAP operations and network latency is killing you (i.e. your box isn't busy when you're doing this, but perhaps your LDAP server (or network) is).

Look for a bulk import/export tool for your LDAP server and use that instead.


In reply to Re: Optimize my code with Hashes by jbert
in thread Optimize my code with Hashes by sukhicool

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (2)
As of 2024-04-20 06:06 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found