Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re^2: No Performance gain with Parallel::ForkManager

by davido (Cardinal)
on Feb 24, 2014 at 01:08 UTC ( [id://1075918]=note: print w/replies, xml ) Need Help??


in reply to Re: No Performance gain with Parallel::ForkManager
in thread No Performance gain with Parallel::ForkManager

The thing is that the files being read are big enough that two different files very likely sit on different physical tracks on the hard drive. If you read one file, then the next file, then the next one, sequentially, the amount of drive head movement is minimized. If you read two, three, four, or ten files in parallel, the drive head has to shift back and forth a lot. Also, the buffering is less effective, since the drive reading ahead and filling the buffer is probably going to not fill the buffer with data that will be useful to the next request coming in from a different forked child. So the forks are actually working against each other, losing all buffering benefit, and even causing the drive to have to seek to and fro repeatedly.

When multiple processes are using the same physical resource to grab information that is distributed all over the place on that resource, in what amounts to be unpredictable order, it's no surprise that they degrade performance.


Dave

  • Comment on Re^2: No Performance gain with Parallel::ForkManager

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1075918]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (7)
As of 2024-04-19 10:01 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found