Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

Re: Is my code forking the way I want it to? Forking and System Calls

by SimonPratt (Friar)
on Jul 08, 2014 at 08:35 UTC ( [id://1092687]=note: print w/replies, xml ) Need Help??

in reply to Is my code forking the way I want it to? Forking and System Calls

OK, from what I can see, you are forking off a separate process for each and every file that you need to process. This is really inefficient and will probably cause your code to run very slowly (more slowly than running a single process in series). I strongly recommend you rethink your forking strategy (ie, build it to not be parallel, then parallelise it when you have it working). Also bear in mind that when you have processes that are fairly busy (more than 50% core utilisation, on average), having more than one thread per physical core in your machine is likely to cause your code to Go Slow.

Next, unless all of your files are in individual folders, the likely cause of your error is that you will have multiple instances of your proprietary decryption program attempting to write to the same log file. This may or may not be the actual cause of your issue, but should be simple to test

The final point is that you say you don't have enough space to convert all of your files before parsing and unlinking them, however your code as it stands does exactly that - converts all files, then unlinks the log files, then parses the output.

  • Comment on Re: Is my code forking the way I want it to? Forking and System Calls

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1092687]
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others surveying the Monastery: (1)
As of 2024-06-12 19:35 GMT
Find Nodes?
    Voting Booth?

    No recent polls found

    erzuuli‥ 🛈The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.