Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

OK, from what I can see, you are forking off a separate process for each and every file that you need to process. This is really inefficient and will probably cause your code to run very slowly (more slowly than running a single process in series). I strongly recommend you rethink your forking strategy (ie, build it to not be parallel, then parallelise it when you have it working). Also bear in mind that when you have processes that are fairly busy (more than 50% core utilisation, on average), having more than one thread per physical core in your machine is likely to cause your code to Go Slow.

Next, unless all of your files are in individual folders, the likely cause of your error is that you will have multiple instances of your proprietary decryption program attempting to write to the same log file. This may or may not be the actual cause of your issue, but should be simple to test

The final point is that you say you don't have enough space to convert all of your files before parsing and unlinking them, however your code as it stands does exactly that - converts all files, then unlinks the log files, then parses the output.


In reply to Re: Is my code forking the way I want it to? Forking and System Calls by SimonPratt
in thread Is my code forking the way I want it to? Forking and System Calls by pimperator

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having a coffee break in the Monastery: (3)
As of 2024-06-16 14:12 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?
    erzuuli‥ 🛈The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.