Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

Re: Will a shared filehandle processed by multiple threads/processes cause problems?

by CountZero (Bishop)
on Jul 01, 2014 at 09:57 UTC ( [id://1091818]=note: print w/replies, xml ) Need Help??


in reply to Will a shared filehandle processed by multiple threads/processes cause problems?

The data comes out of a database: cannot you have your processing and validating scripts access the database directly?

With a little thinking, perhaps you can re-write the SQL (*) in such a way that the task can be split up already at the database-level and then you can more easily have multiple scripts, perhaps even multiple PCs, work on it concurrently.

(*): Say all the data-records have a reference number, or a timestamp, or ... in one of the fields. Then you can have one script access all the even-numbered references, timestamps, ... and the second script handle all the odd-numbered records.

CountZero

A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

My blog: Imperial Deltronics

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1091818]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others rifling through the Monastery: (6)
As of 2024-03-28 19:12 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found