Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Re: Pipes vs. temporary files

by bluto (Curate)
on Jul 25, 2007 at 20:33 UTC ( #628787=note: print w/replies, xml ) Need Help??


in reply to Pipes vs. temporary files

One issue with a pipe is that if the producer process dies, the consumer may wait forever on the pipe after processing part of the data. The benefit of a temporary file in this case is that you an ensure you have all of the data before processing it (i.e. by having the producer rename the file into the final name that the consumer expects). This may or may not matter to you.

Another problem with pipes is that if you use them on a single processor machine you may see a lot of context switching as the OS switches between the producer and consumer (i.e. each time the pipe is filled -- usually after about 4KB).

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://628787]
help
Chatterbox?
[Corion]: Hmmm. I feel a Meditation coming on. I wrote a module, DBIx::PivotQuery, which returns a table-like set of rows (AoA) but some columns are generated from column values, like in an (Excel) pivot table or a ROLLUP command
[Corion]: My current approach for subtotals involves rerunning the given query, with the hint to the user that they should use a temporary table if they want better performance.
[Corion]: But I could create that temporary table in the module and use it for the improved perfomance directly instead.
[Corion]: And the question is, what would be better/preferred ;-)
[Corion]: Hmm - not exactly like the ROLLUP command. Ah well.

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (10)
As of 2017-02-23 15:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Before electricity was invented, what was the Electric Eel called?






    Results (347 votes). Check out past polls.