Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re: Blocking execution of called programs to reduce buffer size

by moritz (Cardinal)
on Feb 23, 2014 at 12:06 UTC ( #1075887=note: print w/ replies, xml ) Need Help??


in reply to Blocking execution of called programs to reduce buffer size

Whats the best way to read in a very large amount of data from an external program

Pipes! See for example perlopentut for some piping examples.

I think not and AFAIK the only way to do this would be to modify the nfdump command to pause execution when blocked by my program (or use files but I prefer not to really). I wonder if its possible to block execution of the external program without modifying it.

Writing to a pipe whose buffer size is exceeded does in fact block, so unless the program that writes stuff takes special care to do non-blocking writes, pipes already do what you want.

Isn't that wonderful?


Comment on Re: Blocking execution of called programs to reduce buffer size

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1075887]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (9)
As of 2015-07-07 10:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (88 votes), past polls