Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Re: Blocking execution of called programs to reduce buffer size

by moritz (Cardinal)
on Feb 23, 2014 at 12:06 UTC ( #1075887=note: print w/ replies, xml ) Need Help??


in reply to Blocking execution of called programs to reduce buffer size

Whats the best way to read in a very large amount of data from an external program

Pipes! See for example perlopentut for some piping examples.

I think not and AFAIK the only way to do this would be to modify the nfdump command to pause execution when blocked by my program (or use files but I prefer not to really). I wonder if its possible to block execution of the external program without modifying it.

Writing to a pipe whose buffer size is exceeded does in fact block, so unless the program that writes stuff takes special care to do non-blocking writes, pipes already do what you want.

Isn't that wonderful?


Comment on Re: Blocking execution of called programs to reduce buffer size

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1075887]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (6)
As of 2014-12-28 11:55 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (180 votes), past polls