Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

Cat multiple files into one pipe

by ehdonhon (Curate)
on Apr 25, 2003 at 03:41 UTC ( #253059=perlquestion: print w/replies, xml ) Need Help??

ehdonhon has asked for the wisdom of the Perl Monks concerning the following question:

Greetings fellow monks.

I'm having trouble getting this code to work. The code should open a pipe to a program, then send that program multiple files. The problem is that the pipe is closing as soon as the first file is done. I'm assuming that is because I'm sending the EOF from the first file. How do I correct for that?

open ( SCRIPT, "| /usr/local/bin/my_program" ) or die $!; foreach ( </my/file/dir/*> ) { if ( /\.gz$/ ) { open ( FILE, "/usr/bin/zcat $_ |" ) or die $!; } else { open ( FILE, "< $_" ) or die $!; } while ( <FILE> ) { print SCRIPT; } close( FILE ); } close( SCRIPT );

I also tried this slight variant which uses chomp, but no luck:

while ( chomp( my $line = <FILE>) ) { print SCRIPT "$line\n"; }

I am finding that after the first file is sent, the program on the receiving pipe doesn't recieve any more data, but Perl keeps chuging along sending the other files to the pipe. As soon as the receiving program exits, my script dies with a "Broken Pipe" error.

Replies are listed 'Best First'.
Re: Cat multiple files into one pipe
by pzbagel (Chaplain) on Apr 25, 2003 at 04:59 UTC

    I understand about not being able to do the cat thing as a solution. But, for the sake of argument, try it as a test. It sounds like wusage is crashing while it tries to read in the files and that is why you are getting the broken pipe error. Testing it with cat can eliminate the perl script and verify that wusage is at least working.

Re: Cat multiple files into one pipe
by pzbagel (Chaplain) on Apr 25, 2003 at 04:03 UTC

    I have some bad news, the script works the way you want it to on my system. That is, I am testing with three short text files and in the first line, I simply open "| cat". That works and cats out the three files as expected.

    So as it stands, the logic you are using works for piping to cat. Can you get more specific on what you are piping your files into? This may be the source of the problem.

      I'm piping http log files into wusage

      My open command looks like this:

      open ( SCRIPT, "| /usr/local/bin/wusage -c ${domain}.conf -l -" ) or d +ie $!;

        Question: If you manually cat those files together and pipe to your program, does it work. That is:

        cat /my/files/dir/* |/usr/local/bin/wusage etc...

        Let's see if this is the perl script that is in fact failing.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://253059]
Approved by graff
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (6)
As of 2022-06-30 07:46 GMT
Find Nodes?
    Voting Booth?
    My most frequent journeys are powered by:

    Results (97 votes). Check out past polls.