Re: zcat pipe gives "gzip: stdout: Broken pipe" error

[ikegami]

Difference: Your large program uses $SIG{ PIPE } = "IGNORE";.

Solution: Add local $SIG{ PIPE } = "DEFAULT"; in scope of your open. Alternatively, you could redirect zcat's STDERR to /dev/null.

---

Take a look at this:

seq 1000000000 | gzip | zcat | head

It should take a long time to generate and unzip that stream, but it doesn't.

$ time ( seq 1000000000 | gzip | zcat | head )
1
2
3
4
5
6
7
8
9
10

real    0m0.051s
user    0m0.056s
sys     0m0.005s

This is what happens:

1. Once head has printed its ten lines, it exits.
2. This breaks the pipe from zcat to head.
3. The next time zcat attempts to write to the pipe, SIGPIPE is sent to it.
4. Upon receiving SIGPIPE, zcat is killed.
5. gzip is similarly killed by SIGPIPE.
6. seq is similarly killed by SIGPIPE.

The above is what happens in your standalone script.

In your larger program, you appear to have used $SIG{ PIPE } = "IGNORE";. It's still just as quick, but the error message now appears.

$ seq 1000000000 | gzip | perl -e'$SIG{ PIPE } = "IGNORE"; exec("zcat"
+)' | head
1
2
3
4
5
6
7
8
9
10

gzip: stdout: Broken pipe

This is what happens now:

1. zcat inherits its parent's disposition of ignoring SIGPIPE.
2. Once head has printed its ten lines, it exits.
3. This breaks the pipe from zcat to head.
4. The next time zcat attempts to write to the pipe, SIGPIPE is sent to it.
5. zcat ignores the SIGPIPE.
6. The write call returns error EPIPE.
7. zcat exits with a message as a result of the error. (zcat is an alias for gzip, which is why the message says gzip.)
8. gzip is killed by SIGPIPE.
9. seq is killed by SIGPIPE.

Add local $SIG{ PIPE } = "DEFAULT"; in scope of your open.

[NERDVANA]

Wow, I did not know signal disposition could persist across an exec(). TIL...

[ikegami]

Linux's https://man7.org/linux/man-pages/man2/execve.2.html says:

All process attributes are preserved during an execve(), except the following:

• The dispositions of any signals that are being caught are reset to the default (signal(7)).

• [...]

So a caught signal such as $SIG{ PIPE } = \&handler; gets reset to $SIG{ PIPE } = "DEFAULT";, but not $SIG{ PIPE } = "IGNORE";.

Most if not all reset things are out of necessity. $SIG{ PIPE } = \&handler can't be kept since &handler will stop existing.