$x = `cat big_file` seg faults

[Danny]

file1 has size 2147483648 (2**31)
file2 has size 2147483647 (2**31-1)

perl -we 'my $x = `cat file1`' # seg faults

perl -we 'my @x = `cat file1`' # works fine

perl -we 'my $x = `cat file2`' # works fine

perl -we 'my $x = `cat file2`; $x .= `cat file2`' # works fine

So it seems that in scalar context if the output of the command in backticks is greater than 2**31-1 then it segfaults. It seems the backticks use some buffer that has a size limit. The results were the same on 64 bit linux and cygwin.

Update: Curiously if I do:

perl -we 'my $x = `cat file2 file2`'

I get a

Out of memory in perl:util:safesysmalloc (cygwin perl 5.40)

or

Out of memory! (linux perl 5.38)

instead of a

Segmentation fault

[tonyc]

Fixed by https://github.com/Perl/perl5/commit/c7e97f620c06b04f0ab6647608212282ef1901b6

[LanX]

I'm not sure why you get different results, but I'm pretty sure using Perl IO via open is better suited than buffering stuff from qx

Anyway from the docs

• In scalar context, it comes back as a single (potentially multi-line) string, or undef if the shell (or command) could not be started. In list context, returns a list of lines (however you've defined lines with $ or $INPUT_RECORD_SEPARATOR), or an empty list if the shell (or command) could not be started./
So your list context might lead to splitting into smaller parts ("lines")

[choroba]

I can confirm the behaviour. OS is Linux, Ubuntu 22.04.

With Perl 5.34.0:

$ time perl -wE 'my $x = `cat 2`'

real    0m4.558s
user    0m0.802s
sys    0m3.192s

$ time perl -wE 'my $x = `cat 1`'
Segmentation fault (core dumped)

real    1m28.649s
user    0m0.430s
sys    0m2.341s

With Perl 5.39.10:

$ time blead -wE 'my $x = `cat 2`'

real    0m8.851s
user    0m0.980s
sys    0m3.795s

$ time blead -wE 'my $x = `cat 1`'
Segmentation fault      (core dumped)

real    0m5.694s
user    0m0.397s
sys    0m1.378s

So the newer perl still fails, but much faster :-)

[cavac]

So the newer perl still fails, but much faster :-)

So much for "Perl is slow". We even optimize the performance of bugs ;-)

But really, i rather suspect that after the first run with 5.34, the operating system cached parts of the file in memory.

[choroba]

You are right. Repeated experiments show comparable times across Perl versions.