Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

unusual perl segfaults

by Anonymous Monk
on Nov 07, 2012 at 21:17 UTC ( #1002749=perlquestion: print w/ replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I have a non-root user that are causing coredumps intermittently when the perl command was called.
[~]# perl Segmentation fault (core dumped)
running the dump file against gdb displays:
gdb perl core.4639 GNU gdb (GDB) CentOS (7.0.1-42.el5.centos.1) Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gp +l.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copy +ing" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/local/bin/perl...(no debugging symbols found +)...done. [New Thread 4639] Reading symbols from /lib64/libresolv.so.2...(no debugging symbols fou +nd)...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /lib64/libnsl.so.1...(no debugging symbols found) +...done. Loaded symbols for /lib64/libnsl.so.1 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found). +..done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/libm.so.6...(no debugging symbols found).. +.done. Loaded symbols for /lib64/libm.so.6 Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols foun +d)...done. Loaded symbols for /lib64/libcrypt.so.1 Reading symbols from /lib64/libutil.so.1...(no debugging symbols found +)...done. Loaded symbols for /lib64/libutil.so.1 Reading symbols from /lib64/libc.so.6...(no debugging symbols found).. +.done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libp +erl.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libper +l.so Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbo +ls found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 warning: no loadable sections found in added symbol-file system-suppli +ed DSO at 0x7ffff0dfd000 Core was generated by `perl'. Program terminated with signal 11, Segmentation fault. #0 0x00002b7ac4367901 in S_hv_fetch_common () from /usr/local/lib/per +l5/5.8.8/x86_64-linux/CORE/libperl.so
Nothing helpful on /var/log/messages
Nov 7 15:12:02 vps kernel: perl[5447]: segfault at 00007fff78ff5ff8 r +ip 00002acd1876d90e rsp 00007fff78ff6000 error 6
I even tried checking the strace when it segfaults, to see if there is some helpful information.
execve("/usr/local/bin/perl", ["perl"], [/* 31 vars */]) = 0 brk(0) = 0x1a1fb000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, +0) = 0x2b67b0d6a000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, +0) = 0x2b67b0d6b000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or d +irectory) open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/tls/x86_64/libresol +v.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/tls/x86_64", 0x7fff +9d796ff0) = -1 ENOENT (No such file or directory) open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/tls/libresolv.so.2" +, O_RDONLY) = -1 ENOENT (No such file or directory) stat("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/tls", 0x7fff9d796ff +0) = -1 ENOENT (No such file or directory) open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/x86_64/libresolv.so +.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/x86_64", 0x7fff9d79 +6ff0) = -1 ENOENT (No such file or directory) open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libresolv.so.2", O_ +RDONLY) = -1 ENOENT (No such file or directory) stat("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE", {st_mode=S_IFDIR| +0755, st_size=4096, ...}) = 0 open("/etc/ld.so.cache", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=70607, ...}) = 0 mmap(NULL, 70607, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b67b0d6c000 close(3) = 0 open("/lib64/libresolv.so.2", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0203\200\2415\ +0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=92816, ...}) = 0 mmap(0x35a1800000, 2181896, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYW +RITE, 3, 0) = 0x35a1800000 mprotect(0x35a1811000, 2097152, PROT_NONE) = 0 mmap(0x35a1a11000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_DENYWRITE, 3, 0x11000) = 0x35a1a11000 mmap(0x35a1a13000, 6920, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_ANONYMOUS, -1, 0) = 0x35a1a13000 close(3) = 0 open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libnsl.so.1", O_RDO +NLY) = -1 ENOENT (No such file or directory) open("/lib64/libnsl.so.1", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240@\300\2365\ +0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=114352, ...}) = 0 mmap(0x359ec00000, 2194096, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYW +RITE, 3, 0) = 0x359ec00000 mprotect(0x359ec15000, 2093056, PROT_NONE) = 0 mmap(0x359ee14000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_DENYWRITE, 3, 0x14000) = 0x359ee14000 mmap(0x359ee16000, 6832, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_ANONYMOUS, -1, 0) = 0x359ee16000 close(3) = 0 open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libdl.so.2", O_RDON +LY) = -1 ENOENT (No such file or directory) open("/lib64/libdl.so.2", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\16\0\2355\0 +\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=23360, ...}) = 0 mmap(0x359d000000, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYW +RITE, 3, 0) = 0x359d000000 mprotect(0x359d002000, 2097152, PROT_NONE) = 0 mmap(0x359d202000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_DENYWRITE, 3, 0x2000) = 0x359d202000 close(3) = 0 open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libm.so.6", O_RDONL +Y) = -1 ENOENT (No such file or directory) open("/lib64/libm.so.6", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`>\200\2355\0\0 +\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=614992, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, +0) = 0x2b67b0d7e000 mmap(0x359d800000, 2629816, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYW +RITE, 3, 0) = 0x359d800000 mprotect(0x359d882000, 2093056, PROT_NONE) = 0 mmap(0x359da81000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_DENYWRITE, 3, 0x81000) = 0x359da81000 close(3) = 0 open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libcrypt.so.1", O_R +DONLY) = -1 ENOENT (No such file or directory) open("/lib64/libcrypt.so.1", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\t\0\2375\0 +\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=48600, ...}) = 0 mmap(0x359f000000, 2322880, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYW +RITE, 3, 0) = 0x359f000000 mprotect(0x359f009000, 2093056, PROT_NONE) = 0 mmap(0x359f208000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_DENYWRITE, 3, 0x8000) = 0x359f208000 mmap(0x359f20a000, 184768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED +|MAP_ANONYMOUS, -1, 0) = 0x359f20a000 close(3) = 0 open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libutil.so.1", O_RD +ONLY) = -1 ENOENT (No such file or directory) open("/lib64/libutil.so.1", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\16\300\240 +5\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=18152, ...}) = 0 mmap(0x35a0c00000, 2105616, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYW +RITE, 3, 0) = 0x35a0c00000 mprotect(0x35a0c02000, 2093056, PROT_NONE) = 0 mmap(0x35a0e01000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_DENYWRITE, 3, 0x1000) = 0x35a0e01000 close(3) = 0 open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libc.so.6", O_RDONL +Y) = -1 ENOENT (No such file or directory) open("/lib64/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\332\301\23 +45\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=1718232, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, +0) = 0x2b67b0d7f000 mmap(0x359cc00000, 3498328, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYW +RITE, 3, 0) = 0x359cc00000 mprotect(0x359cd4e000, 2093056, PROT_NONE) = 0 mmap(0x359cf4d000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED| +MAP_DENYWRITE, 3, 0x14d000) = 0x359cf4d000 mmap(0x359cf52000, 16728, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED| +MAP_ANONYMOUS, -1, 0) = 0x359cf52000 close(3) = 0 open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libperl.so", O_RDON +LY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\375\2\0\0\0 +\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0555, st_size=1128926, ...}) = 0 mmap(NULL, 3140288, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, + 0) = 0x2b67b0d80000 mprotect(0x2b67b0e73000, 2097152, PROT_NONE) = 0 mmap(0x2b67b1073000, 36864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXE +D|MAP_DENYWRITE, 3, 0xf3000) = 0x2b67b1073000 mmap(0x2b67b107c000, 10944, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXE +D|MAP_ANONYMOUS, -1, 0) = 0x2b67b107c000 close(3) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, +0) = 0x2b67b107f000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, +0) = 0x2b67b1080000 arch_prctl(ARCH_SET_FS, 0x2b67b107fc30) = 0 mprotect(0x359cf4d000, 16384, PROT_READ) = 0 mprotect(0x35a0e01000, 4096, PROT_READ) = 0 mprotect(0x359f208000, 4096, PROT_READ) = 0 mprotect(0x359da81000, 4096, PROT_READ) = 0 mprotect(0x359d202000, 4096, PROT_READ) = 0 mprotect(0x359ee14000, 4096, PROT_READ) = 0 mprotect(0x35a1a11000, 4096, PROT_READ) = 0 mprotect(0x359ca1c000, 4096, PROT_READ) = 0 munmap(0x2b67b0d6c000, 70607) = 0 getuid() = 503 geteuid() = 503 getgid() = 501 getegid() = 501 rt_sigaction(SIGFPE, {0x1, [FPE], SA_RESTORER|SA_RESTART, 0x359cc302f0 +}, {SIG_DFL, [], 0}, 8) = 0 brk(0) = 0x1a1fb000 brk(0x1a21c000) = 0x1a1fb000 mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, - +1, 0) = 0x2b67b1081000 brk(0x1a207000) = 0x1a1fb000 brk(0x1a207000) = 0x1a1fb000 brk(0x1a207000) = 0x1a1fb000 brk(0x1a207000) = 0x1a1fb000 brk(0x1a207000) = 0x1a1fb000 brk(0x1a207000) = 0x1a1fb000 brk(0x1a207000) = 0x1a1fb000 brk(0x1a207000) = 0x1a1fb000
the brk calls, continues until it segfaults. I was wondering if it was related to memory limitations for the user but I can't say if it was causing it.
[~]# ulimit -a core file size (blocks, -c) 200000 data seg size (kbytes, -d) 200000 scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 16384 max locked memory (kbytes, -l) 32 max memory size (kbytes, -m) 200000 open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 35 virtual memory (kbytes, -v) 200000 file locks (-x) unlimited
I don't know what else I should check. It doesn't happen with root user. And when I ran the gdb perl for that same user it never segfaults. help!

Comment on unusual perl segfaults
Select or Download Code
Re: unusual perl segfaults
by Anonymous Monk on Nov 07, 2012 at 21:20 UTC
    perl version is perl-5.8.8-32.el5_6.3
    [~]# perl --version This is perl, v5.8.8 built for x86_64-linux
    OS is CentOS release 5.8
      additional info
      Summary of my perl5 (revision 5 version 8 subversion 8) configuration: Platform: osname=linux, osvers=2.6.18-128.el5xen, archname=x86_64-linux uname='linux default.localhost.com 2.6.18-128.el5xen #1 smp wed ja +n 21 11:12:42 est 2009 x86_64 x86_64 x86_64 gnulinux ' config_args='-des -Dprefix=/usr/local -Doptimize=-Os -Duseshrplib +-Dusemymalloc=y -Accflags=-fPIC -DPIC -m64 -Dlibpth=/usr/local/lib64 +/lib64 /usr/lib64 -Aldflags=-L/usr/lib64 -L/lib64 -lresolv -lnsl -ldl + -lm -lcrypt -lutil -lc' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultipl +icity=undef useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=y, bincompat5005=undef Compiler: cc='cc', ccflags ='-fPIC -DPIC -m64 -fno-strict-aliasing -pipe -Wd +eclaration-after-statement -I/usr/local/include -D_LARGEFILE_SOURCE - +D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm', optimize='-Os', cppflags='-fPIC -DPIC -m64 -fno-strict-aliasing -pipe -Wdeclaratio +n-after-statement -I/usr/local/include -I/usr/include/gdbm' ccversion='', gccversion='4.1.2 20080704 (Red Hat 4.1.2-51)', gcco +sandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=1 +6 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', + lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/lib64 -L/lib64 -lresolv -lnsl -ldl -lm +-lcrypt -lutil -lc' libpth=/usr/local/lib64 /lib64 /usr/lib64 libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=/lib/libc-2.5.so, so=so, useshrplib=true, libperl=libperl.so gnulibc_version='2.5' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E - +Wl,-rpath,/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE' cccdlflags='-fpic', lddlflags='-shared -L/usr/lib64 -L/lib64' Characteristics of this binary (from libperl): Compile-time options: MYMALLOC PERL_MALLOC_WRAP USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_PERLIO Built under linux Compiled at Nov 25 2011 14:21:17 @INC: /usr/local/lib/perl5/5.8.8/x86_64-linux /usr/local/lib/perl5/5.8.8 /usr/local/lib/perl5/site_perl/5.8.8/x86_64-linux /usr/local/lib/perl5/site_perl/5.8.8 /usr/local/lib/perl5/site_perl .
        it seems i was able to resolve my problem by setting the data segment size value.
        ulimit -n 1024 -u 35 -m 200000 -d unlimited -s 8192 -c 200000 -v 2000 +00
        after setting I didn't experience core dumps anymore.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1002749]
Approved by davido
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (5)
As of 2014-07-12 06:37 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    When choosing user names for websites, I prefer to use:








    Results (238 votes), past polls