Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

unusual perl segfaults

by Anonymous Monk
on Nov 07, 2012 at 21:17 UTC ( [id://1002749]=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I have a non-root user that are causing coredumps intermittently when the perl command was called.
[~]# perl Segmentation fault (core dumped)
running the dump file against gdb displays:
gdb perl core.4639 GNU gdb (GDB) CentOS (7.0.1-42.el5.centos.1) Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gp +l.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copy +ing" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/local/bin/perl...(no debugging symbols found +)...done. [New Thread 4639] Reading symbols from /lib64/libresolv.so.2...(no debugging symbols fou +nd)...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /lib64/libnsl.so.1...(no debugging symbols found) +...done. Loaded symbols for /lib64/libnsl.so.1 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found). +..done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/libm.so.6...(no debugging symbols found).. +.done. Loaded symbols for /lib64/libm.so.6 Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols foun +d)...done. Loaded symbols for /lib64/libcrypt.so.1 Reading symbols from /lib64/libutil.so.1...(no debugging symbols found +)...done. Loaded symbols for /lib64/libutil.so.1 Reading symbols from /lib64/libc.so.6...(no debugging symbols found).. +.done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libp +erl.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libper +l.so Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbo +ls found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 warning: no loadable sections found in added symbol-file system-suppli +ed DSO at 0x7ffff0dfd000 Core was generated by `perl'. Program terminated with signal 11, Segmentation fault. #0 0x00002b7ac4367901 in S_hv_fetch_common () from /usr/local/lib/per +l5/5.8.8/x86_64-linux/CORE/libperl.so
Nothing helpful on /var/log/messages
Nov 7 15:12:02 vps kernel: perl[5447]: segfault at 00007fff78ff5ff8 r +ip 00002acd1876d90e rsp 00007fff78ff6000 error 6
I even tried checking the strace when it segfaults, to see if there is some helpful information.
execve("/usr/local/bin/perl", ["perl"], [/* 31 vars */]) = 0 brk(0) = 0x1a1fb000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, +0) = 0x2b67b0d6a000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, +0) = 0x2b67b0d6b000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or d +irectory) open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/tls/x86_64/libresol +v.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/tls/x86_64", 0x7fff +9d796ff0) = -1 ENOENT (No such file or directory) open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/tls/libresolv.so.2" +, O_RDONLY) = -1 ENOENT (No such file or directory) stat("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/tls", 0x7fff9d796ff +0) = -1 ENOENT (No such file or directory) open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/x86_64/libresolv.so +.2", O_RDONLY) = -1 ENOENT (No such file or directory) stat("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/x86_64", 0x7fff9d79 +6ff0) = -1 ENOENT (No such file or directory) open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libresolv.so.2", O_ +RDONLY) = -1 ENOENT (No such file or directory) stat("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE", {st_mode=S_IFDIR| +0755, st_size=4096, ...}) = 0 open("/etc/ld.so.cache", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=70607, ...}) = 0 mmap(NULL, 70607, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b67b0d6c000 close(3) = 0 open("/lib64/libresolv.so.2", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0203\200\2415\ +0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=92816, ...}) = 0 mmap(0x35a1800000, 2181896, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYW +RITE, 3, 0) = 0x35a1800000 mprotect(0x35a1811000, 2097152, PROT_NONE) = 0 mmap(0x35a1a11000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_DENYWRITE, 3, 0x11000) = 0x35a1a11000 mmap(0x35a1a13000, 6920, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_ANONYMOUS, -1, 0) = 0x35a1a13000 close(3) = 0 open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libnsl.so.1", O_RDO +NLY) = -1 ENOENT (No such file or directory) open("/lib64/libnsl.so.1", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240@\300\2365\ +0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=114352, ...}) = 0 mmap(0x359ec00000, 2194096, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYW +RITE, 3, 0) = 0x359ec00000 mprotect(0x359ec15000, 2093056, PROT_NONE) = 0 mmap(0x359ee14000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_DENYWRITE, 3, 0x14000) = 0x359ee14000 mmap(0x359ee16000, 6832, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_ANONYMOUS, -1, 0) = 0x359ee16000 close(3) = 0 open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libdl.so.2", O_RDON +LY) = -1 ENOENT (No such file or directory) open("/lib64/libdl.so.2", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\16\0\2355\0 +\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=23360, ...}) = 0 mmap(0x359d000000, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYW +RITE, 3, 0) = 0x359d000000 mprotect(0x359d002000, 2097152, PROT_NONE) = 0 mmap(0x359d202000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_DENYWRITE, 3, 0x2000) = 0x359d202000 close(3) = 0 open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libm.so.6", O_RDONL +Y) = -1 ENOENT (No such file or directory) open("/lib64/libm.so.6", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`>\200\2355\0\0 +\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=614992, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, +0) = 0x2b67b0d7e000 mmap(0x359d800000, 2629816, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYW +RITE, 3, 0) = 0x359d800000 mprotect(0x359d882000, 2093056, PROT_NONE) = 0 mmap(0x359da81000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_DENYWRITE, 3, 0x81000) = 0x359da81000 close(3) = 0 open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libcrypt.so.1", O_R +DONLY) = -1 ENOENT (No such file or directory) open("/lib64/libcrypt.so.1", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\t\0\2375\0 +\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=48600, ...}) = 0 mmap(0x359f000000, 2322880, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYW +RITE, 3, 0) = 0x359f000000 mprotect(0x359f009000, 2093056, PROT_NONE) = 0 mmap(0x359f208000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_DENYWRITE, 3, 0x8000) = 0x359f208000 mmap(0x359f20a000, 184768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED +|MAP_ANONYMOUS, -1, 0) = 0x359f20a000 close(3) = 0 open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libutil.so.1", O_RD +ONLY) = -1 ENOENT (No such file or directory) open("/lib64/libutil.so.1", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\16\300\240 +5\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=18152, ...}) = 0 mmap(0x35a0c00000, 2105616, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYW +RITE, 3, 0) = 0x35a0c00000 mprotect(0x35a0c02000, 2093056, PROT_NONE) = 0 mmap(0x35a0e01000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|M +AP_DENYWRITE, 3, 0x1000) = 0x35a0e01000 close(3) = 0 open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libc.so.6", O_RDONL +Y) = -1 ENOENT (No such file or directory) open("/lib64/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\332\301\23 +45\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=1718232, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, +0) = 0x2b67b0d7f000 mmap(0x359cc00000, 3498328, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYW +RITE, 3, 0) = 0x359cc00000 mprotect(0x359cd4e000, 2093056, PROT_NONE) = 0 mmap(0x359cf4d000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED| +MAP_DENYWRITE, 3, 0x14d000) = 0x359cf4d000 mmap(0x359cf52000, 16728, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED| +MAP_ANONYMOUS, -1, 0) = 0x359cf52000 close(3) = 0 open("/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE/libperl.so", O_RDON +LY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\375\2\0\0\0 +\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0555, st_size=1128926, ...}) = 0 mmap(NULL, 3140288, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, + 0) = 0x2b67b0d80000 mprotect(0x2b67b0e73000, 2097152, PROT_NONE) = 0 mmap(0x2b67b1073000, 36864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXE +D|MAP_DENYWRITE, 3, 0xf3000) = 0x2b67b1073000 mmap(0x2b67b107c000, 10944, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXE +D|MAP_ANONYMOUS, -1, 0) = 0x2b67b107c000 close(3) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, +0) = 0x2b67b107f000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, +0) = 0x2b67b1080000 arch_prctl(ARCH_SET_FS, 0x2b67b107fc30) = 0 mprotect(0x359cf4d000, 16384, PROT_READ) = 0 mprotect(0x35a0e01000, 4096, PROT_READ) = 0 mprotect(0x359f208000, 4096, PROT_READ) = 0 mprotect(0x359da81000, 4096, PROT_READ) = 0 mprotect(0x359d202000, 4096, PROT_READ) = 0 mprotect(0x359ee14000, 4096, PROT_READ) = 0 mprotect(0x35a1a11000, 4096, PROT_READ) = 0 mprotect(0x359ca1c000, 4096, PROT_READ) = 0 munmap(0x2b67b0d6c000, 70607) = 0 getuid() = 503 geteuid() = 503 getgid() = 501 getegid() = 501 rt_sigaction(SIGFPE, {0x1, [FPE], SA_RESTORER|SA_RESTART, 0x359cc302f0 +}, {SIG_DFL, [], 0}, 8) = 0 brk(0) = 0x1a1fb000 brk(0x1a21c000) = 0x1a1fb000 mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, - +1, 0) = 0x2b67b1081000 brk(0x1a207000) = 0x1a1fb000 brk(0x1a207000) = 0x1a1fb000 brk(0x1a207000) = 0x1a1fb000 brk(0x1a207000) = 0x1a1fb000 brk(0x1a207000) = 0x1a1fb000 brk(0x1a207000) = 0x1a1fb000 brk(0x1a207000) = 0x1a1fb000 brk(0x1a207000) = 0x1a1fb000
the brk calls, continues until it segfaults. I was wondering if it was related to memory limitations for the user but I can't say if it was causing it.
[~]# ulimit -a core file size (blocks, -c) 200000 data seg size (kbytes, -d) 200000 scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 16384 max locked memory (kbytes, -l) 32 max memory size (kbytes, -m) 200000 open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 35 virtual memory (kbytes, -v) 200000 file locks (-x) unlimited
I don't know what else I should check. It doesn't happen with root user. And when I ran the gdb perl for that same user it never segfaults. help!

Replies are listed 'Best First'.
Re: unusual perl segfaults
by Anonymous Monk on Nov 07, 2012 at 21:20 UTC
    perl version is perl-5.8.8-32.el5_6.3
    [~]# perl --version This is perl, v5.8.8 built for x86_64-linux
    OS is CentOS release 5.8
      additional info
      Summary of my perl5 (revision 5 version 8 subversion 8) configuration: Platform: osname=linux, osvers=2.6.18-128.el5xen, archname=x86_64-linux uname='linux default.localhost.com 2.6.18-128.el5xen #1 smp wed ja +n 21 11:12:42 est 2009 x86_64 x86_64 x86_64 gnulinux ' config_args='-des -Dprefix=/usr/local -Doptimize=-Os -Duseshrplib +-Dusemymalloc=y -Accflags=-fPIC -DPIC -m64 -Dlibpth=/usr/local/lib64 +/lib64 /usr/lib64 -Aldflags=-L/usr/lib64 -L/lib64 -lresolv -lnsl -ldl + -lm -lcrypt -lutil -lc' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultipl +icity=undef useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=y, bincompat5005=undef Compiler: cc='cc', ccflags ='-fPIC -DPIC -m64 -fno-strict-aliasing -pipe -Wd +eclaration-after-statement -I/usr/local/include -D_LARGEFILE_SOURCE - +D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm', optimize='-Os', cppflags='-fPIC -DPIC -m64 -fno-strict-aliasing -pipe -Wdeclaratio +n-after-statement -I/usr/local/include -I/usr/include/gdbm' ccversion='', gccversion='4.1.2 20080704 (Red Hat 4.1.2-51)', gcco +sandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=1 +6 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', + lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/lib64 -L/lib64 -lresolv -lnsl -ldl -lm +-lcrypt -lutil -lc' libpth=/usr/local/lib64 /lib64 /usr/lib64 libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=/lib/libc-2.5.so, so=so, useshrplib=true, libperl=libperl.so gnulibc_version='2.5' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E - +Wl,-rpath,/usr/local/lib/perl5/5.8.8/x86_64-linux/CORE' cccdlflags='-fpic', lddlflags='-shared -L/usr/lib64 -L/lib64' Characteristics of this binary (from libperl): Compile-time options: MYMALLOC PERL_MALLOC_WRAP USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_PERLIO Built under linux Compiled at Nov 25 2011 14:21:17 @INC: /usr/local/lib/perl5/5.8.8/x86_64-linux /usr/local/lib/perl5/5.8.8 /usr/local/lib/perl5/site_perl/5.8.8/x86_64-linux /usr/local/lib/perl5/site_perl/5.8.8 /usr/local/lib/perl5/site_perl .
        it seems i was able to resolve my problem by setting the data segment size value.
        ulimit -n 1024 -u 35 -m 200000 -d unlimited -s 8192 -c 200000 -v 2000 +00
        after setting I didn't experience core dumps anymore.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1002749]
Approved by davido
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others taking refuge in the Monastery: (4)
As of 2024-03-29 10:10 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found