I have to parse a 50MB file and HAVE TO put the WHOLE file in a dynamic created structure (pointers to arrays of pointers to arrays...;-) into memory. Well it's hard enough that the memory consumption is near to 19 times the filesize > 950MB, but the real hard problem is that the perl script terminiates with "Out of memory!" statment.
So this would not be surprise if i would have not more than
ca 1GB of RAM but the machine has 14GB free memory (and NO the GB is no typo ;-)
$>uname -a
HP-UX edsserv3 B.11.00 U 9000/800 613309356 unlimited-user license
$>ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) unlimited
stack(kbytes) 147456
memory(kbytes) unlimited
coredump(blocks) 4194303
$>perl -V
Summary of my perl5 (revision 5.0 version 6 subversion 0) configuratio
+n:
Platform:
osname=hpux, osvers=11.00, archname=PA-RISC2.0
uname='hp-ux hostname b.11.00 u 9000800 613309356 unlimited-user l
+icense '
config_args=''
hint=recommended, useposix=true, d_sigaction=define
usethreads=undef use5005threads=undef useithreads=undef usemultipl
+icity=undef
useperlio=undef d_sfio=undef uselargefiles=define
use64bitint=undef use64bitall=undef uselongdouble=undef usesocks=u
+ndef
Compiler:
cc='cc', optimize='-O +Onolimit', gccversion=
cppflags='-Ae'
ccflags =' -Ae -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 '
stdchar='unsigned char', d_stdstdio=define, usevfork=false
intsize=4, longsize=4, ptrsize=4, doublesize=8
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=1
+6
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
+ lseeksize=8
alignbytes=8, usemymalloc=y, prototype=define
Linker and Libraries:
ld='ld', ldflags =''
libpth=/usr/local/lib /lib /usr/lib /usr/ccs/lib
libs=-lnsl -lnm -lndbm -ldld -lm -lc -lndir -lcrypt -lsec
libc=/lib/libc.sl, so=sl, useshrplib=false, libperl=libperl.a
Dynamic Linking:
dlsrc=dl_hpux.xs, dlext=sl, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl
+,-B,deferred '
cccdlflags='+z', lddlflags='-b -s -a shared'
Characteristics of this binary (from libperl):
Compile-time options: USE_LARGE_FILES
Built under hpux
Compiled at Mar 6 2001 18:11:51
%ENV:
PERL5LIB=".:/home/ucsab/bin:/hot_work/develop/main/tools"
PERL_DEBUG_MSTATS="2"
@INC:
.
/home/ucsab/bin
/hot_work/develop/main/tools
/opt/perl5/lib/5.6.0/PA-RISC2.0
/opt/perl5/lib/5.6.0
/opt/perl5/lib/site_perl/5.6.0/PA-RISC2.0
/opt/perl5/lib/site_perl/5.6.0
/opt/perl5/lib/site_perl
.
$>top
System: hostname Tue Feb 10 07:31:39
+2004
Load averages: 0.57, 0.52, 0.47
250 processes: 245 sleeping, 5 running
Cpu states:
CPU LOAD USER NICE SYS IDLE BLOCK SWAIT INTR SSYS
0 0.45 1.0% 0.0% 4.0% 95.0% 0.0% 0.0% 0.0% 0.0%
1 0.72 9.1% 0.0% 7.3% 83.5% 0.0% 0.0% 0.0% 0.0%
2 1.00 100.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
3 0.52 23.4% 0.0% 4.2% 72.4% 0.0% 0.0% 0.0% 0.0%
4 0.43 11.5% 0.0% 10.9% 77.6% 0.0% 0.0% 0.0% 0.0%
5 0.41 9.5% 0.0% 3.6% 86.9% 0.0% 0.0% 0.0% 0.0%
6 0.41 3.0% 0.0% 6.7% 90.3% 0.0% 0.0% 0.0% 0.0%
7 0.64 8.3% 0.0% 3.4% 88.3% 0.0% 0.0% 0.0% 0.0%
--- ---- ----- ----- ----- ----- ----- ----- ----- -----
avg 0.57 20.8% 0.0% 5.1% 74.1% 0.0% 0.0% 0.0% 0.0%
Memory: 1373464K (1254588K) real, 1364092K (1263948K) virtual, 1366568
+8K free Page# 1/32
CPU TTY PID USERNAME PRI NI SIZE RES STATE TIME %WCPU %CPU
+ COMMAND
2 pts/1 8404 ucsab 241 20 905M 898M run 9:24 99.78 99.60
+ perl
3 ? 8678 ora_rman 241 20 45404K 17088K run 1:00 62.25 61.76
+ oracleOBAA
$>skript.pl
...
Out of memory!
Error: Script <skript.pl> returned exit value <1>!
$>