cross posted to p5p mailing list
I decided to test a blead perl compiled with Visual C 2003 against Visual C 2008 with perlbench-run. With ithreads and psuedofork (normal win32 config) on 32 bit windows. The 2 tests were run in a Perl folder, that was identical except for perl517.dll and HiRes.dll (the VC 2003 HiRes.dll crashed when run with a VC 2008 perl517.dll, I did not pursue the bug) which were copied from the VC 2008 build on another machine. So all XS dlls on the VC 2008 were compiled with VC 2003 except for Time::HiRes. explorer.exe was the only GUI process running during the test with a context switch delta above 2 in Process Explorer so CPU had almost no background load, I killed every thing else. I prune the Win32 services list to a minimum on my PCs. Tests were on a T7200 Core 2 Duo with CPUs locked to 2 GHZ (no power savings). perlbench-run is not portable and is not Windows compatible. I did a bare minimum of changes to it for it to run without fatally erroring on Windows. -G7 flag does not exist on 2008. I saw no replacement for it on 2008. Remember the 2003 and 2008 perl517s are linked against different CRTs/clibs. Security cookie aka stack frame overrun checking (-GS) is off by default in 2003, I turned it off in 2008 for apple to apple (not standard in Perl's win32 makefile). I also turned on -GL (inter procedure optimization) for 2003 and 2008 to give each the best chance to optimize (not standard in Perl's win32 makefile).
Not Cfg=Debug.
Both are "Snapshot of: 5c26a17602c4585ca111bd154f6c65c681cd0572"
First perlbench run, it took about 5 mins to run.
second run of perlbench, i upped the try count to see if anything signifigant shows up at higher test cycles. It took about 30 mins.
So what is the lesson here? Older is better? newer is not better? Or useless mucking by MS over the years to the compiler that turns out to cancel itself out? Or am I testing the frequency of hardware interrupts and ethernet broadcast traffic? or "faster" and "aligned" is self defeating if it causes something to not be in the CPU cache because it is fatter? Or am I testing changes of MS's clibs from 2003 to 2008? or is perlbench-run a bogus inaccurate test that does not reflect real world conditions?
I decided to test a blead perl compiled with Visual C 2003 against Visual C 2008 with perlbench-run. With ithreads and psuedofork (normal win32 config) on 32 bit windows. The 2 tests were run in a Perl folder, that was identical except for perl517.dll and HiRes.dll (the VC 2003 HiRes.dll crashed when run with a VC 2008 perl517.dll, I did not pursue the bug) which were copied from the VC 2008 build on another machine. So all XS dlls on the VC 2008 were compiled with VC 2003 except for Time::HiRes. explorer.exe was the only GUI process running during the test with a context switch delta above 2 in Process Explorer so CPU had almost no background load, I killed every thing else. I prune the Win32 services list to a minimum on my PCs. Tests were on a T7200 Core 2 Duo with CPUs locked to 2 GHZ (no power savings). perlbench-run is not portable and is not Windows compatible. I did a bare minimum of changes to it for it to run without fatally erroring on Windows. -G7 flag does not exist on 2008. I saw no replacement for it on 2008. Remember the 2003 and 2008 perl517s are linked against different CRTs/clibs. Security cookie aka stack frame overrun checking (-GS) is off by default in 2003, I turned it off in 2008 for apple to apple (not standard in Perl's win32 makefile). I also turned on -GL (inter procedure optimization) for 2003 and 2008 to give each the best chance to optimize (not standard in Perl's win32 makefile).
VC 2003 optimize='-MD -Zi -DNDEBUG -O1 -G7 -GL -arch:SSE2' VC 2008 optimize='-MD -Zi -DNDEBUG -O1 -GL -GS- -arch:SSE2'
Not Cfg=Debug.
Both are "Snapshot of: 5c26a17602c4585ca111bd154f6c65c681cd0572"
In trying to figure out why the code bloat, the most striking thing was, for sv->sv_flags, VC 2003 tests them with test char * [sv+13], 0x08, while VC 2008 does test I32 * [sv+12], 0x00000800 which takes much more machine code to fit that constant in. There are probably other things my eye didn't notice. Anyway, here the results.VC 2003 perl517.dll 956 KB on disk .text 0xC17CF .rdata 0x2140A .data 0x3938 .reloc 0x5F38 VC 2008 perl517.dll 975 KB on disk .text 0xC834A .rdata 0x2178A .data 0x3C58 .rsrc 0x3C0 .reloc 0x622C
First perlbench run, it took about 5 mins to run.
C:\sources\perlbench-0.93>perl perlbench-run -c 100000 "c:/perl517/bin +/perl.exe" "C:\perl517\vc08\bin\perl.exe" Use of uninitialized value in concatenation (.) or string at perlbench +-run line 75. A) version = 5.017006 path = c:/perl517/bin/perl.exe Use of uninitialized value in concatenation (.) or string at perlbench +-run line 75. B) version = 5.017006 path = C:\perl517\vc08\bin\perl.exe A B --- --- arith/mixed 100 114 arith/trig 100 121 array/copy 100 103 array/foreach 100 95 array/index 100 102 array/pop 100 103 array/shift 100 104 array/sort-num 100 100 array/sort 100 110 call/0arg 100 104 call/1arg 100 97 call/2arg 100 105 call/9arg 100 97 call/empty 100 96 call/fib 100 91 call/method 100 90 call/wantarray 100 108 hash/copy 100 100 hash/each 100 105 hash/foreach-sort 100 103 hash/foreach 100 96 hash/get 100 105 hash/set 100 105 loop/for-c 100 106 loop/for-range-const 100 88 loop/for-range 100 89 loop/getline 100 103 loop/while-my 100 102 loop/while 100 101 re/const 100 99 re/w 100 97 startup/fewmod 100 - startup/lotsofsub - - startup/noprog 100 86 string/base64 100 99 string/htmlparser 100 101 string/index-const 100 121 string/index-var 100 102 string/ipol 100 111 string/tr 100 94 AVERAGE 100 101Use of uninitialized value $str in + substitut ion (s///) at perlbench-run line 375. Use of uninitialized value $str in substitution (s///) at perlbench-ru +n line 376 . Use of uninitialized value in concatenation (.) or string at perlbench +-run line 287. Use of uninitialized value $str in substitution (s///) at perlbench-ru +n line 375 . Use of uninitialized value $str in substitution (s///) at perlbench-ru +n line 376 . Use of uninitialized value in concatenation (.) or string at perlbench +-run line 287. Results saved in file:///C|/sources/perlbench-0.93/benchres-001/index. +html C:\sources\perlbench-0.93>
C:\sources\perlbench-0.93>perl perlbench-run -c 500000 "c:/perl517/bin +/perl.exe" "C:\perl517\vc08\bin\perl.exe" Use of uninitialized value in concatenation (.) or string at perlbench +-run line 75. A) version = 5.017006 path = c:/perl517/bin/perl.exe Use of uninitialized value in concatenation (.) or string at perlbench +-run line 75. B) version = 5.017006 path = C:\perl517\vc08\bin\perl.exe A B --- --- arith/mixed 100 107 arith/trig 100 120 array/copy 100 103 array/foreach 100 95 array/index 100 99 array/pop 100 104 array/shift 100 104 array/sort-num 100 99 array/sort 100 109 call/0arg 100 106 call/1arg 100 96 call/2arg 100 104 call/9arg 100 98 call/empty 100 97 call/fib 100 92 call/method 100 91 call/wantarray 100 105 hash/copy 100 100 hash/each 100 104 hash/foreach-sort 100 103 hash/foreach 100 97 hash/get 100 103 hash/set 100 108 loop/for-c 100 104 loop/for-range-const 100 90 loop/for-range 100 89 loop/getline 100 104 loop/while-my 100 103 loop/while 100 102 re/const 100 99 re/w 100 99 startup/fewmod 100 90 startup/lotsofsub 100 - startup/noprog 100 104 string/base64 100 100 string/htmlparser 100 101 string/index-const 100 118 string/index-var 100 101 string/ipol 100 110 string/tr 100 91 AVERAGE 100 101Use of uninitialized value $str in + substitut ion (s///) at perlbench-run line 375. Use of uninitialized value $str in substitution (s///) at perlbench-ru +n line 376 . Use of uninitialized value in concatenation (.) or string at perlbench +-run line 287. Use of uninitialized value $str in substitution (s///) at perlbench-ru +n line 375 . Use of uninitialized value $str in substitution (s///) at perlbench-ru +n line 376 . Use of uninitialized value in concatenation (.) or string at perlbench +-run line 287. Results saved in file:///C|/sources/perlbench-0.93/benchres-002/index. +html C:\sources\perlbench-0.93>
|
|---|
Back to
Meditations