my versus our, why is my slower when script ends (global cleanup)

[marioroy]

Respected Monks,

On Mac OS X and Linux (not tested on other Unix platforms), there appears to be extra overhead by Perl prior to the script exiting for variables declared with my. The extra time is noticeable after seeing the "end" output.

Why does Perl have this odd behavior during cleanup?

use strict;
use warnings;

$| = 1;

my $size = 2e6;

my $data =
  [ 'AGCTCGTTGTTCGATCCA', 'GAGAGATAGATGATAGTG', 'TTTT_CCCC', 0 ];

print "begin\n";
my %barcode_hash = map { $_ => $data } 1 .. $size;
print "end\n";

There is practically no delay after seeing "end" if I declare the variable with our.

...
our %barcode_hash = map { $_ => $data } 1 .. $size;
...

I made a script to capture the running time, helpful on Windows. Unix has time command and shell builtin time.

use strict;
use warnings;

use Time::HiRes 'time';

my $start_time = time;
system(@ARGV) == 0 or die "\"@ARGV\" failed with error: $?";

printf "%0.03f seconds\n", time - $start_time;

Results:

perl timeit.pl perl test.pl

my  %big_hash:  2.726 seconds
our %big_hash:  1.351 seconds

Thank you, karlgoethebier for this enlightenment. I do not understand what Perl is doing during cleanup. Has anyone encountered this behavior? I tested on Windows and is nothing like seen on Mac OS X and Linux.

Thanks, Mario

[Anonymous Monk]

The reason is simple , lexicals (my) are destroyed/destructored/cleanedup in-order

Where as, globals are cleaned up during "global destruction" before program exits but not in-order as there is no order they're globals

You can short-circuit cleanup with POSIX::exit() for even faster exit, but then destructors and stuff dont ...

Example

##  $stuff not exist
{  our $stuff = 1234; }
## $stuff still exist
{ my $lexi = 1234; }
## $lexi cleaned up here but $stuff still exists
{ my $flexi = 1234; } 
## $flexi cleanedup but stuff still exists
exit;
## $stuff cleaned up here in whatever fast order it happens to come fr
+om the perl memory stack

There are discussions on this just search for global destruction order, site:perlmonks.org global destruction order

see sub DESTROY: Strange ordering of object destruction (global destruction order not guaranteed, Re^3: when is destroy function called ( perl -d:Trace dalek ), Re^2: Perl Gtk2 - ->destroy() is Not Causing the Script to Exit

Why does Perl have this odd behavior during cleanup?

Heheh, its always funny when people (me too) say something is odd/weird ... almost as if they're trying to say I know stuff but this doesn't behave like it should (i used to think this but I know better now)... but what they really mean is always I don't know what should be happening, why is there a difference in what is happening, even if they don't think this is what they mean

[marioroy]

Thank you for this enlightenment. I'm learning Perl all the time. I don't know what should be happening, why there is a difference until reading your reply. I tested with require POSIX and POSIX::_exit(0) placed at the end of the script. The MCE and MCE::Hobo modules call kill 9 when posix_exit => 1 is given for minimum consumption so not having to load POSIX before spawning workers.

use strict;
use warnings;

$| = 1;

my $size = 2e6;

my $data =
  [ 'AGCTCGTTGTTCGATCCA', 'GAGAGATAGATGATAGTG', 'TTTT_CCCC', 0 ];

print "begin\n";

my %barcode_hash = map { $_ => $data } 1 .. $size;

print "end\n";

require POSIX;
POSIX::_exit(0);

Calling POSIX::_exit(0) seems helpful for scripts making a big array or hash. Regarding MCE, it means having to call MCE::Loop->finish or $mce->shutdown explicitly so that socket handles and what not are released immediately and not linger around until the OS releases them. Regarding MCE::Shared, calling MCE::Shared->stop prior to exiting.

Thank you, Anonymous_Monk. I never thought to call POSIX::_exit(0) from the main process. That works out well. I'm going to take the next hour or so and update the few examples in reply to karlgoethebier's post. Out to the rescue for this wonderful Monk++. All credits to you++ for explaining what's going on.

Regards, Mario.

Update: Oh my. I like having the destructors run in the event omitting calling MCE::Loop->finish or $mce->shutdown. It's nice that Perl gives us the option to declare a variable global when having to construct a very big array or hash and still have destructors run.