Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Re^3: Threads and fork and CLONE, oh my!

by adrianh (Chancellor)
on Aug 17, 2005 at 15:08 UTC ( #484462=note: print w/ replies, xml ) Need Help??


in reply to Re^2: Threads and fork and CLONE, oh my!
in thread Threads and fork and CLONE, oh my!

for reasons lost to history, someone used "$self" (hey, it's unique, right?) as cheaper than generating a unique ID

Unfortunately it turns out that it isn't very cheap at all (speed wise). In fact it's just about the worse possible choice :-) On my perl 5.8.7 this basic benchmark:

#! /usr/bin/perl use strict; use warnings; { package Base; use Devel::Size (); sub make_n { my ($class, $n) = @_; my @objects; push @objects, $class->new for 1..$n; return \@objects; } sub hidden_stuff { return } sub size_of_n { my ($class, $n) = @_; my $objects = $class->make_n( $n ); my $stuff = [ $objects, $class->hidden_stuff ]; return( Devel::Size::total_size( $stuff ), $objects ); } sub do_something { my $class = shift; my $self = $class->new; $self->set_a( $self->get_a + 1 ) for ( 1 .. 100 ); die unless $self->get_a == 101; } } { package BlessedHash; use base qw( Base ); sub new { bless { a => 1, b => 2, c => 3, d => 4 }, shift; } sub get_a { $_[0]->{a} }; sub set_a { $_[0]->{a} = $_[1] }; } { package RefaddrCached; use Scalar::Util (); use base qw( Base ); my (%a, %b, %c, %d); sub new { my $self = bless \my $_tmp, shift; $$self = Scalar::Util::refaddr( $self ); $a{ $$self } = 1; $b{ $$self } = 2; $c{ $$self } = 3; $d{ $$self } = 4; return $self; } sub get_a { $a{ ${$_[0]} } }; sub set_a { $a{ ${$_[0]} } = $_[1] }; sub DESTROY { my $self = shift; delete $a{ $$self }; delete $b{ $$self }; delete $c{ $$self }; delete $d{ $$self }; } sub hidden_stuff { return [ \%a, \%b, \%c, \%d ] } } { package RefaddrCall; use Scalar::Util qw( refaddr ); use base qw( Base ); my (%a, %b, %c, %d); sub new { my $self = bless \my $_tmp, shift; $a{ refaddr $self } = 1; $b{ refaddr $self } = 2; $c{ refaddr $self } = 3; $d{ refaddr $self } = 4; return $self; } sub get_a { $a{ refaddr $_[0] } }; sub set_a { $a{ refaddr $_[0] } = $_[1] }; sub DESTROY { my $self = shift; delete $a{ refaddr $self }; delete $b{ refaddr $self }; delete $c{ refaddr $self }; delete $d{ refaddr $self }; } sub hidden_stuff { return [ \%a, \%b, \%c, \%d ] } } { package SelfAsIndex; use Scalar::Util qw( refaddr ); use base qw( Base ); my (%a, %b, %c, %d); sub new { my $self = bless \my $_tmp, shift; $a{ $self } = 1; $b{ $self } = 2; $c{ $self } = 3; $d{ $self } = 4; return $self; } sub get_a { $a{ $_[0] } }; sub set_a { $a{ $_[0] } = $_[1] }; sub DESTROY { my $self = shift; delete $a{ $self }; delete $b{ $self }; delete $c{ $self }; delete $d{ $self }; } sub hidden_stuff { return [ \%a, \%b, \%c, \%d ] } } { package NumSelfAsIndex; use Scalar::Util qw( refaddr ); use base qw( Base ); my (%a, %b, %c, %d); sub new { my $self = bless \my $_tmp, shift; $a{ 0+$self } = 1; $b{ 0+$self } = 2; $c{ 0+$self } = 3; $d{ 0+$self } = 4; return $self; } sub get_a { $a{ 0+$_[0] } }; sub set_a { $a{ 0+$_[0] } = $_[1] }; sub DESTROY { my $self = shift; delete $a{ 0+$self }; delete $b{ 0+$self }; delete $c{ 0+$self }; delete $d{ 0+$self }; } sub hidden_stuff { return [ \%a, \%b, \%c, \%d ] } } { package ClassStd; use base qw( Base ); use Class::Std; my %a : ATTR( :get<a> :set<a> ); my (%b, %c, %d) :ATTR; sub BUILD { my ($self, $id) = @_; $a{ $id } = 1; $b{ $id } = 2; $c{ $id } = 3; $d{ $id } = 4; } # just a guess - maybe more hidden behind the scenes sub hidden_stuff { return [ \%a, \%b, \%c, \%d ] } } use Devel::Symdump; my @classes = sort grep { eval { $_->isa( 'Base' ) } && $_ ne 'Base' } + Devel::Symdump->rnew->packages; my $n = 10000; my @objects; foreach my $class (@classes) { my ($size, $objects) = $class->size_of_n( $n ); # we keep the objects around just in case having heavily populated # attribute hashes affects the performance of the inside-out objec +ts push @objects, $objects; print "$class x $n\t= $size bytes\n"; }; print "\n"; use Benchmark qw /cmpthese/; cmpthese( -1, { map { my $class = $_; $class => sub { $class->do_something } } @classes});

Gives me:

BlessedHash x 10000 = 2265688 bytes ClassStd x 10000 = 2222948 bytes NumSelfAsIndex x 10000 = 2219534 bytes RefaddrCached x 10000 = 2300888 bytes RefaddrCall x 10000 = 2226816 bytes SelfAsIndex x 10000 = 2436816 bytes Rate SelfAsIndex ClassStd NumSelfAsIndex RefaddrCall +RefaddrCached BlessedHash SelfAsIndex 1000/s -- -9% -12% -44% + -57% -59% ClassStd 1100/s 10% -- -3% -38% + -53% -55% NumSelfAsIndex 1131/s 13% 3% -- -36% + -52% -54% RefaddrCall 1778/s 78% 62% 57% -- + -24% -27% RefaddrCached 2349/s 135% 114% 108% 32% + -- -4% BlessedHash 2443/s 144% 122% 116% 37% + 4% --

with a plain $self index coming in a lot worse than the faster alternatives.


Comment on Re^3: Threads and fork and CLONE, oh my!
Select or Download Code
Re^4: Threads and fork and CLONE, oh my!
by xdg (Monsignor) on Aug 17, 2005 at 15:40 UTC

    I sort of meant that tongue-in-cheek... but the benchmarks are neat to see. Actually I mean 'cheap' to use $self as opposed to some other "guaranteed" unique ID like a UUID (e.g. Data::UUID). I suspect that any of the $self as index variations will be faster than a UUID-as-index variation.

    I've also pondered a lighter-weight, pure-perl alternative like packing Time::HiRes::gettimeofday() and the memory address of an anoymous lexical during construction, as a memory address at a point in time should be unique on a single machine and "global" uniqueness isn't so much an issue for this kind of object ID.

    On the other hand, I'm personally moving away from the unique ID answer as CLONE works for those few who dare to muck with threads, and I don't think the performance hit of sharing objects across threads for those few people who might want it is going to be worth giving up the promiscuous nature of inside-out objects as a general property.

    (And for the next headache/magic-trick I'm considering with Object::LocalVars: trying out lexical closures to anonymous globrefs instead of package globals for storage to give local aliasing and encapsulation. And then see if I can get it running without too much of a performance hit against other options. Sign me up for The Perl Crackpot Index, I guess.)

    -xdg

    Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re^4: Threads and fork and CLONE, oh my!
by jdhedden (Deacon) on Sep 16, 2005 at 17:57 UTC
    The performance of the blessed hash case is dependent on the length of the keys used in the hash: The longer the key, the more time it takes!

    For one-character keys, blessed hashes are slightly faster than the cached refaddr case. (I got 2% when I did the timings.) However, one-character keys are rather unrealistic, and definitely not good programming practice.

    For two-character keys, the performance is the same.

    For three or more characters, cached refaddr is faster! I think five characters is realistic, and their performance is 2% slower. For ten characters, 7% slower!

    So if I were to call a winner, cached refaddr would be it.

    On another minor note, 0+$self yields the same result as the refaddr function. So you can eliminate 'use Scalar::Util', and just cache 0+$self.


    Remember: There's always one more bug.

      Cool. Hadn't realized that was the case about hash keys.

      I'm a little surprised at the refaddr versus 0+$self conclusion, though -- I would have thought that refaddr is just XS that returns a memory address, whereas 0+$self would wind up casting things to Perl scalars with associated overhead. I guess it's optimized away. Good to know.

      -xdg

      Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

        Even more efficient would be to write a custom hash loading routine in XS that a) extracts the refaddr automatically, and b) uses the binary representation of the address as the key for the hash lookup/storage. Currently when you use $hash{0+$self}=$foo you are doing itoa() on the address, then using the resulting string for the hash key. If the addresses binary representation was used as the key they keys would be fixed length, and would not require the itoa() step, and would be much faster. I imagine this would about 4-8 lines of XS code and would be drammatically faster than using $hash{0+$self}. Ie I can imagine an API like:

        ref_key_store(%hash,$ref,$value); my $val=ref_key_fetch(%hash,$ref);
        ---
        $world=~s/war/peace/g

      The performance of the blessed hash case is dependent on the length of the keys used in the hash: The longer the key, the more time it takes!

      You're right, but this isn't the reason that using $self is so much slower. Stringification of references is just slow:

      #! /usr/bin/perl use strict; use warnings; use Benchmark qw( cmpthese ); my $self = bless {}, 'SomeClass'; my $string = "$self"; my %a = ( $self => 0 ); my %b = ( $string => 0 ); cmpthese(-1, { self => sub { $a{ $self } = $a{ $self } + 1 }, string => sub { $b{ $string } = $b{ $string } + 1 }, }); __END__ # on my perl 5.8.7 Rate self string self 156393/s -- -83% string 927942/s 493% -- +
      On another minor note, 0+$self yields the same result as the refaddr function.

      Unless you overload arithmetic.

      On another minor note, 0+$self yields the same result as the refaddr function. So you can eliminate 'use Scalar::Util', and just cache 0+$self.

      Only when nummification is NOT overloaded. And in earlier perls you can't unoverload nummification.

      ---
      $world=~s/war/peace/g

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://484462]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (17)
As of 2014-09-23 12:59 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (221 votes), past polls