Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Learning to *really* love references

by astaines (Curate)
on Jul 29, 2002 at 23:44 UTC ( [id://186106]=perlmeditation: print w/replies, xml ) Need Help??

The following code used twice as much memory as necessary - why? And what would you (or did I) do about it...

process($data_ref,\@set_up,$template,*OUT); # Process sub process { my @data = @{ shift()}; my @set_up = @{ shift()}; my $template = shift; #For unpack local *FH = shift; foreach my $record (@data) { --do things--} }


The @data array was very big, and the subroutine above actually makes a copy of it. I didn't appreciate this when I first wrote it, but it was slow, and was consuming silly amounts of memory (even by Perl standards). Changing the relevant lines as follows makes it run faster, and it now uses less memory - about half as much.

my $data_ref = shift; #Big array foreach my $record (@{$data_ref}) { -- do things more efficiently -- }

I am now a happy perl loving bunny, and I can read in my very big files ;-)

--
Anthony Staines

PS - What's happening here, for monks whose Perl skills are closer to my own, is that I passed the array as a reference ($data_ref) to the subroutine. There, partly from force of habit I was turning it back into an array. This, as was obvious with hindsight, doubled the memory requirements of the program. Now @data was big, 10 MB to 50MB or so, and the program was not working well. Accessing the array, through the reference only, fixed the problem - hence the @{$data_ref} bit.

Replies are listed 'Best First'.
Re: Learning to *really* love references
by vladb (Vicar) on Jul 30, 2002 at 02:26 UTC
    Well, I guess I had that in me from years spent coding C/C++, but alwasy pass references to data instead of replicating it. Undoubtfully this is a rather trivial skill to grasp and anyone who'd spent at least 6 month coding would *I hope* know that.

    Perl is actually quite good when it comes to references. Certainly much easier to handle than C pointers for example. Take a look at this snippet:
    my ($a, $b, $c) = qw/asdf foo bar/; s/d/D/g for ($a, $b, $c); print "$a;$b;$c\n";
    Perl is farily well optimized and in this for loop no memory gets duplicated. Instead, the s/// operator works on data stored in the original place (allocated for the a,b,c variables).

    There are numerous other examples one could bring up. Not the least is the ease with which you can pass references around in perl. In fact, I believe this is one of the first basic things that any novice Perl book would mention to its reader. For those who don't have a book however, I suggest refering to this excellent 'article' on Perl references here (written by fair brother dominus.

    _____________________
    # Under Construction
Re: Learning to *really* love references
by japhy (Canon) on Jul 30, 2002 at 03:49 UTC
    Perl really needs better aliasing features. Currently, you can only create an alias with a package variable (using a glob) or in a for-loop. Other methods require you to use tie() or some other less elegant means.

    _____________________________________________________
    Jeff[japhy]Pinyan: Perl, regex, and perl hacker, who'd like a job (NYC-area)
    s++=END;++y(;-P)}y js++=;shajsj<++y(p-q)}?print:??;

      While afaik you cant alias scalar directly you can do aliasing with arrays and hashes using Array::RefElem and even do aliasing with arrays without anything more than the following utility sub:
      sub alias_array { \@_ }
      But you are right, perl needs better ways to define and manipulate aliases.

      Yves / DeMerphq
      ---
      Writing a good benchmark isnt as easy as it might look.

      Monks~

      Well, it might not do much now, but Perl6 will support aliasing via the := operator...

      That doesn't help you in the short run, but after reading the apocalypses (especially 4) I have been waiting with bated breath for Perl6.

      Boots
      ---
      Computer science is merely the post-Turing decline of formal systems theory.
      --???
      you can only create an alias with a package variable (using a glob) or in a for-loop.

      Not quite true, Japhy. In a subroutine call, the elements of @_ are aliased to the actual arguments.

      I find that that and the for-loop technique answer about 98% of my aliasing needs.

Re: Learning to *really* love references
by LanceDeeply (Chaplain) on Jul 30, 2002 at 15:17 UTC
    My question is:

    What's the commonly accepted syntax to do this? I've never used:
    foreach my $record (@{$array_ref}) {

    The notation I'm comfortable w/ is:
    foreach my $record ( @$array_ref ) {

    Is there an accepted standard?


    astaines:
    since you said you're still learning about refferences - you might want to try deref'ing them like so:
    my $arrayElement = $$arrayRef[0]; my $hashValue = $$hashRef{someKey};

    Also, you may want to checkout wantarray for returning array's or their references to the caller.

    Hope this helps
        my $arrayElement = $$arrayRef[0]; my $hashValue = $$hashRef{someKey};

      I prefer:

      my $array_element = $array_ref->[0]; my $hash_value = $hash_ref->{some_key};

      Using arrow notation makes it easier for me to recognize that the thingys in question are references (I find that the extra leading sigil gets lost in the code soup to my eyes). It also makes doing nested dereferences easier:

      my $foo = $AoH_ref->[0]->{some_key};

      although you'd be better off doing:

      my $hash_ref = $AoH_ref->[0]; my $foo = $hash_ref->{some_key};

      in most cases.

      --
      The hell with paco, vote for Erudil!
      :wq

      Well...

      Long time ago i used to write my code like this:
      my $arrayElement = $$arrayRef[0]; my $hashValue = $$hashRef{someKey};
      but when I tried to debug or simply read it a few months after coding, it took me a long time to make my brain think the same, wired way as when I wrote the code ;-)

      Since then I use
      my $arrayElement = ${$arrayRef}[0]; my $hashValue = %{$hashRef}{key};
      or
      my $arrayElement = $arrayRef->[0]; my $hashValue = $hashRef->{key};
      and it depends of the mood ;-)

      Anyway IMHO the $$something notation is the worst to read or understand again after some time... If you code alone it's up to you which one to choose, but if I'm a part of a team I just try to be polite, writing my code more readable and easy to understand to my co-workers...

      Greetz, Tom.

      The notation is what I always use - I find it easier to stick to one form throughout. Don't get me wrong I appreciate Perl's flexibility, but consistency makes my life simpler. I make no claim for superiority here! However, I do agree that the $$ notation is hard to follow, and I never use it


      --
      Anthony Staines

      PS -thanks for the tip about deref'ing - I'll try it out next time.

      Until you want to do fancy things. What if you have an object that contains a hash or array ref? Ie...
      $this = { _hashRef => {}, _arrayRef => [], };

      Then you are stuck doing....
      $x = ${$this->{'_hashRef'}}->{'blowMeDown'} @x = reverse @{$this->{'_arrayRef'}};

      And that's one to grow on.
Re: Learning to *really* love references
by tadman (Prior) on Jul 31, 2002 at 15:15 UTC
    That function isn't very graceful, especially with that repetitive shifting and data duplication. Here's how you could rewrite it:
    process($data_ref, \@set_up, $template, $fd_out); sub process { my ($data, $set_up, $template, $fd_out) = @_; foreach my $record (@$data) { # ... } }
    Maybe it's just me, but I find reading function arguments with shift is usually pointless since you can just declare the works in a single line. It's also nice to have the function argument declaration in a similar format to how you call it, so you can see if things match up.

    As of Perl 5.6 or thereabouts, you can open a filehandle that is put into a scalar (glob reference) using the open function:
    my $fd_out; open($fd_out, $what_file); while (<$fd_out>) { # ... }
    Or, of course, there is always IO::File:
    my $fd_out = IO::File->new($what_file, 'r'); while (<$fd_out>) { # ... }
    These filehandles are a lot easier to pass back and forth than the regular globs. If you're even thinking about using them as function arguments, don't use globs.

    As a note: ${$foo} and $$foo are equivalent. It's often simpler to just leave off the extra braces since they don't serve any practical purpose. There are occasions like ${$foo->{bar}} where they more appropriate.
Re: Learning to *really* love references
by TGI (Parson) on Jul 31, 2002 at 20:59 UTC

    You can also use sub prototypes to help make things pretty. They'll give you some argument checking capability, and will convert your arrays into refs for you. Here's a simple example:

    my @foo = qw(123 456 789); my @bar = qw(abc def ghi); sub qux(\@\@*) { print join "\t", @_, "\n"; my $foo = shift; my $bar = shift; print "foo: ", join "\t", @$foo, "\n"; print $FH "bar: ", join "\t", @$bar, "\n"; } qux(@foo, @bar, 'STDERR'); print "\n"; print \@foo . "\t" . \@bar;

    Applying this to your code, we get something like (notice how clean the subroutine call is):

    # the filehandle name must be quoted if you have strict subs enabled. process(@data,@set_up,$template, 'OUT'); # Process sub process (\@\@$*) { my $data_ref = shift(); my $set_up_ref = shift(); my $template = shift; #For unpack my $FH = shift; # The * gets us a handle reference foreach my $record (@data) { --do things--} }

    Check out perlsub's section on prototypes for more info.

    Update: Of course protoypes are a bit controversial...


    TGI says moo

Re: Learning to *really* love references
by ash (Monk) on Jul 31, 2002 at 14:42 UTC

    I prefer:
     
    For scalars:
    $$scalar
     
    For hashes:
    $hashref->{$key}
     
    For arrays:
    $arrayref->[$index]

    IMHO i.e
    $hashref->{key}{next}{next}{$_}
    is much better to reaad than:
    ${$hashref}{key}{next}{next}{$_}

    -- Ash/asksh <ask@unixmonks.net>

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://186106]
Approved by VSarkiss
Front-paged by Courage
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others meditating upon the Monastery: (2)
As of 2024-03-19 06:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found