Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Re^4: Handling HUGE amounts of data

by Dandello (Scribe)
on Jan 31, 2011 at 19:45 UTC ( #885341=note: print w/ replies, xml ) Need Help??


in reply to Re^3: Handling HUGE amounts of data
in thread Handling HUGE amounts of data

Well, it still throws an 'out of memory' when I close the little Perl/Tk that announces the script has finished running.

I assume I've done this right as BrowserUk suggested using Tie::Array::Packed to save on RAM:

tie @aob, 'Tie::Array::Packed::DoubleNative'; #model 1; sub popnum1 { ( $x, $y, $z ) = @_; if ( $y == 0 ) { $aob[$x][0] = $initial + $z; $zaza = $aob[$x][0]; } else { if ( substr( $aod[ $y - 1 ], $x, 1 ) ne 'a' ) { $aob[$x][$y] = $initial + $z; $zaza = $aob[$x][$y]; } else { $aob[$x][$y] = $z + $aob[$x][ $y - 1 ]; $zaza = $aob[$x][$y]; } } return $zaza; }

I figure that returning a single variable ($zaza)is more efficient than returning $aob[$x][$y] - it's hard to tell.


Comment on Re^4: Handling HUGE amounts of data
Select or Download Code
Replies are listed 'Best First'.
Re^5: Handling HUGE amounts of data
by BrowserUk (Pope) on Jan 31, 2011 at 20:07 UTC
    I figure that returning a single variable ($zaza)is more efficient than returning $aob$x$y

    Returning $aob[$x][$y], is returning a single variable. Whether you derefence the arrays here:

    $zaza = $aob[$x][$y];

    Or here:

    return $aob[$x][$y];

    Makes no difference.

    However, using my for ( $x, $y, $z ) & $zaza would make some difference as lexicals are more efficient than globals. Plus you could then benefit from use strict.

    But your subroutine can be refactored as:

    sub popnum1 { my( $x, $y, $z ) = @_; if ( $y == 0 ) { return $aob[$x][0] = $initial + $z; } else { if ( substr( $aod[ $y - 1 ], $x, 1 ) ne 'a' ) { return $aob[$x][$y] = $initial + $z; } else { return $aob[$x][$y] = $z + $aob[$x][ $y - 1 ]; } } }

    which saves a temporary variable and two, double dereferences.

    Personally, I think I'd code that as:

    sub popnum1 { my( $x, $y, $z ) = @_; return $aob[ $x ][ $y ] = $y && substr( $aod[ $y - 1 ], $x, 1 ) ne 'a' ? $initial + $z : $z + $aob[$x][ $y - 1 ]; }

    Though I'd want to verify that my logic transformation was correct. That should be appreciably more efficient than your original above.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      Actually I am using strict - the my declaration for $x, $y, and $z was elsewhere. And this was my original before I got creative:

      sub popnum1 { ( $x, $y, $z ) = @_; if ( $y == 0 ) { $aob[$x][0] = $initial + $z; } else { if ( substr( $aod[ $y - 1 ], $x, 1 ) ne 'a' ) { $aob[$x][$y] = $initial + $z; } else { $aob[$x][$y] = $z + $aob[$x][ $y - 1 ]; } } return $aob[$x][$y]; }

      I've moved the Perl/Tk done snippet into a different script and running the full load still throws an 'out of memory' during what should be cleanup. Now I have to track down exactly where.

        Actually I am using strict - the my declaration for $x, $y, and $z was elsewhere.

        Well that's just silly.

        Now I have to track down exactly where.

        If you scoped your variables properly, it might make that task easier.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://885341]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (19)
As of 2015-07-28 17:06 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (258 votes), past polls