Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Array of arrays - redux

by JockoHelios (Scribe)
on Jun 17, 2013 at 15:50 UTC ( #1039401=perlquestion: print w/ replies, xml ) Need Help??
JockoHelios has asked for the wisdom of the Perl Monks concerning the following question:

I'd posted a problem here and gotten a number of good responses. But since I'm still spinning my wheels, I'll state here a description of the problem I'm trying to solve, which I didn't do in the original post. I thought the added detail would just cloud the issue, but since I'm still stuck here goes.

The original post, with demo code and my confusion both in full display, are here http://www.perlmonks.org/?node_id=1039251

I have a large array of arrays, the original data. When going through that data, serially, at each row I need to check back over the previous 15 rows to see if a value exists in a specified field of those previous rows. Call it a look-back. The value I'm looking for can change based on the values in the current row, so this has to be evaulated with every original data row.

I decided to populate a look-back array with rows from the original data. This would be the 15 previous rows. The flow concept goes like this:

run through the original data via foreach. For every original data row :
while there are less than 15 rows in the look-back array, push each original data row onto the look-back array
after there are 15 rows in the look-back array, scan the look-back array for the specified value,
then shift the first row off of the look-back array and push the current original data row onto the look-back array
repeat until the end of the original data

The look-back array would thus always have the previous 15 rows, based on the current original data row.

Where I'm stuck is at the look-back array. I'm unable to come up with the code to pull out a specified field from each row of the look-back array (of arrays). I've tried several methods to get the original data rows into the look-back array, and more methods to pull the specified element from it, but all I've gotten are error messages. Am I pounding my head against the wall here ? Is this flow concept so far off the mark that it can't be done ?

I had considered setting up a loop with a counter to go through the original data, and using that counter to look back over the last 15 rows. That seems clumsy to me, but maybe in this case ! TMTOWTDI...

Dyslexics Untie !!!

Comment on Array of arrays - redux
Re: Array of arrays - redux
by hdb (Parson) on Jun 17, 2013 at 16:03 UTC

    An illustration to see whether I understand your problem: I create an array of array refs with 5 lines (instead of your 15 to keep it small) and 11 columns (0 to 10) called @lookback. Then I extract column number 3 (the fourth column) from this AoA, result called @column. Is that the right direction?

    use strict; use warnings; use Data::Dumper; my @lookback; for my $line (1..5) { push @lookback, [ map { "$line.$_" } 0..10 ]; } print Dumper \@lookback; my $col = 3; my @column = map { $_->[$col] } @lookback; print Dumper \@column;
      It's close, but I'm copying the rows into the look-back array. I don't see why it should make a difference, but it does.

      I can read the any selected field from the original array of arrays, but when I copy the rows into the look-back array I can't access any field in those rows.

      Should I be copyng references to those rows instead of copying the rows themselves ?
      Dyslexics Untie !!!
Re: Array of arrays - redux
by BrowserUk (Pope) on Jun 17, 2013 at 16:03 UTC

    Is this close to what you're describing?

    #! perl -slw use strict; my @data; $data[ $_ ] = [ map int( rand 100 ), 1 .. 10 ] for 0 .. 1e3; my @lookback; my $matches = 0; for my $row ( @data ) { for my $lb ( @lookback ) { if( $row->[ 3 ] < 10 && $lb->[ 7 ] > 50 ) { print "condition matched ", ++$matches; last; } } push @lookback, $row; shift @lookback if @lookback > 15; }

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      It might be, but my Perl knowledge isn't up to the level at which I could understand that code :)
      Dyslexics Untie !!!

        It is not complicated code. Take it one line at a time and study it.

        Play with it. Add print statements to see what is going on. Ask questions -- in this thread -- about things you do not understand.

        With a few hours or a coupe of days you should be able to make sense of it. And that time will stand you in good stead for the rest of your Perl experience.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Array of arrays - redux
by space_monk (Chaplain) on Jun 17, 2013 at 16:07 UTC

    Why do I have a feeling that the more I read about your problems, the more you should tell us the entire problem you are trying to solve instead of just a small portion of it, and how you have coded that small portion? i.e. why do you have an array of arrays, why do you need to look back over 15 rows, why does the data you are looking for change etc.


    If you spot any bugs in my solutions, it's because I've deliberately left them in as an exercise for the reader! :-)
      The small portion of it is the problem - that's where I'm getting stuck with error message. The rest of the script is working.

      The reason I have to look back over the previous 15 rows is not relevant to the array problem. It's a requirement of the processing, nothing more.

      The data value that I'm looking for changes because it's based on the several values in the current row of original data (which has 17 fields). Deriving that value is not an issue, the code produces the correct value to search for.
      Dyslexics Untie !!!
Re: Array of arrays - redux
by poj (Curate) on Jun 17, 2013 at 16:40 UTC
    I had considered setting up a loop with a counter to go through the original data, and using that counter to look back over the last 15 rows

    That seems the simplest way to me but with 2 counters say $ix,$n. [$ix] for current position and $n counting from 1 to 15 pointing to previous row [$ix-$n] in turn. You will need to adjust the logic for the first 15 records of course.

    poj
      I'm reluctantly coming to the conclusion that I'll have to abandon the look-back array concept. Apparently, two-dimensional arrays in Perl are too difficult to actually use for someone at my level :(

      I'll need at least three counters, though. The original array has data for up to a full year, and I have to start the comparison search over with each new day. So I'd need an overall original array counter, and the other two counters would reset when the day turns over.

      Dyslexics Untie !!!
Re: Array of arrays - redux
by Anonymous Monk on Jun 17, 2013 at 18:17 UTC
    "Look back" across a slice() of the existing array, using "-15" to get at the last up-to-15 entries.
Re: Array of arrays - redux
by space_monk (Chaplain) on Jun 18, 2013 at 11:26 UTC
    If you spot any bugs in my solutions, it's because I've deliberately left them in as an exercise for the reader! :-)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1039401]
Approved by hdb
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (4)
As of 2014-08-02 05:27 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Who would be the most fun to work for?















    Results (54 votes), past polls