Re^7: Getting for() to accept a tied array in one statement

[ikegami]

Why do you want to make something so simple so complicated and expensive? Use an iterator!

sub make_list_iterator {
   my @list = @_;
   return sub {
      return () if !@list;
      return shift(@list);
   };
}

my $iter = make_list_iter("some", "el", "ems");
while ( my ($item) = $iter->() ) {
   ...
}

As a bonus, one doesn't need to know the length of the list in advance with an iterator!

[Veltro]

I believe (or at least this is how I understand it) that the point of the exercise is SoC. The 'wrapper' causes a separate function to execute every time an element inside of the array is accessed and this function is defined somewhere else (not inside the while loop). Your code could work in case it would have used a reference array instead. But I think you mean with 'expensive' you mean using the tie?

Anyways, if you would use an array reference your code does work:

use lib '.' ;
use strict ;
use warnings ;
use ar ;
my @ar ;
my $ar2 = tie @ar, "ar" ;
@ar = (1, 2, 3) ;

sub make_list_iterator {
   my $list = $_[0] ;
   return sub {
      return () if !@{$list};
      return shift(@{$list});
   };
}

my $iter = make_list_iterator(\@ar) ;
while ( my ($item) = $iter->() ) {
   print "$item\n" ;
}

I'm not showing how to implement ar, plenty of examples in previous posts

[ikegami]

Why????? The code I posted already worked as-is. No need to create a tied array and the underlying class. My code already calls a sub for each element.

[Veltro]

I'm not saying that your method is incorrect. I am just looking at a different form of abstraction. In the case of using the Iterator, where does it resides? I assume that is in the collection class. Then the Algorithm and the Iterator are combined into one. But then I think, this is not right. The sole purpose of an Iterator should be traversing (up, down) through the elements of the collection. Then algorithms can be defined that take iterators as an input arguments. Also I feel that Iterators should be objects themselves and contain begin and end methods (maybe even overloading ++ and --)

CollectionX
   ->begin (Iterator)

CollectionY
   ->begin (Iterator)

CollectionZ
   ->begin (Iterator)
   ->end (Iterator)

ProgressBar

Flowers

Algorithms
   - showProgress - Bound to ProgressBar
   - growFlowers - Bound to Flowers

code:
   ...
   use CollectionX ;
   use CollectionY ;
   use Algorithms qw( showProgress, growFlowers ) ;
   ...
   my colX = new CollectionX ;
   colX->... ;
   while( showProgress(CollectionX->begin) ) {
   }
   my colY = CollectionY ;
   colY->... ;
   while( showProgress(colY->begin) ) {
   }
   my colY = CollectionY ;
   colY->... ;
   while( growFlowers(colY->begin) ) {
   }

With this level of abstraction it is only needed to define iterators for each type of collection, and only one algorithm for each added functionality. Otherwise I would need to create an iterator for each added functionality for each class (in this example 6 iterators instead of 3) and repeat code 3 + 3 times. I'm not going to build examples on this, just too much work right now. But I think the method with the tie shown by hdb makes this possible and that is why I said like it so much. I hope this helped understanding where I am coming from.

[perlancar]

Yes, the goal is not finding out the best way to do a progress-bar API (I think doing the Progress::Any-style progress bar is still okay), but how to emulate the style of a Python library.

As a bonus, one doesn't need to know the length of the list in advance with an iterator!

With for(@tied_array) we also doesn't need to calculate the length of the list in advance. We can just retrieve an item one by one, and for() will still invoke FETCHSIZE on each iteration. Which incurs an extra cost, admittedly.

[ikegami]

but how to emulate the style of a Python library.

I presume Python's has native support for iterators. Perl 6 has lazy lists, but Perl 5's doesn't, and nothing's going to make for work with lazy lists short of overriding for/foreach. Tied arrays definitely can't achieve that.

There are a few modules that provide iterator-aware looping primitives. Search for "iterator" on CPAN.

With for(@tied_array) we also doesn't need to calculate the length of the list in advance

That's not true.

Whether it's called for every loop pass or not, it's still called before the first pass, so you still need to know the size up front. And FETCHSIZE must always return a correct value because it's called once in other situations.

In fact, I consider it a bug that FETCHSIZE is called more than once in that situation.

Ironically, if you circumvent the optimisation of for (@a) by using for ((), @a), you actually get something faster for tied arrays since the latter only calls FETCHSIZE once.

BEGIN {
    package My::TiedArray;

    use strict;
    use warnings;

    use Tie::Array qw( );

    our @ISA = 'Tie::StdArray';

    sub TIEARRAY {
        my $class = shift;
        bless [@_], $class
    }

    sub FETCHSIZE {
        my $self = shift;
        print("FETCHSIZE\n");
        return $self->SUPER::FETCHSIZE(@_);
    }

    $INC{"My/TiedArray.pm"} = 1;
}

use strict;
use warnings;
use feature qw( say );

use My::TiedArray qw( );

tie my @a, My::TiedArray::, qw( a b c );

say "Before loop";
for (@a) {
   say "In Loop";
}

say "";

say "Before loop";
for ((), @a) {
   say "In Loop";
}

Before loop
FETCHSIZE
In Loop
FETCHSIZE
In Loop
FETCHSIZE
In Loop
FETCHSIZE

Before loop
FETCHSIZE
In Loop
In Loop
In Loop

[perlancar]

But isn't the fact that FETCHSIZE is called "more than once" (a.k.a. once per iteration) is how we achieve "don't have to calculate the length of the list in advance?". The first call to FETCHSIZE, we fetch from the iterator. If there is no item, we return 0, ending the for(). If there is an item, we return 1 and push() the fetched item in a buffer, and for() continue. In the next iteration, FETCHSIZE is called again. We fetch from the iterator. If there is no item, we still return 1, ending the for(). If there is still an item, we return 1+1=2 and push() the item in a buffer.

FETCH will shift() item from buffer.

Where do we calculate the length of the list in advance?

Range::ArrayIter is a proof of concept of range iterator using tied array. You can use range_iter(1, Inf), for example. Yes, it's slower than object-based and coderef-based iterator.