Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Re^3: Derangements iterator (callbacks)

by Anonymous Monk
on Jan 01, 2006 at 17:21 UTC ( #520263=note: print w/replies, xml ) Need Help??

in reply to Re^2: Derangements iterator (callbacks)
in thread Derangements iterator

Towards the best collection traversal interface
Most programming languages support collections, represented by an in-memory data structure, a file, a database, or a generating function. A programming language system gives us typically one of the two interfaces to systematically access elements of a collection. One traversal API is based on enumerators -- e.g., for-each, map, filter higher-order procedures -- of which the most general is fold. The second approach relies on streams, a.k.a. cursors, lazy lists. Generators such as the ones in Icon, Ruby and Python are a hybrid approach.

It is well-known that given a cursor interface to a collection, we can implement an enumerator. It is less appreciated that given an enumerator interface, we can always derive a cursor -- in an automatic way. We demonstrate that generic procedure for languages with and without first-class continuations.

Now that cursors and enumerators are inter-convertible, an implementor of a collection has a choice: which of the two interfaces to implement natively? We argue that he should offer the enumerator interface as the native one. The paper elaborates that enumerators are superior: in efficiency; in ease of programming; in more predictable resource utilization and avoidance of resource leaks. We present a design of the overall optimal collection traversal interface, which is based on a left-fold-like combinator with premature termination. The design has been implemented and tested in practice.

  • Comment on Re^3: Derangements iterator (callbacks)

Replies are listed 'Best First'.
Re^4: Derangements iterator (callbacks)
by Corion (Pope) on Jan 01, 2006 at 17:53 UTC

    Only having read the two abstracts, it seems to me that the authors rely on a Scheme-like language that has continuations or at least coroutines. With coroutines, it's trivial to convert between an "enumerator" (callback) and a "stream" (list/iterator). Without a coroutine mechanism, it's not as easy. As you seem to have read the paper, can you maybe post a link to the actual paper or, even better, give an application of the automatic way discussed there in Perl?

    Using the module, it's quite easy to have generators and to convert between enumerator and iterator, but Coro has the disadvantage of relying on a GPLed library and it doesn't (immediately) work on Win32. Without Coro, one has to manage the stack oneself and/or to create a large buffer for all values passed by the enumerator from what I know. But maybe the paper shows a technique I don't know (yet).

    Update: I had another look at Coro, and it isn't under the GPL. I also found that half-support for Win32/MSVC is there, now. Yay! ;)

      From enumerators to cursors: turning the left fold inside out...
      The mechanical inversion procedure presented in * had a catch: it relies on shift/reset (or call/cc plus a mutable cell, which is the same thing). How can we do such an inversion in Haskell? We can introduce a right fold enumerator, which is more amenable to such transformations. Or we can use a continuation monad and emulate shift/reset. The present article demonstrates the third approach: a non-recursive left-fold. We argue that such a left fold is the best interface for a collection. Indeed, given the non-recursive left-fold we can:
      • instantiate it into the ordinary left fold
      • instantiate in into a stream
      If we turn two enumerators into streams, we can *safely* interleave these streams.

      We should point out that the relation between the left fold, the non-recursive left fold and the stream is deep. The ordinary, recursive left fold is the fix point of the non-recursive one. On the other hand, the instantiation of the non-recursive left fold as a stream, as we shall see, effectively captures a continuation in a monadic action. We see once again that call/cc and Y are indeed two sides of the same coin **.

      The rest of the article demonstrates the inversion procedure. The procedure is generic, as evidenced by its polymorphic type. We illustrate the technique on an example of a file considered a collection of characters. Haskell provides a stream interface to that collection: hGetChar. We implement a left fold enumerator. We then turn that enumerator back to a stream: we implement a function 'myhgetchar' _only_ in terms of the left fold enumerator. The approach is general and uses no monadic heavy lifting.

        Well, seeing as apparently you understand this article maybe you could give us an example how to turn foreach (@list){ ... } into a cursor based approach. Since perl doesnt support first class continuations I guess you will need to implement this "left fold" operator. Which in itself would be pretty interesting. Actually even explaining in normal english (unlike the functional jargon gobbly-gook that the article uses) what this "left fold" operator does would be nice.


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://520263]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (4)
As of 2018-07-20 09:20 GMT
Find Nodes?
    Voting Booth?
    It has been suggested to rename Perl 6 in order to boost its marketing potential. Which name would you prefer?

    Results (427 votes). Check out past polls.