Re^2: regarding 1.02 (was Re: IO::Lambda: call for participation)

[merlyn]

Now, you may think you just communicated something to me in:

First parallel is that the sequence matters. As in normal, blocking code, you would expect programming a HTTP request as

print $socket ...
readline $socket;

Here, the code is different:

writable {
   syswrite ...
readable {
   sysread ...
}}

but the sequence is the same! To emphasize this fact, there's no additional indentation for inner closures, to highlight that the execution is top-down, one way, linear.

But nothing clicked for me. Why are they nested? When does the closure passed to readable {} get executed? When does the code within the closure passed to writable get executed {}? And why?

This is the magic that is then fundamental to the rest of your description. Please explain it so we mere mortals can follow along. I can't understand anything further, because it seems to be based on something you think you are showing with this code.

In other words, you have writable, which takes a coderef. And readable, which takes a coderef. Somehow, the coderef passed to writable is built up by calling some other code and then the result of calling readable. Ooof. Too many layers. Head is hurting. What's the flow, and why?

And why are they nested!?!

[runrig]

I'm gonna take a swing at it, because I think I get it now, and someone will correct me if I'm wrong :-)

writable {...} creates an event for the IO::Lambda event loop to wait for the socket to be writable before it executes the code ref. But the actual call to writable doesn't block, if you put code after writable's closing brace, it would get executed almost immediately. One of the examples in the docs confused me at first, where there was an again;, with some code right after it. At first I was thinking that "again" was like a goto (of the next/retry sort), and the code after would never be executed, but then I realized that it was just setting up another event for IO::Lambda to wait for, and the actual call to again() would return immediately.

So when the socket is writable, you write to it in the code ref. But if you want IO::Lambda to wait for something else after you've written to the socket, it needs to be inside writable's code ref, after you've written. So that's when we call readable() to make IO::Lambda wait for the socket to be readable.

When does the closure passed to readable {} get executed? When does the code within the closure passed to writable get executed {}?

As soon as the enclosing lambda {...} thing is started, writable is executed, and the event (of waiting for $socket to be writable) is added to the event loop. The code ref is called when $socket is writable. When the code ref is called, readable is called, and the event loop will now wait for the socket to be readable (along with any other events that it's still waiting for), and when the $socket is readable, that code ref will be called.

So if writeable {..} followed readable {...} without the nesting, you'd be creating two distinct events at the same time, and it would call one code ref if the socket was readable, and the other if it was writable in whichever order the events happened.

[dk]

You are 100% correct. Thanks for the swing!

[dk]

All right, the key word "flow" has been uttered. Here's the flow:

1. lambda {X} has just been called. It returns an empty object, blessed hash of IO::Lambda with a single coderef X, which is not executed yet. This object has a method wait(), which is then called.

2. wait() figures out that lambda (object, that is) hasn't been started yet. It switches the lambda into "started" state, and invokes callback X. Then it enters event loop sleep/select cycle, until the lambda state gets "finished":

sub wait # simplified
{
    my $lambda = shift;
    $lambda-> start if $lambda-> is_passive;
    event_loop while not $lambda-> is_finished;
    return $lambda-> peek;
}

3. Coderef X creates a socket, calls a non-blocking TCP connect on it using IO::Socket::new, and passes that socket, along with a coderef Y to standard condition writable:

context $socket;
writable { Y }

and then exits. The lambda state is unchanged by calling writable, however it adds the socket into the listening queue of the underlying event loop (select, AnyEvent, POE, whatever), and asks the loop to do something when that socket gets writable. Meanwhile, wait still processes the event loop.

4. TCP syn-synack-ack succeeded, event loop says the socket is writable, and calls the lambda that waits for the socket. Lambda figures out that the socket waiting conditions should result in calling the coderef Y, which it immediately does.

5. Coderef Y does two things: it prints to the socket, and registers another coderef Z, that will be called when the socket gets readable:

syswrite $socket ....;
readable { Z }

Note, that these callbacks are one-shot only. The subscription to the writable events is not renewed, so the socket is not listened for write events anymore.

6. Socket gets readable, Z gets executed. It reads some bytes, but wants more, so it re-subscribes on the same socket, with the same coderef Z, to be woken up when there are more data. This is done with calling again. The latter knows the last coderef used, and spares the programmer from calling readable with the same coderef.

7. Socket gets readable, Z gets executed again. It reads zero bytes, figures it's an EOF, it says "return $buf". Before that, result of the last executed coderef was accumulated inside the lambda object. Now, $buf overwrites that and is stores its own $buf, which is then accessible through method peek.

8. Lambda, having called coderef Z, finds out that it has no more subscriptions, so it switches itself to "finished" state.

9. wait sees that the lambda is finished, it stops the cycle and and returns whatever was left on lambda, in our case $buf.

I almost feel how it gets better :) Please tell me what do you think.