http://www.perlmonks.org?node_id=1110800

BroFish has asked for the wisdom of the Perl Monks concerning the following question:

Hello all.

I am building a web app that will run compliance checks on client-submitted files placed in a special directory. These checks can run for more than a minute, and I want the output from those checks to be displayed in the web browser as they are generated...not waiting to the end of the run. The non-blocking Mojolicious server and WebSockets seem like a good solution technique.

In the following code, &myapp() simulates the long running checks, occasionally producing output. The good news is the desired output appears in the web browser, that output being "Connection open. Entering log4. Entering myapp. 0... 1... 2... 3... Leaving myapp. Leaving log4." The problem is the web browser sits idle for 12 seconds, and then the output displays all at once. Instead, I would like the output to appear in the web browser as soon as $tx->send() is called. How can that be accomplished with Mojolicious?

Tcpdump shows the web server is sending the output all at once, not as $tx->send() is called.

Here is the complete, running code (in one file):

#!/usr/bin/perl ## run as: ./wbsckt.pl daemon use Mojolicious::Lite; ## Automatically enables "strict", "warnings", "utf8" and Perl 5.10 fe +atures; &startup; ##------------------------------------------------------------------- sub myapp{ my $tx=shift; $tx->send("Entering myapp."); foreach my $xx (0..3) { $tx->send("$xx..."); sleep(3); } $tx->send("Leaving myapp."); return "You should never see this."; } ##------------------------------------------------------------------- sub startup{ get '/' => sub { my $c=shift; $c->render('index'); }; get '/log2' => sub { my $c=shift; $c->render('log2'); }; websocket '/log3' => \&ws_log4; app->secrets(['password' => '8675309J']); app->start; } ##------------------------------------------------------------------- sub ws_log4{ my $self= shift; my $tx= $self->tx; my $ip= $tx->remote_address; app->log->debug("Client '$ip' connected"); $tx->send("Entering log4."); $self->on(&myapp($tx) => sub { my($ws,$msg)= @_; $ws->inactivity_timeout(50); $ws->send("Time is: " . scalar(localtime())); ## odd, we neve +r see this output }); $self->on(finish => sub { my($c,$code,$reason)= @_; $c->app->log->debug("WebSocket closed with status $code."); }); $tx->send("Leaving log4."); } ##------------------------------------------------------------------- __DATA__ @@ index.html.ep <!DOCTYPE html> <html> <head><title>Static Page</title> </head> <body> <h1>Index Page</h1> <p>This is a static page. For WebSockets example, click <a href="/log2">here</a>. </p> </body> </html> @@ log2.html.ep <!DOCTYPE html> <html> <head> <title>WebSockets Example</title> <script src="//ajax.googleapis.com/ajax/libs/jquery/1.10.1/jquery.min. +js"></script> </head> <body> <p id="result"></p> %= javascript begin var ws = new WebSocket('<%= url_for('log3')->to_abs %>'); ws.onopen = function() { $('#result').text("Connection open."); //ws.send("Hi."); }; ws.onmessage = function (e) { $('#result').append( "\n" + e.data); }; % end </body> </html>

This code is running on CentOS 6.5, Perl 5.10.1 with these perl modules: EV 4.18; IO::Socket::Sockets 0.65; IO::Socket::SSL 2.007.

Thank you very much for your insight and patience.

Replies are listed 'Best First'.
Re: real-time output from Mojolicious WebSockets?
by Anonymous Monk on Dec 19, 2014 at 08:33 UTC

    If you try sleep(1) for 1..3; does it change anything?

    If you switch webservers (hypnotoad?) does it change anything?

    not having played with websocket thats the first things I would check :)

      Thank you for your suggestions.

      Changing "sleep(3);" to "sleep(1) for 1..3;" had no obvious effect. I also tried running the web server via "morbo ./wbsckt.pl" and "hypnotoad --foreground ./wbsckt.pl", and the behavior did not change. In each case tcpdump shows the server is sending back all output at once.

Re: real-time output from Mojolicious WebSockets?
by Anonymous Monk on Dec 19, 2014 at 13:34 UTC
    Well I don't know any Mojolicious but
    $self->on(&myapp($tx) => sub { ...
    Wouldn't that be evaluated as
    $self->on('You should never see this.' => sub { ...
    after &myapp($tx) returns and before registering the handler?

    Also, please tell why you're using ampersands before function names.

    $ws->send("Time is: " . scalar(localtime())); ## odd, we never see this output
    Maybe because Mojolicious takes 'You should never see this.' seriously? :)

    Or maybe I'm wrong and Mojo is just too magical for people like me...

      Why ampersands you ask? Because I started programming in perl late last century :), and that's how you called procedural functions back then. Today I call OO methods via $obj-<method(), but I still call procedural functions the old fashioned way.

      "As iron sharpens iron, so one man sharpens his friend." So I'm open being brought into the 21st century. :) Perhaps I should read "Modern Perl"?

      To your point, and the point of the other anonmonk, I am slightly perplexed on how to call my own web service. I did try:
      $self->on(json => sub...
      but the affects were still the same in that output from the anonymous procedure called, or methods called from inside that anonymous procedure, etc. was being saved until the procedure ended, not being delivered as it was produced. (I stripped down my code to produce this simplified example. I would be happy to repost using "on(json" if you'd like.)

        ... but the affects were still the same in ...

        :) in that one subroutine the comments says odd, we never see this output but Mojo doesn't have a "You should never see this." event, so it makes sense that sub is never executed and output is never seen ... mojo will never call it

        OTOH why the buffering happens, is odd ... I'll take a closer look later

        OTOH, standard way of doing this, instead of calling a sub that sleeps, Proc::Background a process that sleeps, basic technique from Watching long processes through CGI (Aug 02)

        Yup, somebody is definitely buffering the messages (mojo)

        Connection open. 12/19/2014, 5:08:52 PM : Entering log4. 12/19/2014, 5:08:52 PM : Leaving log4. 12/19/2014, 5:09:06 PM : Entering myapp. 12/19/2014, 5:09:06 PM : 0...Fri Dec 19 17:08:54 2014 12/19/2014, 5:09:06 PM : 1...Fri Dec 19 17:08:57 2014 12/19/2014, 5:09:06 PM : 2...Fri Dec 19 17:09:00 2014 12/19/2014, 5:09:06 PM : 3...Fri Dec 19 17:09:03 2014 12/19/2014, 5:09:06 PM : Leaving myapp.

        Forking the job in the background and doing some IPC ought to work , but may not be satisfying :)

        My version of your code with Re: indenter / prettyprinter / beautifier / perltidy for mojolicio.us Mojo::Template and perltidy -olq  -csc -csci=3 -cscl="sub : BEGIN END if while " -otr -opr -ce -nibc -i=4 -pt=0   "-nsak=*"

Re: real-time output from Mojolicious WebSockets?
by DaveARoberts (Acolyte) on Sep 13, 2016 at 21:13 UTC

    I've being looking for a working example using websockets to provide a stream of output based on a server back end process - one that did not save all the messages until the end of the process. Found one such here which worked for me.

    https://nnc3.com/mags/LJ_1994-2014/LJ/207/10995.html