Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Progress Bar in Perl script

by slayedbylucifer (Scribe)
on Jul 08, 2012 at 09:48 UTC ( #980562=perlquestion: print w/ replies, xml ) Need Help??
slayedbylucifer has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

I have written a perl script which will write some output to a file. However, it takes about 2-3 minutes for the script to complete due to the amount of data involved. So, I wanted to show some progress bar so that user would know that something is going on and that the script is not stuck.

I checked on the CPAN and find there are lots of modules for this job. However, all of them need a predefined MAX counter as an input. Now, in my case I cannot define the MAX counter because it is not constant. The number of items the script will be working on with is not constant.

So is there a way to display progress bar in this case wherein I do not have to provide a MAX counter.

Thanks.

Comment on Progress Bar in Perl script
Re: Progress Bar in Perl script
by Athanasius (Monsignor) on Jul 08, 2012 at 10:06 UTC

    Since you don’t have a MAX counter, the progress you want to show is not a proportion of the total job, but just something that shows that ‘progress’ is being made. So, why would you need a CPAN module? Presumably, your output occurs in a loop, so:

    my $progress = 100; # adjust as needed my $count = 0; while (...) { ... show_progress() if (++$count % $progress == 0); } progress_complete(); # continue main code sub show_progress { print '*'; } sub progress_completed { print " completed\n"; }

    You don’t say whether the output is via console or GUI. The above is written for the console, but should be easily adaptable for a GUI if required.

    HTH,

    Athanasius <°(((><contra mundum

      Yes. that is correct. I just wanted to show something like a "spinning" cursor progress which will tell the user that script is doing its job. I do not want to show how much percentage is completed. This is meant for Linux console on a RHEL 6.2 box.

      Thanks Athanasius for your time.

        So you can use the Term::Spinner module. It can be easily done manually, too:
        $|=1; #autoflush sub spin { my @spinners = qw{/ - \ |}; my $index = $_[0]; print $spinners[$index]."\033[1D"; # man console_codes, ECMA-48 CSI se +quences, "CUB" return $index == 3 ? 0 : $index++; } my $j=0; for (something) { do_something; $j=&spin($j); }
        Sorry if my advice was wrong.
Re: Progress Bar in Perl script
by zentara (Archbishop) on Jul 08, 2012 at 10:54 UTC
    A spinner in a thread is your best bet.
    #!/usr/bin/perl use strict; use threads qw[ yield ]; use threads::shared; # by chanio of perlmonks # This alternative should print a line only if it succeeds # in what is doing as job. That way, there is a real # significance of the spinning wheel (sort of:). When the # job is not succeeding, the wheel should stay calm (panic!). my $ready : shared = 0; my $isOk : shared = 0; async { local $| = 1; while ( !$ready ) { do { select undef, undef, undef, 0.2; printf "\r ($_)" if ($isOk); } for qw[ / - \ | * ]; } print "\rReady"; $ready = 0; } ->detach; # do your work here for ( 1 .. 10 ) { ## Busy, busy, busy $isOk = 1; sleep 1; $isOk = 0; } $isOk = 0; $ready = 1; yield while $ready;

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh
      Hi, thanks for your time. But honestly, i did not understand the thread stuff. May be this is more intended for Developers.
Re: Progress Bar in Perl script
by frozenwithjoy (Curate) on Jul 08, 2012 at 17:42 UTC

    Your question reminded me of a quick little progress bar I wrote during Nick Patch's Unicode talk at YAPC::NA 2012. I had never messed with unicode before, but he inspired me with the U+1F4A9 on one of his slides. This won't help you the way it is written, but maybe it can inspire you!

    #!/usr/bin/env perl use strict; use warnings; use utf8; $| = 1; binmode STDOUT, ":utf8"; my $camel = "\N{U+1F42A}"; my $poop = "\N{U+1F4A9}"; print $camel; for (1..60) { sleep 1; print " $poop"; } print "\n";

    Also, the other inspiration for this was Perl Object Oriented Programming

      I saw something similar written for BASH on internet. Thank you very much for your suggestion. I will definitely work on it.
Re: Progress Bar in Perl script
by kcott (Abbot) on Jul 08, 2012 at 21:30 UTC

    You could take a look at Smart::Comments - this might do what you want (it doesn't require a MAX counter). Here's a quick commandline test you can run:

    $ perl -Mstrict -Mwarnings -e ' use Smart::Comments; for (1 .. 500) { ### Running... done qx{ls -l > /dev/null 2>&1}; } '

    Here's my take on a Roll-Your-Own spinner. This has 6 tests: the first 3 are expected to fail; the next three have fast, medium and slow spin rates. The spinner becomes an asterisk if the process completes successfully.

    #!/usr/bin/env perl use strict; use warnings; my $CMD = q{ls -l > /dev/null}; for ([], [q{}], [q{dummy}], [$CMD, 10], [$CMD], [$CMD, 50]) { eval { run_with_progress(@$_); }; warn $@ if $@; } sub run_with_progress { print q{-} x 60, qq{\nStarting ...\n}; my ($cmd, $skip) = @_; die q{Nothing to run!} unless $cmd; $skip ||= 25; # Arbitrary my @sails = map { "\b$_" } qw{- \ | /}; my $sail = 0; { local $| = 1; print q{Running: }; for (0 .. 1000) { system $cmd and die qq{'$cmd' failed! $? : $!}; print $sails[$sail++ % @sails] if not $_ % $skip; } } print qq{\b*\nFinished.\n}; }

    Output:

    $ pm_text_spinner_ryo.pl ------------------------------------------------------------ Starting ... Nothing to run! at ./pm_text_spinner_ryo.pl line 20. ------------------------------------------------------------ Starting ... Nothing to run! at ./pm_text_spinner_ryo.pl line 20. ------------------------------------------------------------ Starting ... Running: Can't exec "dummy": No such file or directory at ./pm_text_s +pinner_ryo.pl line 32. 'dummy' failed! -1 : No such file or directory at ./pm_text_spinner_ry +o.pl line 32. ------------------------------------------------------------ Starting ... Running: * Finished. ------------------------------------------------------------ Starting ... Running: * Finished. ------------------------------------------------------------ Starting ... Running: * Finished.

    Update: I revisited this and reworked run_with_progress() to use Smart::Comments but with a spinner instead of the normal progress bar. The messages are the same as shown above and the spin rates vary in the same manner; the formatting is different.

    sub run_with_progress { use Smart::Comments; ### Starting ... my ($cmd, $skip) = @_; die q{Nothing to run!} unless $cmd; $skip ||= 25; # Arbitrary my @sails = map { ($_) x $skip } qw{- \ | /}; my $sail = 0; for (0 .. 1000) { ### Running: $sails[$sail++ % @sails] system $cmd and die qq{'$cmd' failed! $? : $!}; } ### Finished. no Smart::Comments; return; }

    This code is a lot cleaner; however, I'm not a big fan of the output.

    Update 2: I also reworked run_with_progress() to use Term::Spinner. Messages, spin rates, etc. work the same as the other two; the output is almost identical to the Roll-Your-Own version. This would be my favourite ... at least until I find something better. :-)

    sub run_with_progress { print q{-} x 60, qq{\nStarting ...\n}; my ($cmd, $skip) = @_; die q{Nothing to run!} unless $cmd; $skip ||= 25; # Arbitrary use Term::Spinner; my $spinner = Term::Spinner::->new(); print q{Running: }; for (0 .. 1000) { system $cmd and die qq{'$cmd' failed! $? : $!}; $spinner->advance() if not $_ % $skip; } $spinner->finish(); print qq{\nFinished.\n}; undef $spinner; return; }

    -- Ken

      This is a great explanation. thank you for your time. i will work on the 3rd option that you mentioned.
      I've been away from Perl for too long... I hope I'm not misunderstanding some fundamental principal here...

      When I run this code from the command line, what I don't understand is how the "spinner" is printing over itself to the screen.
      I understand that @sail is being interated through by incrementing $sail, but what part of this code is making the contents of @sail print on top of each other, i.e. making the "spinner" effect?

      my @sails = map { "\b$_" } qw{- \ | /}; my $sail = 0; { local $| = 1; print q{Running: }; for (0 .. 1000) { system $cmd and die qq{'$cmd' failed! $? : $!}; print $sails[$sail++ % @sails] if not $_ % $skip; } }


      Any help here? What am I overlooking?

        Each of the sails is created by prepending a "\b" to the spinner character:

        my @sails = map { "\b$_" } qw{- \ | /};

        "\b" is a backspace character; so, when each sail is printed, it deletes the previous one. In the initial state, "Running:" is followed by two spaces (print q{Running:  };); the second space is overwritten by the first sail.

        "\b", and other special characters, are listed in perlop: Quote and Quote-like Operators.

        -- Ken

Re: Progress Bar in Perl script
by Jim (Curate) on Jul 10, 2012 at 00:16 UTC

    I usually do something simple like this:

    printf {*STDERR} "%d\r", $INPUT_LINE_NUMBER;

    …or…

    printf {*STDERR} "%d\r", ++$tally_of_something;

    This has the advantage of not only showing that the script is actively doing something, but of also giving the user an idea of the amount of progress even when the percentage of progress cannot be computed.

    Jim

Re: Progress Bar in Perl script
by Rhandom (Curate) on Jul 10, 2012 at 14:36 UTC
    If you are on a system that supports zenity - it is yet another option. It uses a real progress bar which may not be what you want (will only work on your local machine). Here is a very long one liner that makes the use of most zenity progress options (you could shorten it to zenity --progress --pulsate):

    perl -e 'open my $fh, "|-", q{zenity --title=Working --width=400 --pro +gress --text="Still going..." --percentage=50 --no-cancel --auto-kill + --auto-close}; select $fh; $|=1; select(undef,undef,undef,.1),print +$fh "$_\n#Now on $_...\n" for 1..100'

    Here is the code broken out into more of a program.

    use IO::Handle; # for older perls use Time::HiRes qw(usleep); my $pid = open my $fh, "|-", q{zenity --title=Working --width=400 --pr +ogress --percentage=50 --no-cancel --auto-kill --auto-close} or die "Could not open zenity: $!"; $fh->autoflush(1); for my $i (1 .. 100) { usleep 100_000; # microseconds print $fh "$i\n"; # move the bar print $fh "#Now on $i...\n"; # update the text print "We've done $i\n"; if ($i >= 50) { #kill 2 $pid; # not as portable print $fh "100\n"; # more portable with --auto-kill last; } }

    my @a=qw(random brilliant braindead); print $a[rand(@a)];
Re: Progress Bar in Perl script
by Anonymous Monk on Oct 07, 2014 at 20:25 UTC

    I think I am about to ramble on a bit.

    I would really be surprised if there was a "generic" progress reporting function in CPAN. creating tag files or write over a line is almost too simple to be worth a CPAN module. doing something operating system specific (a progress bar on a Windows screen for example) would be a different story. I think a "good" CPAN module would end up having so many options it would be way too big to be practical since it would probably at a minimum include tag file and console options, estimating routines (for completion estimates), memory usage reporting, all the options to change when reporting is done, and probably a variety of error checks along with a variety of ways to report the progress to reduce processing overhead.

    I have been working on a Perl based system where we normally provide feedback by creating "tag" files in the processing directory to show the progress (this is running as a background/batch process). the progress code does have the option to display to the console if running manually. based on the progress I have been providing the most important thing is to show that something is being done (processing record ####). additional information (processing record #### of ####) may be better in some cases where it is expected to take a long time. I report progress regularly (typically every 5 seconds) - that seems to be good enough for most cases. all of my common progress code is responsible for determining when to show the progress - the calling program just "reports" the progress.

    when I started to write the progress code I am currently using there was almost no progress reporting for anything that was going on. the code I am using now grew from a few simple routines to a whole collection of routines with special progress reporting options (percentages, time estimates, memory usage, etc.) so it is probably not a good example to be posted. if you write your own package it may start out as around 100-150 lines of code. if you do a lot of progress reporting then the code will probably grow based on what you report and how you want to report it. my progress reporting related code (in at least 3 files) totals around 1000 lines of code and modest comments.

    my recommendation - try something (anything) and then if that does not work to your satisfaction - try something else.

    a word of caution - if you are going to report progress for every operation then be careful about building strings for each progress report - can be a significant amount of processing. in my case I had alternate ways of reporting which did not have much overhead (the progress routine would build the variable string from a fixed base string and a record count if it was time to report). and while you may be able to test if it is time to report progress for each case where you report progress it is a whole lot easier to have a common package of some sort do it so all you really need to do is insert calls wherever you need to report progress.

    in the late 1990s I worked on a system (C/Assembly) using PC hardware I actually modified the VGA character set to add some characters for a "heartbeat" (characters shaped like a heart in various sizes), "spinner" characters (circles with different sections filled), and progress bar characters. once the setup was figured out (those old DOS books were handy at the time) it made an otherwise boring ASCII display a little more interesting. sometimes it is also handy to have something "clever" like this when talking to a customer about a problem ("is the heart beating"). it turns out that all of those little characters provided a lot of information with very little code (good things come in small packages or something like that).

    now an opinion. I feel that the progress that should be reported should be targeted to the end user of the program. the objective is to have the end user know that things are working (so they are happy) and have a piece of information (and possible failure indication) that can be used to help isolate where a problem is (so the developer is less unhappy when debugging). I do not think there are any real rules for providing progress to an end user except that if they wonder what is going on or complain that something is taking a long time then they need to be given some indication that things are OK. and as a developer I may want even more progress because I want to know that the software is actually working. just be careful about providing too much information.

    end of ramble - sorry to put you to sleep....

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://980562]
Approved by Corion
Front-paged by toolic
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (8)
As of 2014-12-19 02:16 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (70 votes), past polls