Re^4: Problem with ReadKey

[liverpole]

Hi jimhenry,

I agree with jethro that the best way to do this is to read the list of files in the program.  You shouldn't have to specify STDIN in your Term::ReadKey calls, either.  Also, consider putting "$| = 1" at the beginning of the program to flush STDOUT.

Here's a small subroutine which will let you avoid using File::Find and just get the files ending in ".txt":

sub find_all_textfiles {
    my ($dir) = @_;
    my $fh = new IO::File;
    opendir($fh, $dir) or die "Failed to read dir '$dir' ($!)\n";
    my @files = readdir($fh);
    closedir $fh;

    my @found = ( );
    foreach my $fname (@files) {
        next if $fname eq '.' or $fname eq '..';
        my $path = "$dir/$fname";
        if (-f $path and $path =~ /[.]txt$/) {
            push @found, $path;
        } elsif (-d $path) {
            push @found, find_all_textfiles($path);
        }
    }

    return @found;
}

Of course this will require that you read each file individually, and somehow store their contents.

If it matters which file the text came from, you could save the lines from each file in a separate array reference, and then return a reference to those array references (note my use of Data::Dumper, which is invaluable for visualizing your data!):

use strict;
use warnings;
use Data::Dumper;
use IO::File;
use Term::ReadKey;
use Time::HiRes qw(time);

$| = 1;
my @files  = find_all_textfiles(".");
my $a_text = read_files(@files);
printf "Results of reading [@files] => %s\n", Dumper($a_text);

sub read_files {
    my (@files) = @_;
    my $a_text = [ ];
    foreach my $fname (@files) {
        print "Reading '$fname' ...\n";
        my $fh = new IO::File($fname) or die "Can't read '$fname' ($!)
+\n";
        chomp(my @text = <$fh>);
        push @$a_text, [ @text ];
    }
    return $a_text;
}

If you don't care which text came from which file, just throw it all in one big array:

use strict;
use warnings;
use Data::Dumper;
use IO::File;
use Term::ReadKey;
use Time::HiRes qw(time);

$| = 1;
my @files  = find_all_textfiles(".");
my @text   = read_files(@files);
printf "Results of reading [@files] => %s\n", Dumper([ @text ]);

sub read_files {
    my (@files) = @_;
    my @text = ( );
    foreach my $fname (@files) {
        print "Reading '$fname' ...\n";
        my $fh = new IO::File($fname) or die "Can't read '$fname' ($!)
+\n";
        chomp(my @lines = <$fh>);
        push @text, @lines;
    }
    return @text;
}

In fact, in the last example, you could even combine the finding of files with the saving of each file's text, to create a single subroutine.  I'm using a nifty trick here, which is to pass the array reference $a_text which is the aggregate of all text in each recursive call to read_textfile_lines; only at the top level is it undefined (but in that case, you initialize the array ref with:  $a_text ||= [ ];):

use strict;
use warnings;
use Data::Dumper;
use IO::File;
use Term::ReadKey;
use Time::HiRes qw(time);

$| = 1;
my $a_text = read_textfile_lines(".");
printf "Text from ALL textfiles in current dir:  %s\n", Dumper($a_text
+);


sub read_textfile_lines {
    my ($dir, $a_text) = @_;
    $a_text ||= [ ];

    my $fh = new IO::File;
    opendir($fh, $dir) or die "Failed to read dir '$dir' ($!)\n";
    my @files = readdir($fh);
    closedir $fh;

    foreach my $fname (@files) {
        next if $fname eq '.' or $fname eq '..';
        my $path = "$dir/$fname";
        if (-f $path and $path =~ /[.]txt$/) {
            $fh = new IO::File($fname) or die "Can't read '$fname' ($!
+)\n";
            chomp(my @lines = <$fh>);
            push @$a_text, @lines;
            close $fh;
        } elsif (-d $path) {
            read_textfile_lines($path, $a_text);
        }
    }

    return $a_text;
}

Now you're ready to call some subtroutine process_text (or whatever), passing the ref to the array of all text $a_text.  It will do something like:

sub process_text {
    my ($a_text) = @_;

    while (1) {
        my $key;
        my $wait_until = time + 3;
        while ( time < $wait_until ) {

        ReadMode 3;
        $key = ReadKey( -1 );
            if ( defined $key ) {
                print "keystroke $key\t";
            }
            ReadMode 0;
        }
        print_some_random_text($a_text);
    }
}


sub print_some_random_text {
    my ($a_text) = @_;

    # Replacing the next line with meaningful code is left as
    # an exercise for the OP.
    ##
    print "[Debug]\n";
}

As you see, I've left for you the fun of deciding what to print out in the subroutine print_some_random_text, as well as adding code for trapping relevant keys from the user in the subroutine process_text.  Good luck!

[jimhenry]

Thanks for the additional suggestions. I've implemented a recursive function with readdir to gather the list of filenames, but decided against loading the contents of the files up front. This script may have to deal with arbitrarily large filesystems with lots and lots of text files, on systems with arbitrarily small amounts of memory. So it reads the filenames up front, then randomly picks a subset of files to read and save a random subset of paragraphs from. I've got it pretty much working now. The slow_print function now looks like this:

sub slow_print {
    my $subscript = shift;
    my $para = $paras[ $subscript ];

    if ( $wrapping ) {
    $para = &rewrap( $para );
    }

    # include the filename/line number index in the paragraph instead
    # of printing it separately, so we don't have to duplicate the
    # sleeping/keypress-handling logic below.
    if ( $print_filenames or $debug ) {
    $para = $indices[ $subscript ] . $para;
    }

    my @lines = split /\n/, $para;
    foreach ( @lines ) {
    print $_ . "\n";

    # If the user presses a key before the pause time for
    # the current line has passed, we don't necessarily skip
    # to the next line with no further pause.
    my $start = time;
    my $remaining_wait =  $pause_len * length $_;
    while ( time < ( $start + $remaining_wait ) ) {
        my $key = ReadKey( $remaining_wait );
        if ( defined $key ) {
        &handle_keystroke( $key );
        }
        # the $pause_len might have been changed by user's keypress
        $remaining_wait = ($pause_len * length $_) - (time - $start);
    }
    }
    print "\n\n";
}

(I'm setting ReadMode 3 near the beginning of the script, and setting ReadMode 0 in an END{} block.) The directory read function, with its helper want_file, looks like this:

# comments refer to benchmark tests using ~/Documents/ and ~/etext/ di
+rs
sub want_file {
    my $filename = shift;
    if ( $check_type && -T $filename ) {
    # 15061 filenames in 0.692 sec
    return 1;
    } elsif ( $check_extensions ) {
    # 8857 filenames in . 0.794 sec with 
    # --extensions=txt,pl,html,htm 
    if ( ( grep { $filename =~ m(\.$_$) } @file_extensions ) && -e $fi
+lename) {
        return 1;
    }
    } else {
    # this test finds 5066 files in ~/Documents and ~/etext 
    #  in 0.218 sec
    if ( $filename =~ m(\.txt$) &&  -e $filename  ) {
        return 1;
    }
    }
    return 0;
}


sub recurse_dir {
    my $dirname = shift;

    local *D;
    opendir D, $dirname;

    my $fname;
    while ( $fname = readdir D ) {
    my $name = $dirname . "/" . $fname;
    if ( -d $name ) {
        # don't recurse on . or .. or dotfiles generally
        if ( $fname !~ /^\./ ) {
        print "$name is a dir\n"    if $debug >= 3;
        &recurse_dir( $name );
        }
    } elsif ( &want_file( $name ) ) {
        print "$name is a text file\n"    if $debug >= 3;
        push @filenames, $name;
    } else {
        print "skipping $name\n"    if $debug >= 3;
    }
    if ( $preload_paras and ((rand 100) < 1) ) {
        print "preload mode so printing something while still gatherin
+g filenames (" . (scalar @filenames) . " read so far)\n"    if $debug
+;
        if ( scalar @filenames ) {
        &add_paras;
        &slow_print( int rand scalar @paras );
        } else {
        print "...but there are no usable files yet\n"     if $debug;
        }
    }
    }
    closedir D;
}

I originally had the recurse_dir making a local list of filenames, which it returns to its caller; but after seeing how long it takes to read my entire home directory with this (vs. the benchmarks mentioned in comments above using just a couple of directories with lots of text files), I added the code near the end there, which required making the recurse_dir function add files directly to the global @filenames. This seems to work very well now. Thanks for all your help. The complete script from which the above excerpts are taken is at http://jimhenry.conlang.org/scripts/textual-slideshow.zip.