Importing Data and Exporting Data Issues

[MikeyG]

I'm having difficulties compiling my script. The purpose of this script it to take numerical data from file #1 (input file), finding the largest, smallest, and the average of the said data and printing it to file #2 (output file).

I haven't attempted to go an further than the average because my script doesn't seem to recognize my data in the input file.

Input file as off right now just has the numbers 1-10. Here is my code:

#!/usr/bin/perl -w
use strict;

my @lines;
my $ave;

print "Enter an input file name: ";
chomp (my $infile = <STDIN>);

while (!(-e $infile)) {
    print "Invalid file name, Try again: ";
    chomp ($infile = <STDIN>);
}
print "\n";

print "Enter an output file name: ";
chomp (my $outfile = <STDIN>);

open (FIN,'<', $infile) or die "$!\n";
chomp (@lines = <FIN>);
open (FOUT, '>', $outfile) or die "$!\n";

while (<STDIN>) {
    $ave = average(<FIN>);
     print FOUT $ave;
}
close FOUT;

sub average {
    my $sum = 0;
    my $size = @lines;
    $sum = $sum + $_ foreach @lines;
    $sum / $size;
}

sub largest {
    my $max;
    for (@lines){
        $max = $_ if !$max || $_ > $max;
    } 
    $max
}

I keep getting an error saying Illegal division by zero in bsad.pl at line 52. Please help me solve this mystery.

[GrandFather]

It kinda looks like you have copied code from somewhere else without much understanding and poured it into an editor. Reading and understanding other code is a good way to learn. Copying without understanding is a good way to get frustrated. Lets see if we can tip the balance toward understanding and introduce some good habits along the way.

1. First off, use strict is good! You should use warnings rather than adding the -w switch though.
2. Don't declare variable until you need them. Both @lines and $ave are declared out of context.
3. If you are looking for a file use -f instead of -e. (See -X documentation.)
4. Mega points for three parameter open and checking the result. Bonus points if you use lexical file handles and show the file name in the die: open my $fIn, '<', $infile or die "Can't open '$inFile': $!\n"
5. What do you understand chomp (@lines = <FIN>); to do?
6. What do you expect while (<STDIN>) {...} do to?
7. What value do you expect average to see in average(<FIN>);
8. Always use an explicit return statement if a sub is expected to return a result.

[MikeyG]

1. Done!

2. Should I declare @lines after opening the files then? Should I declare $ave in my average function?

3. I'm using -e to check and see if the file even exists before attempting to open and read from it.

4. I have not used lexical file handles before, are they used similarly to how file handles are used normally?

5. I was attempting to put all my data into an array so my functions would read it and work accordingly.

6. Experiment, meant to change it back to  (<FIN>)

7. Experiment again, was originally @lines.

8. Done!

[Marshall]

I'll address one point. Grandfather gave good advice.

3. -e vs -f. The documentation is not super clear about this. Basically a directory is also a "file", albeit a special one. The -e test will return true if the name is a directory or some other weirdo things that aren't relevant now. The -f test will return true if (a)the file exists AND (b)the file is a "plain file". Use the -f test. The -e test can return true for some things that aren't simple readable files.

[GrandFather]

Good answers. Mind you, that does show a degree of unwarranted laziness in cleaning up your code for posting! In fact, if you'd cleaned up your code you quite likely would have fixed the problem.

However, to continue the lesson lets see what the code might look like if you follow the advice. Consider:

#!/usr/bin/perl
use strict;
use warnings;

my $inputData = <<IN;
1
2
3
4
5
6
7
8
9
10
IN

open my $fIn, '<', \$inputData;
chomp (my @lines = <$fIn>);

printf "Average: %.2f\n", average(@lines);
printf "Largest: %d\n", largest(@lines);


sub average {
    my $sum  = 0;
    $sum += $_ for @_;
    return $sum / @_;
}


sub largest {
    my $max = shift @_;
    
    for my $value (@lines) {
        $max = $value if $value > $max;
    }
    
    return $max;
}

Prints:

Average: 5.50
Largest: 10

Note that I've stripped out the code dealing with getting a file name, validating it and reading a file from disk. Instead I "open" a string as a file. That gives me a quick test framework so I can easily make changes to the code and test them.

$fIn is a lexical file handle (variables declared with my are lexical). Lexical file handles have the advantage that strict can better check their usage so you have fewer issues with misspelled file handle identifiers. When lexical file handles go out of scope (execution leaves the block the variable was declared in) the file is closed so you mostly avoid files remaining open too long if you forget a close. Aside from that lexical file handles work pretty much the same.

Note that we declare @lines where we assign content to it and we don't need $avg at all.

[Athanasius]

Hello MikeyG,

GrandFather has already highlighted the loop while (<STDIN>) { ... } as a problem area in your code. I want to elaborate a little on the issues this raises, because one of them (context) is a key feature of the Perl language.

First, the existence of a loop in this part of the code makes little sense, since by this point the input has been read in and won’t change — so what’s the point of calculating the average of the same set of numbers more than once?

Second, what does while (<STDIN>) actually do? If you look at the documentation for Perl’s builtin readline function, you’ll see that <STDIN> is a convenient synonym for readline(*STDIN), which means that the expression <STDIN> is really a call to the readline function. Here is an extract from the documentation for that function:

In scalar context, each call reads and returns the next line until end-of-file is reached, whereupon the subsequent call returns undef. In list context, reads until end-of-file is reached and returns a list of lines.

So <STDIN> reads a single line from the keyboard, but only if it’s called in scalar context. You can see the difference in the following two assignments:

my $line  = <FIN>;    # reads one line from the file referenced by the
+ filehandle FIN
my @lines = <FIN>;    # reads all the lines at once

The first reads a single line only, because assignment to a scalar variable imposes scalar context on the readline operation. But the second reads the whole file, because assignment to an array variable imposes list context on the operation.

And now the point: the condition of a while loop is evaluated in list context, so while (<STDIN>) goes on reading until it encounters end-of-file. On my Windows system, you can enter end-of-file from the keyboard by typing Control-Z followed by Return; but otherwise (if the user doesn’t know about Control-Z), you have an infinite loop.

Update (Apr 07, 2016): This is wrong, because the condition of a while loop is actually evaluated in scalar (boolean) context. The reason the while (<STDIN>) loop is not terminated by simply pressing Return is that the input line still contains a newline character, so Perl considers it “true.” My point should have been directed to the next line of code, $ave = average(<FIN>);, where, as the argument to a subroutine call, <FIN> is evaluated in list context.

See Context tutorial.

Hope that’s of interest,

[1nickt]

Hi MikeyG,

Finding the maximum value in a list is a common task. So is adding up the values. Reading from and writing to files are very common tasks. One of the reasons to use Perl is so you can get stuff done quickly: it "makes the easy stuff easy and the hard stuff possible."

So you can use standard tools for doing everyday tasks, and get back to solving your actual problem.

use strict; use warnings;
use Path::Tiny qw/ path /;
use List::Util qw/ max sum /;

my $in  = '1159573.dat'; my $out = '1159573.out';

my @lines = path( $in )->lines({ chomp => 1 });

my $max = max @lines;
my $avg = sum( @lines ) / scalar( @lines );

path( $out )->spew( @lines . " values; max: $max; avg: $avg" );

__END__

Output:

$ cat 1159573.out
10 values; max: 10; avg: 5.5

Note: you'll want to handle the case of no lines in your file so you don't try to divide by zero. If you're running your script periodically and you don't want to create a fresh output file each time, use path( $out )->append() instead.

Note: List::Util is distributed with Perl so you have it already, and Path::Tiny is one file, pure-Perl (no compiler required) with no non-core dependencies, so you can run it no matter what.

Hope this helps!

[Anonymous Monk]

Where is the sample data file?

You're using subroutines, this is very good, now go the extra step and have them accept arguments :)

[MikeyG]

As of right now I'm using a simple txt file called input.txt for the input and the output is as of right now just another txt called output.txt

1
2
3
4
5
6
7
8
9
10

That is my input.txt file. My output file doesn't display anything. I'm little confused on what you mean by making them accept arguments can you please elaborate?

[Anonymous Monk]

See http://perldoc.perl.org/perlintro.html#Writing-subroutines and Modern Perl chapter 5