Re: Reading (the same) data in different ways & memory usage

[jwkrahn]

This doesn't answer your question but:

sub ReadData ($$) {
    my ($self, $filename) = @_;
    my $ar_returnvalue = [];
    if (!-e "$filename") {
        Carp::carp("File [$filename] does not exist");
        return undef;
    }
    open (FLATFILE, '<', $filename) or Carp::croak("Cannot open file [
+$filename]");
    while (<FLATFILE>) {
        chomp;
        push (@{$ar_returnvalue}, Interfaces::FlatFile::ReadRecord($se
+lf, $_));
    }
    close (FLATFILE);
    return $ar_returnvalue;
} ## end sub ReadData ($$)

You are using prototypes but prototypes were introduced to allow programmers to imitate Perl's built-in functions, not for user code per se.    FMTEYEWTK on Prototypes in Perl

You are testing for the existence of a file twice, first with stat and then with open.    In the stat test you are unnecessarily copying the file name to a string before testing it.    What's wrong with always quoting "$vars"?

You should include the $! variable in your error messages so you know why they failed.

        $hr_returnvalue->{$CurrentColumnName} =~ s/^\s*(.*?)\s*$/$1/; 
+   # Trim whitespace

That is usually written as:

        s/^\s+//, s/\s+$// for $hr_returnvalue->{$CurrentColumnName}; 
+   # Trim whitespace

Which avoids unnecessary substitution.

                if ($self->datatype->[$_] =~ /^(?:CHAR|VARCHAR|DATE|TI
+ME|DATETIME)$/) {
                    $hr_returnvalue->{$CurrentColumnName} = sprintf ("
+%s", $self->standaard->[$_]);
                } else {
                    $hr_returnvalue->{$CurrentColumnName} = $self->sta
+ndaard->[$_];
                }

What is the sprintf doing that the simple assignment is not doing?    It looks like you don't need this test at all.

[Neighbour]

And so I'm learning new things every day :)


I have experienced difficulties with the @ and % prototypes, as I read in the link you provided, and stopped using those. The scalar prototypes are now purely used to enforce the right amount of arguments (as an added bonus, perl implicitly coerces any arguments supplied to scalars...this will definitely mess things up, but if you were supplying non-scalars to this function, that was bound to happen anyway, so I'm not worried about that much).

It seems that open accepts one thing besides strings as the EXPR-argument (2nd or 3rd if a MODE is supplied), and that is a reference to a scalar to be used as an in-memory file. Even though this is not the case here, I've removed the stat-check, since open will give an error anyway if the file doesn't exist. Also $! has been included in the errormessage, should open fail.

How does one substitution with capture compare to doing 2 substitutions without capture? I would have to benchmark this to figure out which is faster.

The sprintf seems out of place, though, as with all user-supplied data, it's not certain that the default-values (looks like I missed one when translating "standaard" to "default") for (VAR)CHAR fields actually contains a string-value. However, this can also be done just using ""'s.

[jwkrahn]

How does one substitution with capture compare to doing 2 substitutions without capture?

In your example:

        $hr_returnvalue->{$CurrentColumnName} =~ s/^\s*(.*?)\s*$/$1/; 
+   # Trim whitespace

The regular expression /^\s*(.*?)\s*$/ will always match, regardless if there is or is not whitespace present, so the substitution will always be done.

In my example:

        s/^\s+//, s/\s+$// for $hr_returnvalue->{$CurrentColumnName}; 
+   # Trim whitespace

The regular expressions /^\s+/ and /\s+$/ will only match if there is whitespace present and so the substitution will only be done on the occurrence of whitespace.

Running a benchmark would be good start, and you should try to use data similar to, or the actual data that your program uses.