Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re: More efficient munging if infile very large

by synapse0 (Pilgrim)
on Jul 26, 2001 at 15:40 UTC ( #99947=note: print w/ replies, xml ) Need Help??


in reply to More efficient munging if infile very large

Well, I think you're way overdoing it here.. if you're worried about efficiency and resources, you probably shouldn't be reading in the array at all.. there's no need. And, since you've already determined that $uclc is going to be either one of 'uc' or 'lc', you don't need the extra check in there..
Here's how I woulda done it..

#!/usr/bin/perl -w use strict; my $uclc = shift or Usage(); my $infile = shift or Usage(); my $outfile = shift or Usage(); Usage() unless ($uclc eq 'lc' || $uclc eq 'uc'); open (IN, "< $infile") or die "Error opening $infile for read: $!"; open (OUT, "> $outfile") or die "Error opening $outfile for write: $!" +; foreach (<IN>) { # just print directly on reading, easier on the mem.. # quicker too, and we know that $uclc is going to # be either lc or uc at this point. print OUT $uclc eq 'lc' ? lc() : uc(); } close IN or die "Error closing $infile after read: $!"; close OUT or die "Error closing $outfile after write: $ +!"; ###################################################################### +#### sub Usage { die "\n Usage: uclc.pl (lc|uc) infile outfile\n"; } ###################################################################### +####
-Syn0

Update: didn't look too closely at opening code before, but fixed same bug after Hofmator mentioned it. Also, his elimination of the string equality check in the loop was a good call. ++


Comment on Re: More efficient munging if infile very large
Download Code
Re: Re: More efficient munging if infile very large
by Hofmator (Curate) on Jul 26, 2001 at 16:06 UTC

    I would implement the algorithm in nearly the same way, but I think I would implement it as a filter-like command which you can easily call in pipe sequences. So read-in with the magic <> (this allows for STDIN or infiles) and print to STDOUT (which can be easily redirected into outfile from the command line). This has the additional advantage of shortening the program a bit:

    #!/usr/bin/perl -w use strict; my $uclc = shift or Usage(); # fixed small bug Usage() unless ($uclc eq 'lc' or $uclc eq 'uc'); # to speed up the loop (no string compare on every line) my $lc = $uclc eq 'lc'; while (<>) { print $lc ? lc : uc; } sub Usage { die << EO_USE; Usage: uclc.pl (lc|uc) [infile] reads from STDIN if no infile given EO_USE }

    Update: Oops, thanks tilly, fixed: while instead of foreach.

    -- Hofmator

      Change the foreach to a while.

      The difference is that foreach puts the file read into list context so the entire file has to be held in memory. The while prints as it reads and so will work much better on large files.

      Otherwise I like your changes.

(code) Re: (2) More efficient munging... (improved lc() uc() snippet)
by ybiC (Prior) on Jul 26, 2001 at 18:16 UTC
    Thanks, synapse0 (and others who replied also) for the suggestions.   8^)

    I'm not familiar with the eq 'lc' ? lc() : uc(); syntax from print OUT $uclc eq 'lc' ? lc() : uc();.   Would you be so kind as to explain how that works?

    Updated: 2001-07-29 07:40 CDT

    Here's what I ended up using.   Thanks again, and ++ to all who offered suggestions in nodes and CB!
        cheers,
        Don
        striving toward Perl Adept
        (it's pronounced "why-bick")

    #!/usr/bin/perl -w use strict; my $uclc = shift or Usage(); Usage() unless ($uclc eq 'uc' or 'lc'); while(<>) { print $uclc eq 'uc' ? uc() : lc(); } sub Usage { print "\n Usage: uclc.pl (uc|lc) < infile > outfile\n\n"; exit; } =head1 NAME uclc.pl =head1 SYNOPSIS uclc.pl (uc|lc) < infile > outfile" Convert alpha characters in text file to uppercase (or lowercase). =head1 UPDATED 2001-07-26 16:30 CDT Simplify code for (in|out)put of STD(IN|OUT). Remove unnecessary "or Usage()" from first two input lines. Replace if/else with ternary "?:". my $munged; if ($uclc eq 'uc') { $munged = uc(); } else {$munged = lc(); } print OUT $munged; While instead of for so file read line-by-line not slurp entire fil +e. Post efficiency SoPW to PerlMonks 2001-07-25 Initial working code. =head1 TODOS None that I know of. =head1 TESTED ActivePerl 5.61 on Win2kPro =head1 AUTHOR ybiC =head1 CREDITS Thanks to synapse0, OeufMayo, crazyinsomniac, Masem, MZSanford, virtualsue, Hoffmater, tilly, clemburg, and ichimunki for suggestions. And to davorg for "Data Munging with Perl". Oh yeah, and to some guy named vroom. =cut
      see perlman:perlop and look for the section on Conditional Operator. It's basically an in-place if-then type of construct that evaluates the EXPR $uclc eq 'lc' and returns lc() if the EXPR is true or uc() if the EXPR is false.

      from perlop:

      Conditional Operator Ternary "?:" is the conditional operator, just as in C. It works mu +ch like an if-then-else. If the argument before the ? is true, the arg +ument before the : is returned, otherwise the argument after the : is returned. For example: printf "I have %d dog%s.\n", $n, ($n == 1) ? '' : "s";
      for the gory details read on there ...

      -- Hofmator

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://99947]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (17)
As of 2014-10-22 15:29 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    For retirement, I am banking on:










    Results (119 votes), past polls