Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

I don't see how that would work. You're proposing fixing a performance issue caused by fork/exec/re-reading a large file from the beginning on each iteration, by doing exactly the same thing but with a virtual machine added in the middle instead of optimised C code? This isn't a perl-specific question you're asking, it's fairly generic. The proposed solutions (e.g., Tie::File) include some perl-specific suggestions, and some that aren't (read the file into memory as a list/array - you can do that in C++ with the STL fairly easily, and Java should make it pretty simple, too), but the general issue is language-agnostic.

Instead, if you read it all into memory, you can use likely just a line to duplicate. Without testing or even compiling:

# do this once. OUTSIDE OF YOUR LOOP. my @read_source_lines = do { open my $fh, '<', $read_source or die "Can't read from $read_source: + $!"; <$fh> }; # you may also need: chomp @read_source_lines; # gets rid of \n's. # inside the loop, instead of $strx/$str5: my $str5 = $read_source_lines[$recnum]; print REPORT "$_|$recnum$str5\n";
Assuming you don't start swapping, this should eliminate most of your time. Note that there are better/faster ways to do this, but this will get you most of the benefit for the least amount of effort. Many of those better ways are actually embedded in Tie::File, IIRC (reading only as many lines as is currently needed, continuing from where you left off, maybe you don't need to read the entire file, this may also allow the OS to continue reading the file in the background to fill up your input buffers while you go do other work, that type of thing).

In reply to Re^3: Need help to fine tune perl script to make it faster( currently taking more than 30 minutes) by Tanktalus
in thread Need help to fine tune perl script to make it faster( currently taking more than 30 minutes) by anujajoseph

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and all is quiet...

    How do I use this? | Other CB clients
    Other Users?
    Others chilling in the Monastery: (4)
    As of 2018-05-26 21:59 GMT
    Find Nodes?
      Voting Booth?