Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re: Making this script process 56,000 lines 5 times faster

by rizzo (Deacon)
on Mar 22, 2018 at 07:27 UTC ( #1211488=note: print w/replies, xml ) Need Help??


in reply to Making this script process 56,000 lines 5 times faster

To me it seems that by rebuilding grep's and awk's "line-by-line" processing you are taking Perl's drawbacks(starting the interpreter) but you don't profit from its power.

Why not loop over the response directly using the /g modifier:

while($response =~ /(\n0\.0\.0\.0)(\s)([\w\.]+?)\s+/g ) { print ".... $3 ..."; }

Replies are listed 'Best First'.
Re^2: Making this script process 56,000 lines 5 times faster
by kris004 (Initiate) on Mar 22, 2018 at 18:23 UTC

    Hi, thank you for this tip, it makes sense as long as I'm not worried about running out of memory.

      > as long as I'm not worried about running out of memory.

      Where do you see a memory problem here?

      Cheers Rolf
      (addicted to the Perl Programming Language and ☆☆☆☆ :)
      Wikisyntax for the Monastery

        If I say, read in a 4gb file. Not really a problem with this hosts file. I assumed that if it wasn't line by line, then the whole file would have to be loaded into memory.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1211488]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (3)
As of 2022-09-27 17:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    I prefer my indexes to start at:




    Results (122 votes). Check out past polls.

    Notices?