Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re^3: How does the while works in case of Filehandle when reading a gigantic file in Perl

by Corion (Patriarch)
on Feb 06, 2015 at 12:43 UTC ( [id://1115780]=note: print w/replies, xml ) Need Help??


in reply to Re^2: How does the while works in case of Filehandle when reading a gigantic file in Perl
in thread How does the while works in case of Filehandle when reading a gigantic file in Perl

There is nothing particular with a while(<>){ loop that would make the time go up linearly. Maybe some of your processing is consuming memory or accumulating data in an array that gets larger and larger without ever getting cleared. We will need to see more accurate code than the reduced version that was posted.

If the time taken per line gets larger and larger, maybe you can post a small/short XML example and some code more to the point so we can try to replicate the problem? Please also make sure that the problem appears with the code you post.

If you do other work, like for example, inserting the data into a database instead of writing it to a file, that could get slower with each new row that gets added.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1115780]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chilling in the Monastery: (6)
As of 2024-03-28 13:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found