Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery

Re: Out of Memory - Line of TXT too large (mmap)

by oiskuu (Hermit)
on Jan 02, 2014 at 15:45 UTC ( #1068977=note: print w/replies, xml ) Need Help??

in reply to Out of Memory - Line of TXT too large

Scan a very big file, working on some portions? This a good opportunity for mmap. For example:
use File::Map ':all'; my $file = shift; my $SEP = "\n"; # line/chunk separator my ($s, $e); open(my $fh, '<', $file) || die "$!"; map_handle(my $mmap, $fh, '<'); sub process {} for ($e = 0;; $e += length($SEP)) { ($s, $e) = ($e, index($mmap, $SEP, $e)); print "fragment [$s, $e)\n"; process($e < 0 ? substr($mmap, $s) : substr($mmap, $s, $e-$s)); last if $e < 0; }

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1068977]
[Happy-the-monk]: LanX&Corion: renting a car might be an option... ~150 but we need to travel early on Tuesday.
[LanX]: early on tuesday (morning)? why?
[LanX]: trains are probably not that bad...
[Happy-the-monk]: LanX: Perl@embedded/IoT workshop on Tuesday Morning.

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (7)
As of 2018-03-19 12:50 GMT
Find Nodes?
    Voting Booth?
    When I think of a mole I think of:

    Results (240 votes). Check out past polls.