Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Re^2: mmaping a large file

by grondilu (Pilgrim)
on Aug 24, 2012 at 00:10 UTC ( #989414=note: print w/ replies, xml ) Need Help??


in reply to Re: mmaping a large file
in thread mmaping a large file

If you have a large dataset in a file: and a) only need access to small bits of it in any given run; b) you can find those bits without reading through the whole file from the beginning; then mapping can be an effective way on minimising the number of pages read from disk.

yes, that's exactly the use case.

I've just tried:

use Sys::Mmap; new Sys::Mmap my $f, 8192, q(bigfile);

but now I get an error during cleaning:

(in cleanup) munmap failed! errno 22 Invalid argument

This is not going to be easy, is it?


Comment on Re^2: mmaping a large file
Download Code
Re^3: mmaping a large file
by BrowserUk (Pope) on Aug 24, 2012 at 00:26 UTC

    Warning: I don't use *nix, so I cannot test anything I'm about to say.

    The first thing I notice is that the POD for Sys::Mmap uses

    new Mmap my $f, 8192, q(bigfile);

    not

    new Sys::Mmap my $f, 8192, q(bigfile);

    I would have expected you to receive a compile-time error message from that, which suggests you aren't using strict/warnings. It might be a good idea to start.

    The other thought is that mapping a 2GB file through a 8k window is going to involve a lot of shuffling.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://989414]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (5)
As of 2014-11-23 09:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (129 votes), past polls