Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"

Re^2: mmaping a large file

by grondilu (Friar)
on Aug 24, 2012 at 00:10 UTC ( #989414=note: print w/replies, xml ) Need Help??

in reply to Re: mmaping a large file
in thread mmaping a large file

If you have a large dataset in a file: and a) only need access to small bits of it in any given run; b) you can find those bits without reading through the whole file from the beginning; then mapping can be an effective way on minimising the number of pages read from disk.

yes, that's exactly the use case.

I've just tried:

use Sys::Mmap; new Sys::Mmap my $f, 8192, q(bigfile);

but now I get an error during cleaning:

(in cleanup) munmap failed! errno 22 Invalid argument

This is not going to be easy, is it?

Replies are listed 'Best First'.
Re^3: mmaping a large file
by BrowserUk (Pope) on Aug 24, 2012 at 00:26 UTC

    Warning: I don't use *nix, so I cannot test anything I'm about to say.

    The first thing I notice is that the POD for Sys::Mmap uses

    new Mmap my $f, 8192, q(bigfile);


    new Sys::Mmap my $f, 8192, q(bigfile);

    I would have expected you to receive a compile-time error message from that, which suggests you aren't using strict/warnings. It might be a good idea to start.

    The other thought is that mapping a 2GB file through a 8k window is going to involve a lot of shuffling.

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://989414]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (4)
As of 2018-02-18 08:01 GMT
Find Nodes?
    Voting Booth?
    When it is dark outside I am happiest to see ...

    Results (251 votes). Check out past polls.