Re^2: mmaping a large file

[grondilu]

in reply to Re: mmaping a large file
in thread mmaping a large file

« If you have a large dataset in a file: and a) only need access to small bits of it in any given run; b) you can find those bits without reading through the whole file from the beginning; then mapping can be an effective way on minimising the number of pages read from disk. »

yes, that's exactly the use case.

I've just tried:

use Sys::Mmap;
new Sys::Mmap my $f, 8192, q(bigfile);

but now I get an error during cleaning:

(in cleanup) munmap failed! errno 22 Invalid argument

This is not going to be easy, is it?

[BrowserUk]

Warning: I don't use *nix, so I cannot test anything I'm about to say.

The first thing I notice is that the POD for Sys::Mmap uses

new Mmap my $f, 8192, q(bigfile);

not

new Sys::Mmap my $f, 8192, q(bigfile);

I would have expected you to receive a compile-time error message from that, which suggests you aren't using strict/warnings. It might be a good idea to start.

The other thought is that mapping a 2GB file through a 8k window is going to involve a lot of shuffling.