Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re^2: Possible to have regexes act on file directly (not in memory)

by karlgoethebier (Curate)
on May 02, 2014 at 18:41 UTC ( #1084836=note: print w/ replies, xml ) Need Help??


in reply to Re: Possible to have regexes act on file directly (not in memory)
in thread Possible to have regexes act on file directly (not in memory)

This is :mmap described in the Custom Layers section of PerlIO?

Regards, Karl

«The Crux of the Biscuit is the Apostrophe»


Comment on Re^2: Possible to have regexes act on file directly (not in memory)
Download Code
Re^3: Possible to have regexes act on file directly (not in memory)
by BillKSmith (Chaplain) on May 02, 2014 at 23:05 UTC
    I have always been intimidated by perl's IO layer concept. The documentation is sparse at best. Perhaps it is time for another look.
    Bill
Re^3: Possible to have regexes act on file directly (not in memory)
by Tanktalus (Canon) on May 03, 2014 at 02:14 UTC

    From a cursory glance at the documentation, no. That's still treating that memory map as a filehandle. You need to treat it as a string or scalar. A quick check on cpan gives Sys::Mmap as a reasonable candidate based on its documentation.

      In principle this seems to work. Here's an example:

      use strict; use warnings; use Sys::Mmap; my $string; open my $fh, '<', '/usr/share/dict/words' or die $!; Sys::Mmap::mmap( $string, 0, MAP_SHARED, PROT_READ, $fh ); while( $string =~ m/^(wal.*)$/mg ) { print $1, "\n"; } Sys::Mmap::munmap( $string ) or die "munmap: $!"; close $fh;

      On my system I get a dump of everything in the dict/words file starting with 'wal'. However, I end up getting the following error message when munmap is called:

      variable is not a string at ./mytest.pl line 17.

      The XS code testing whether the variable being unmapped is a string is this:

      if(SvTYPE(var) != SVt_PV) { croak("variable is not a string"); return; }

      ...which doesn't strike me as wrong. Must be something in the XS for 'mmap' that is not happening properly.


      Dave

      The winner seems to be File::Map. It is less complicated to use, and at least performs in correct accordance to its documentation (unlike Sys::Mmap). ++ to leont once again.

      use strict; use warnings; use File::Map 'map_file'; map_file my $map, '/usr/share/dict/words', '<'; while( $map =~ m/^(wal.*)$/mg ) { print "$1\n"; }

      Though its user interface is fairly simple, it is a module where reading 100% of the documentation is probably a very good idea.


      Dave

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1084836]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (6)
As of 2014-09-16 02:04 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite cookbook is:










    Results (155 votes), past polls