Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re^2: Possible to have regexes act on file directly (not in memory)

by karlgoethebier (Vicar)
on May 02, 2014 at 18:41 UTC ( #1084836=note: print w/ replies, xml ) Need Help??


in reply to Re: Possible to have regexes act on file directly (not in memory)
in thread Possible to have regexes act on file directly (not in memory)

This is :mmap described in the Custom Layers section of PerlIO?

Regards, Karl

«The Crux of the Biscuit is the Apostrophe»


Comment on Re^2: Possible to have regexes act on file directly (not in memory)
Download Code
Re^3: Possible to have regexes act on file directly (not in memory)
by BillKSmith (Deacon) on May 02, 2014 at 23:05 UTC
    I have always been intimidated by perl's IO layer concept. The documentation is sparse at best. Perhaps it is time for another look.
    Bill
Re^3: Possible to have regexes act on file directly (not in memory)
by Tanktalus (Canon) on May 03, 2014 at 02:14 UTC

    From a cursory glance at the documentation, no. That's still treating that memory map as a filehandle. You need to treat it as a string or scalar. A quick check on cpan gives Sys::Mmap as a reasonable candidate based on its documentation.

      In principle this seems to work. Here's an example:

      use strict; use warnings; use Sys::Mmap; my $string; open my $fh, '<', '/usr/share/dict/words' or die $!; Sys::Mmap::mmap( $string, 0, MAP_SHARED, PROT_READ, $fh ); while( $string =~ m/^(wal.*)$/mg ) { print $1, "\n"; } Sys::Mmap::munmap( $string ) or die "munmap: $!"; close $fh;

      On my system I get a dump of everything in the dict/words file starting with 'wal'. However, I end up getting the following error message when munmap is called:

      variable is not a string at ./mytest.pl line 17.

      The XS code testing whether the variable being unmapped is a string is this:

      if(SvTYPE(var) != SVt_PV) { croak("variable is not a string"); return; }

      ...which doesn't strike me as wrong. Must be something in the XS for 'mmap' that is not happening properly.


      Dave

      The winner seems to be File::Map. It is less complicated to use, and at least performs in correct accordance to its documentation (unlike Sys::Mmap). ++ to leont once again.

      use strict; use warnings; use File::Map 'map_file'; map_file my $map, '/usr/share/dict/words', '<'; while( $map =~ m/^(wal.*)$/mg ) { print "$1\n"; }

      Though its user interface is fairly simple, it is a module where reading 100% of the documentation is probably a very good idea.


      Dave

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1084836]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (6)
As of 2015-07-01 23:46 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (25 votes), past polls