Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Re: How can you sysread an entire file?

by ikegami (Pope)
on Jan 12, 2006 at 20:15 UTC ( #522800=note: print w/ replies, xml ) Need Help??


in reply to How can you sysread an entire file?

sysread(DF, $rec, -s DF); will save you from specifying an arbitrary number.

foreach(split("\n",$rec)){push(@array,"$_\n";)} causes three copies of the file to be in memory at once. ($rec, foreach's list and @array).

push(@array, $1) while $rec =~ /(.*?\n)/g; will save you from having an third copy of the file in memory, and will save you from re-adding the "\n". Caveat: It will skip the last line if it isn't terminated by a "\n".

Unfortunately, splitting on "\n" is not portable. Keep in mind that sysread has a bug causing it to never translate CRLF to LF in Windows.

Is push(@array, $_) while <DF>; really that much slower?

Any why not use undef $rec; instead of $rec = '';? I prefer to just use curlies, though: my @array; { my $rec; ... }


Comment on Re: How can you sysread an entire file?
Select or Download Code
Re^2: How can you sysread an entire file?
by davidrw (Prior) on Jan 12, 2006 at 20:24 UTC
    Is push(@array, $_) while <DF>; really that much slower?
    Why not just @array = <DF> or push @array, <DF> ?
    Hmm.. as i was about to hit 'Create' i think i answered my own question -- those two make a copy of the whole <DF> array first, right? where your push/while avoids it ...


    Also, to OP's first part, you can use Perl Idioms Explained - my $string = do { local $/; <FILEHANDLE> }; to slurp the whole file ...
      Unfortunately I don't think I can use that "idioms" as I really do need to use sysread... (Due to IO operations being counted by ISP.)

      my $string = do { local $/; <FILEHANDLE> };
      looks nicer than
      my $string; { local $/; $string = <FILEHANDLE> }
      but takes twice as much memory. That's probably not that wise when dealing with entire files.

        takes twice as much memory.

        Do you have some evidence of that as it appears to make no difference at all on my system?

        Update: I take that back. It does double the memory usage.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        I think I made an interesting discovery. As you suggest above, if I do this it only makes one copy of the file in memory:

        P:\test>perl -le" my $s; do{ local $/; $s=<>}; print length $s; system qq[tasklist /fi \"pid eq $$\" ]" 1000000.dat 11000001 Image Name PID Session Name Session# Mem Usag +e ========================= ====== ================ ======== =========== += perl.exe 604 0 12,860 +K

        And if I do it this way it uses two copies:

        P:\test>perl -le" my $s=do{ local $/; <>}; print length $s; system qq[tasklist /fi \"pid eq $$\" ]" 1000000.dat 11000001 Image Name PID Session Name Session# Mem Usag +e ========================= ====== ================ ======== =========== += perl.exe 1372 0 23,624 +K

        But what fooled me, as I was using my perl shell, is that if you eval the statement, only one copy is made:

        P:\test>perl -le" my $s = eval q[do{ local $/; <>}]; print length $s; system qq[tasklist /fi \"pid eq $$\" ]" 1000000.dat 11000001 Image Name PID Session Name Session# Mem Usag +e ========================= ====== ================ ======== =========== += perl.exe 1764 0 12,880 +K

        And I can't quite figure out why that should be so?


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
Re^2: How can you sysread an entire file?
by NeilF (Sexton) on Jan 12, 2006 at 22:23 UTC
    Thanks...

    Not sure if I fully understand what you mean by "\n" is not portable? What exactly is the implication. I've run some tests and it seems to work OK on the files I've used it on (Windows XP & Unix).

    My ISP counts IO operations. Doing a "while <DF>" would mean an IO operation (I believe) for every 512 bytes of data as it is buffered in. By using sysread the whole file is read in by one IO operation. Hence me going down this approach...
      try this under windows and linux
      use strict; use warnings; open(FILE, '>', 'foo.bar') or die "Daim!"; print FILE "\n"; close FILE;
      under windows, the file will have a size of two bytes (\r\n)- under linux just 1 (\n) comapre it to
      use strict; use warnings; open(FILE, '>', 'foo.bar') or die "Daim!"; binmode FILE; print FILE "\n"; close FILE;
        I'm aware of that... All my files on Windows & Unix follow that rule... What's the problem?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://522800]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (5)
As of 2014-07-26 08:33 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (175 votes), past polls