Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re: Does Perl have a baked-in memory llimit?

by eyepopslikeamosquito (Archbishop)
on Nov 11, 2023 at 22:33 UTC ( [id://11155564]=note: print w/replies, xml ) Need Help??


in reply to Does Perl have a baked-in memory llimit?

I'm trying to read in using File::Slurp. I get an out of memory error.

Don't use File::Slurp!

Just do it idiomatically without a module (my $string = do { local $/; <$fh> }) as described in more detail by haukex here.

See also:

Updated: extra references were added later

👁️🍾👍🦟

Replies are listed 'Best First'.
Re^2: Does Perl have a baked-in memory llimit?
by Bod (Parson) on Nov 12, 2023 at 00:06 UTC

      Is that... Don't use File::Slurp for this use case or is it don't use File::Slurp EVER! ???

      For me, it's ever ... despite being good mates with File::Slurp's original author Uri Guttman. :)

      Though mostly a matter of taste for individuals working alone, dependencies are crucial when working in teams and especially for CPAN authors (such as the BOD :) because the price of depending on simple convenience modules, such as File::Slurp, is way too high:

      • What if your dependent module has a security vulnerability?
      • What if the author abandons it?
      • How quickly can you isolate/troubleshoot a bug in its code? (e.g. in a vital production system)

      It's a different story for more complex domains, such as DBI and XML, where it makes sense to leverage the work of experts in fields that you are not expert in.

      More detail on this topic can be found in the Dependencies section at Writing Solid CPAN Modules.

      👁️🍾👍🦟

        Most software these days has a fairly large number of dependencies, which is part of the reason software bill of materials requirements are becoming a thing. Certainly managing that is a consideration. On the other hand.... Popular CPAN modules provide a lot of benefits including:

        • More eyes are likely present
        • More hands to take over where needed

        My advice instead is to read through code of many of the dependencies you want to use and then decide if you might want have to maintain that code. The bar is far lower in test cases (where I usually use File::Slurp) than in production (where I don't).

        Reading through the code gives some sense though of why not to use File::Slurp.....And I do pity the guy who took over maintaining it. And I say this as someone who has taken over CPAN modules in the past over maintenance needs....

      I don't use File::Slurp ever (but without the uppercase and the punctuation :-)

      Leon's blog post explains (some of) the problems with File::Slurp and at the end offers three alternatives. I tend to use Path::Tiny in preference because it isn't overkill, is already widely used and does just about anything I am likely to want in regards to files and directories, including slurping and spewing. YMMV.

      I should also say that for published code which only slurps one single file I would probably just do that without a module as the overhead isn't really worth it in terms of efficiency and maintenance.


      🦛

        This is not relevant to the OP's issue, where slurping the entire file (by whichever method) hits memory limits

        How can you be sure, given the OP has not provided a SSCCE, and has not posted any details on the out of memory error message?

        👁️🍾👍🦟

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11155564]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others wandering the Monastery: (3)
As of 2025-05-21 02:48 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?
    erzuuliAnonymous Monks are no longer allowed to use Super Search, due to an excessive use of this resource by robots.