Re^2: Does Perl have a baked-in memory llimit?

[Bod]

Don't use File::Slurp

Is that...
Don't use File::Slurp for this use case or is it don't use File::Slurp EVER! ???

[eyepopslikeamosquito]

Is that... Don't use File::Slurp for this use case or is it don't use File::Slurp EVER! ???

For me, it's ever ... despite being good mates with File::Slurp's original author Uri Guttman. :)

Though mostly a matter of taste for individuals working alone, dependencies are crucial when working in teams and especially for CPAN authors (such as the BOD :) because the price of depending on simple convenience modules, such as File::Slurp, is way too high:

• What if your dependent module has a security vulnerability?
• What if the author abandons it?
• How quickly can you isolate/troubleshoot a bug in its code? (e.g. in a vital production system)

It's a different story for more complex domains, such as DBI and XML, where it makes sense to leverage the work of experts in fields that you are not expert in.

More detail on this topic can be found in the Dependencies section at Writing Solid CPAN Modules.

[einhverfr]

Most software these days has a fairly large number of dependencies, which is part of the reason software bill of materials requirements are becoming a thing. Certainly managing that is a consideration. On the other hand.... Popular CPAN modules provide a lot of benefits including:

• More eyes are likely present
• More hands to take over where needed

My advice instead is to read through code of many of the dependencies you want to use and then decide if you might want have to maintain that code. The bar is far lower in test cases (where I usually use File::Slurp) than in production (where I don't).

Reading through the code gives some sense though of why not to use File::Slurp.....And I do pity the guy who took over maintaining it. And I say this as someone who has taken over CPAN modules in the past over maintenance needs....

[eyepopslikeamosquito]

> The bar is far lower in test cases (where I usually use File::Slurp) than in production (where I don't)

I feel this is a dangerous mindset, especially in larger organisations, as indicated by these two points from Effective Automated Testing:

• Automated test suite needs ongoing maintenance. So test code should be well-designed and maintainable; that is, you should avoid the common pitfall of "oh, it's only test code, so I'll just quickly cut n paste this code".
• It's vital to quarantine intermittently failing tests quickly and to fix them quickly ... only returning them to the main build when reliable (if you don't do that, people start ignoring test failures!). No broken windows.

[ikegami]

yeah, cause you should be using File::Slurper :)

[hippo]

I don't use File::Slurp ever (but without the uppercase and the punctuation :-)

Leon's blog post explains (some of) the problems with File::Slurp and at the end offers three alternatives. I tend to use Path::Tiny in preference because it isn't overkill, is already widely used and does just about anything I am likely to want in regards to files and directories, including slurping and spewing. YMMV.

I should also say that for published code which only slurps one single file I would probably just do that without a module as the overhead isn't really worth it in terms of efficiency and maintenance.

[gnosti]

Path::Tiny is preferred to File::Slurp for security reasons. This is not relevant to the OP's issue, where slurping the entire file (by whichever method) hits memory limits.

[eyepopslikeamosquito]

This is not relevant to the OP's issue, where slurping the entire file (by whichever method) hits memory limits

How can you be sure, given the OP has not provided a SSCCE, and has not posted any details on the out of memory error message?

[gnosti]

You're correct. The second sentence is not justified.