Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid

Swiss army knife module for files?

by nysus (Vicar)
on Apr 12, 2019 at 20:30 UTC ( #1232514=perlquestion: print w/replies, xml ) Need Help??

nysus has asked for the wisdom of the Perl Monks concerning the following question:

Very often I find myself hunting around for Perl modules to do things to files. Seems like there might be a need for a big, giant, bloated module chock full of all kinds of utilities and other modules to get just about any kind of information you can imagine about a file and do things to it. I couldn't find a module like this though. There's probably a good reason, but I'm not sure what it is. Thinking of building one just for fun, though.

$PM = "Perl Monk's";
$MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest Vicar";
$nysus = $PM . ' ' . $MCF;
Click here if you love Perl Monks

Replies are listed 'Best First'.
Re: Swiss army knife module for files?
by eyepopslikeamosquito (Chancellor) on Apr 13, 2019 at 00:05 UTC

        Yes, IO::All was the brainchild of mad scientist Ingy aka Brian Ingerson aka Ingy Dot Net (who is also the original co-author of YAML).

        Since you are developing your own modules, you might be interested in On Interfaces and APIs, especially the "Conway's Sufficiently Advanced Technologies" section, which describes an approach to module design by another mad scientist, Damian Conway. Conway's SAT-approach can be seen in an older module by yet another mad scientist, Mark Jason Dominus aka MJD, the original author of Tie::File (found some old MJD slides on Tie::File here). While Tie::File made it into the Perl core, IO::All did not.

      Ah, great find. Never even heard of it. Thanks.

      $PM = "Perl Monk's";
      $MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest Vicar";
      $nysus = $PM . ' ' . $MCF;
      Click here if you love Perl Monks

        It's been mentioned here many, many times. :P

Re: Swiss army knife module for files?
by Your Mother (Bishop) on Apr 12, 2019 at 21:30 UTC

    If Path::Tiny and built-in ops don’t do what you want then what you want is probably … not ideal. Always build new stuff though. That’s half the fun.

      There's this one:

      $PM = "Perl Monk's";
      $MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest Vicar";
      $nysus = $PM . ' ' . $MCF;
      Click here if you love Perl Monks

      Yeah, that's what I'm thinking. I'll just do it for the fun and to learn new stuff. If people use find use for it and it helps them be lazy and they can spare the extra few MB of memory and tenth of a second extra load time, that'll be an extra bonus.

      The module would basically do all the built-ins and subsume other useful modules like Path::Tiny but put it all under a common interface.

      $PM = "Perl Monk's";
      $MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest Vicar";
      $nysus = $PM . ' ' . $MCF;
      Click here if you love Perl Monks

Re: Swiss army knife module for files?
by haukex (Chancellor) on Apr 13, 2019 at 07:50 UTC
Re: Swiss army knife module for files?
by stevieb (Abbot) on Apr 13, 2019 at 19:36 UTC

    Hey nysus,

    I don't mean to take away from other suggestions here, so I'm going to divert into something I've learned when I set out to re-invent the wheel for learning purposes.

    First thing that's a good idea, is to literally write out a point-form list of ALL of the functionality you want to include, including sub-points for top-level ones if they are complex. Don't think about every single detail here, just the general overall functionality you want to provide. You can add/subtract items later.

    Next, because apparently you'll be including several other distributions to fulfill your overall objective of consolidation, document down what distributions and functions/methods you'll need for each point you created above. If you know you'll need to add your own or write wrappers around what currently exists, document this. A spreadsheet (or project doc or equivalent) really helps at this point.

    Third, go over it all thoroughly. You may spot things you forgot, or need to change.

    Fourth, ask around for advice on your plan now that you've completed and revised it. People may know things you're missing, forgetting or essentially just don't know (none of us know everything!).

    After that, dig into the code of the modules you'll be using to pull together your plan, and start designing a consistent API around all of the things you plan on putting together. A good idea is to write some tests against your initial functionality as you envision it (a couple of functions at a time), then start writing the code.

    Because you've written your tests from the perspective of a "user" using your software, you can then proceed to start writing actual code to match up with the tests. As the tests pass, write your documentation, even if it's in a basic form. I've found that if this isn't done early (even if it needs to be re-written), it often doesn't get done, or is done incorrectly (causing frustration from your users).

    On the testing part, don't get ahead of yourself. A few test files for a few pieces of functionality isn't too bad in if you change your mind and then have to re-write everything. Writing too many tests at first may cause you frustration if you do decide to change the API mid-flight (because you haven't even really fully designed it yet). You may have to start over a few times, but at least you've got test code that can be easily modified to fit API changes.

    Get some feedback at this point on your API, then start moving forward with more tests, more functionality, more tests, more functionality etc.

    Just my 5 cents (because anything below that is rounded nowadays anyhow... at least in Canada (Go Leafs Go! ;)


Re: Swiss army knife module for files?
by karlgoethebier (Monsignor) on Apr 14, 2019 at 16:36 UTC
    "...any kind of information you can imagine about a file and do things to it"

    Nomem est omem. See the section about method roll call. eyepopslikeamosquito linked already to a node about some pros and cons. I'm glad that some fellow monks remember what i wrote years ago. See also. Best regards, Karl

    «The Crux of the Biscuit is the Apostrophe»

    perl -MCrypt::CBC -E 'say Crypt::CBC->new(-key=>'kgb',-cipher=>"Blowfish")->decrypt_hex($ENV{KARL});'Help

Re: Swiss army knife module for files?
by learnedbyerror (Monk) on Apr 22, 2019 at 06:14 UTC

    nysus, my imagination is likely smaller than yours. Path::Tiny by xdg aka DAGOLDEN provides me all of the file information that I need and more. He also has a module Path::Iterator::Rule that provides complimentary functionality to iterate over a path selecting files that meet certain criteria.

    In my personal experience, I often think about writing a new module to share with my fellow perl devotee's. However, as I start digging into it, often find that someone has beet me to it. I don't know if this will be the case for you here.

    Good Luck,


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1232514]
Approved by dorko
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (4)
As of 2019-06-19 03:11 GMT
Find Nodes?
    Voting Booth?
    Is there a future for codeless software?

    Results (83 votes). Check out past polls.

    • (Sep 10, 2018 at 22:53 UTC) Welcome new users!