Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Bundling commonly-used modules into a toolset

by xdg (Monsignor)
on Oct 25, 2005 at 10:37 UTC ( #502654=perlmeditation: print w/replies, xml ) Need Help??

A minor pet peeve for me when programming Perl is the degree to which I wind up repeating code for things that I use over and over again. At the subroutine level, it's my $self = shift. At the file level, a prime example is use strict and use warnings, but it includes importing utility modules like Carp and Scalar::Util as well.

I was hopeful when I heard about TheDamian's Toolkit, but was then disappointed to see that it uses config files in PERL5LIB -- which works fine for my own programs but is not necessarily easily shared with others or with end users of things.

I decided to pick up and dust off an old idea of mine -- a module that bundles other modules into a toolset that imports all my "usual" stuff in one shot. In essence, I want this:

use My::Tools; # turns strict on # turns warnings on # makes all my usual imports available

This is now possible with a new module I wrote called ToolSet. (The alpha version is now on CPAN.) For example:

package My::Tools; use base 'ToolSet'; # handles the import() ToolSet->set_strict(1); ToolSet->set_warnings(1); ToolSet->export( 'Carp' => undef, # all the Carp defaults 'Scalar::Util' => 'refaddr blessed weaken', 'List::Util' => 'first reduce' ); 1; # return true

One of the more interesting discoveries I made along the way is how to get strict and warnings to be "contagious":

use My::Tools; $var = 1; # causes a compile error because we're using strict

The breakthrough came from reading the source for strict and reading about $^H in perlvar. I realized that all I had to do was call strict's import sub without creating a new BEGIN block during ToolSet's own import routine:

# simple example of strict contagion sub import { require strict; strict->import; }

This applies the strictures when the BEGIN block for use My::Tools finishes -- i.e. to the code that called use My::Tools. The same principle applied for warnings.

The rest so far is just using eval to import the defined export modules into the calling package. In subsequent versions, I'll look to add the equivalent of @EXPORT_OK and @EXPORT_TAGS. I'd also like to add lazy exporting -- where a stub is installed that, when called, will require a module and replace the stub with the real function.

I hope some people may find this useful. As always, I'd appreciate any feedback or ideas.

Update: As another example of how this might be used, imagine a Perl6::Now ToolSet, that turned on strict and loaded all of the relevant Perl6::* modules.


Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Replies are listed 'Best First'.
Re: Bundling commonly-used modules into a toolset
by blazar (Canon) on Oct 25, 2005 at 11:47 UTC
    Seems extremely interesting. But in passing it may be worthwile to remind a cheap alternative that would yield the absolute maximum of portability: editor macros or templates. For example you may create a template like thus
    #!/usr/bin/perl use strict; use warnings; __END__
    If you know that you're using a certain module, say File::Find, quite often, then you can include it with a cmt in said template:
    # use File::Find;
    and you may create a suitable macro for inserting subs: I don't have any, but with my editor (jed]) it would be easy to create one that would ask me for a name, say foo and create this code for me:
    sub foo { my $var=shift; }
    positioning the cursor on the empty line, or on $var, to allow me to switch to a more sensible var name.
      But in passing it may be worthwile to remind a cheap alternative that would yield the absolute maximum of portability: editor macros or templates.
      Seconded. Whenever I start a perl file, I start with a macro that loads one of three templates for me (one for programs, one for non-OO modules, and one for OO modules). They include things like a she-bang line, use strict, use/no warnings, modules I often use with a '#' in front of the use so I don't pay a price when I don't need them, but can enable them by just deleting a single character.

      And Perl isn't the only language I use templates for. I also have them for C, awk, FORTRAN, and Pascal. Not to mention tons of (retired) templates for all kinds of projects.

      Perl --((8:>*
Re: Bundling commonly-used modules into a toolset
by hv (Parson) on Oct 25, 2005 at 11:26 UTC

    Looks interesting. One minor point: I most commonly use Class ();. Before I read the full docs, I guessed that ... Class => '' ... would give me that, by symmetry with the other available options:

    Toolset->export( Class => 'sub1 sub2', Class => '' Class => [qw/sub1 sub2/], Class => [], );

    I'd suggest making it so, and leaving undef as the only way to get use Class;.

    I think I'd rarely use this for applications, where cut and paste can happily replace repeated typing, and where there are benefits to having the modules and pragmas in use clear to see.

    But for command-line invocations I can definitely imagine using this - when testing my work application, I often need 100-200 characters on the CLI just to get all the modules and initialisation I need, before I get to the one method call I'm trying to test.


Re: Bundling commonly-used modules into a toolset
by demerphq (Chancellor) on Oct 25, 2005 at 15:05 UTC

    I really like this idea. I had a dream years ago of a 'caution' pragma. So you could say

    use caution;

    instead of

    use warnings; use strict;

    It seems like such a pragma could be implemented nicely using your idea. Kudos.


Re: Bundling commonly-used modules into a toolset
by radiantmatrix (Parson) on Oct 25, 2005 at 20:09 UTC

    Part of what you say is a good idea, but part seems somewhat... well, wasteful.

    I have a toolkit-type module that I import to give some commonly-used functions, like a sub that implements s/^\s+|\s+$//g;.

    But for stuff like use strict; use warnings; use Carp;, what's wrong with using a template (or macro, or whatnot) in your editor? It makes it much easier for future maintainers to see what's going on, rather than making them dig into another module just to find out that it repackages common pragmas and modules.

    Besides, my shortcut for such things ('ns'+Ctrl-Space) is much faster than even typing 'use My::Tools;'. I guess I don't see why you'd want common template-esque things packaged into a module.

    A collection of thoughts and links from the minds of geeks
    The Code that can be seen is not the true Code
    "In any sufficiently large group of people, most are idiots" - Kaa's Law
Re: Bundling commonly-used modules into a toolset
by creamygoodness (Curate) on Oct 25, 2005 at 19:04 UTC
    Delicious! This is what I had hoped Toolkit would be!

    I'd like to use this on a distro destined for CPAN which numbers 47 modules and counting. One of those modules is a base class for almost all the others, supplying new(), etc. A typical module begins:

    package Restaurant::People; use strict; use warnings; use base qw( Restaurant::Util::Class ); use Carp; use Scalar::Util qw( blessed dualvar );

    Now, say I have other classes which inherit from Restaurant::People: Customer, Chef, Server, Busser, Manager, etc. The ultimate in laziness would be this, if I turned Restaurant::Util::Class into a ToolSet-enabled superclass:

    package Restaurant::People::Cook; use base qw( Restaurant::People );

    However, it doesn't look like I can get away with that -- the code needs to look like this instead, correct?

    package Restaurant::People::Cook; use base qw( Restaurant::People ); use Restaurant::Util::ToolSet;

    That's still going to save a bunch of space. I always need strict and warnings, nearly always need Carp, and often need Scalar::Util, so I'll just throw 'em all in the ToolSet. Sweet!

    Regarding the Alpha status: I didn't see a warning anywhere in the docs. Is the interface stable? Without any explicit notice, I infer that distributions are in Beta if the version is less than 1.0.

    Marvin Humphrey
    Rectangular Research ―

      Cool. That's one of the applications for which I thought it might be useful. I don't think that use base will work because it doesn't call import. You could do it in two steps, though:

      package Restaurant::People::Cook; use Restaurant::People; # A ToolSet module our @ISA = qw( Restaurant::People );

      I suppose I could add a method to ToolSet that adds the ToolSet module to the caller's @ISA. Then you'd get the imports and set up a subclass.


      The other approach would be to explicitly set base modules, so it doesn't necessarily have to be the ToolSet module:


      Any preference?


      Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

        After some experimentation, I've concluded that it's a good thing that use base and use My::ToolSet require separate lines -- because of what other people will expect when they subclass.

        If someone writes this:

        package BunnyBurgers::PRFlack; use base qw( Restaurant::People );

        ... they won't be expecting to have have a whole bunch of extra functions pulled into their namespace, or changes in pragma behavior. Now it's true that all of a sudden all of the imports I brought in via Restaurant::Util::ToolSet to Restaurant::People are available as methods, so $pr_flack->can('dualvar'). But that happens anyway -- $nearly_any_object->can('carp') -- and at least subclassers don't have to worry about surprise namespace polution in their actual module code.

        For the time being, I've settled on starting all the modules like this:

        package Restaurant::People::Cook; use Restaurant::Util::Toolset; use base qw( Restaurant::People );

        There will only be one ToolSet in the entire distro, and it will bring in strict, warnings, Carp, Scalar::Util, and a global RESTAURANT_DEBUG constant. Restaurant::Util::Toolset class won't be public, so it will be possible to modify it, but I'm expecting to keep it stable. That should cause minimum confusion. The only people who need to know what's in Restaurant::Util::Toolset are people who are hacking/debugging modules which use it.

        So, to answer your question, I wouldn't really use either one of those. Multiple inheritance of ToolSet-enabled modules would just get too messy.

        I'm quite happy with what we've got now. By compressing several lines at the top of the file into one, a bunch of meaningful, frequently edited code now appears in the the first screenful, so when I type "gg" in vim, I'm right in the game.

        Marvin Humphrey
        Rectangular Research ―
Re: Bundling commonly-used modules into a toolset
by metaperl (Curate) on Oct 28, 2005 at 21:25 UTC
    Shouldn't both of these modules be in the Module::* top-level namespace? I.e, Module::Toolset, Module::Toolkit

      I was wondering if this would come up. My hope was that people would create and publish their toolsets underneath the ToolSet namespace similar to the way that people release Bundles today.


      Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://502654]
Approved by Corion
Front-paged by planetscape
[ambrus]: And even if this works, I'm still not sure you can't get double timeouts from a Timer.
[ambrus]: Corion: well Prima::Object says something like that the cleanup method will send an onDestory message and that you can't get more messages after cleanup, or something.
[Corion]: ambrus: Yeah - I don't think the deep source dive will be necessary if things are implemented as simple as they could be :)) And hopefully I won't need (more) timely object destruction. I can update the screen at 60Hz and hopefully even do HTTP ...
[Corion]: ... transfers in the background. Now that I think about it, this maybe even means that I can run the OpenGL filters on Youtube input :)
[ambrus]: Corion: I mentioned that the unix event loop of Prima always wakes up at least once every 0.2 seconds. Have you found out whether the win32 event loop of Prima does that too?
[Corion]: ambrus: Hmm - I would assume that the onDestroy message is sent from the destructor and doesn't go through the messageloop, but maybe it is sent when a window gets destroyed but all components are still alive...
[ambrus]: Corion: partly deep source dive, partly just conservative coding even if it adds an overhead.
[Corion]: ambrus: Hmm - no, I haven't looked at wakeup intervals ... I wonder why it should want to wakeup periodically because it gets a lot of messages from the Windows message loop (on Windows obviously)
[ambrus]: (Alternately a deep source dive and then rewrite that event loop to make it better, and then as a bonus you get an idle method.)
[ambrus]: The 0.2 seconds wakeup is likely a workaround for some bug, but I can't guess what bug that is.

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (6)
As of 2016-12-09 10:27 GMT
Find Nodes?
    Voting Booth?
    On a regular basis, I'm most likely to spy upon:

    Results (150 votes). Check out past polls.