Use and Require - making an informed choice.

[g0n]

This is a relatively simple issue, but one that I've never seen discussed, so I thought I'd bring it up.

My default position when I use external modules is:

#!/usr/bin/perl
use strict;
use warnings;
use <MODULE>;
use <MODULE>;
use <MODULE>;

(but then I have quite a lot of 'C-ish' habits). If I have a particular need to use 'require' then I will, but normally 'use' is the norm and 'require' the exception. The important distinction between 'use' and 'require' is that 'use' is called at compile time, and 'require' at run time (plus, 'use' calls the modules import method implicitly). A few days ago I came across a design & implementation issue that makes this distinction very important.

I have a module that contains a range of utility functions, which I use with varying frequency. Two of the rarely used functions rely on the ability of Data::Dump::Streamer to serialise coderefs, but I don't use them very often. Since 'require' is executed at run time, I can put the require Data::Dump::Streamer; line in the functions that need it, rather than a 'use' at the top of the module, and the code compiles and runs fine on my PDA without DDS installed, as long as I don't want to use the 'save' or 'load' functions that depend on DDS.

So, the factors for choosing between the two seem to me to be as follows:

• Do you always need the module to be imported (i.e. do you want to use the code for things that don't rely on the module)?
• Does the module need to be imported at compile time (i.e. does it rely on using BEGIN)?
• Do you want to ensure that the module is present for your script to work?

If the answers are 'no', then it seems to me it would be worth importing the module with 'require' where it's needed. The point of the third is that this only really applies to things like utility scripts & modules used across multiple machines by informed users (like yourself) - if you are writing a deployed application or an unattended script you will want the dependencies to be resolved.

Can anyone suggest other factors influencing the choice?

Update: There will also be a speed of execution hit at the time that the module is being imported, so I guess that's another factor. But then, restricting 'use' to modules you really need should reduce compile time, leaving the overhead of importing a module with 'require' to when you actually need it.

Update: Two other issues raised by zentara: require will accept a string instead of a namespace, so permits import of non ~.pm files, and also require can present unexpected collisions with global variables.

[zentara]

Passing a run time string to require

strict/warnings interference with global declarations

[merlyn]

My habits are as follows:
• mostly needed modules that have exports: put the "use" at the top of the file, and if I'm un-lazy, list the exports explictily in a qw() list for documentation and safety.
• mostly needed OO modules (no exports): ditto, listing an "empty list" () for the import list.
• rarely needed OO modules: I put a require in the code path just before access, so that the module loaded lazily.
• rarely needed modules that have exports: this is the tough category. I either punt and put it at the top, or put a require in the code path and live with unimported names (full package spelled out). Or, I rely on autouse, but that can get tricky for prototypes.

[martin]

The speed argument cuts both ways: Loading modules just in time can cause unexpected delays long after the start of the program.

You also seem to focus on "the module's" need (suggesting requisites for the code to run) rather than more general quality aspects (like maintainability benefits through isolation of dependencies).

To summarize, I agree with your habits but would make the necessity of conditional loading the distinctive question. And even if conditional loading is called for, like in your PDA example, I would try hard not to sprinkle "requires" all over my code but isolate that again, perhaps in a meta module.

One exception to this is that I have sometimes followed suit with module authors not using "Carp" up front. It might be time to reconsider that, however.

[g0n]

The meta module is an interesting idea - some mechanism for loading & keeping a list of potentially required modules. Perhaps something like this:

package Foo;
use strict;
use warnings;

my $foo = Foo->new;

$foo->rarelyusedfunction;

Foo::CodeLoader->new->list();



sub new
{
        return bless \{},shift;
}

sub commonlyusedfunction
{
        # do stuff
}

sub rarelyusedfunction
{
        my $self = shift;
        my $cl = Foo::CodeLoader->new();

        $cl->load('DDS');

        my $dds = Data::Dump::Streamer->new;
        $dds->Dump($self)->Out();

}

package Foo::CodeLoader;

sub new
{
        my $class = shift;
        my %modules = (      DDS        =>      'Data::Dump::Streamer'
+    );
        return bless \%modules,$class;
}

sub load
{
        my $self = shift;
        my $param = shift;
        eval "require $self->{$param}";
}

sub list
{
        my $self = shift;
        for (values %{$self})
        {
                print "$_\n";
        }
}


1;

Although I'm always slightly uncomfortable with eval require

[merlyn]

You seem to be reinventing autouse, which has been in the distro since 5.6, if I recall.

[xdg]

The meta module is an interesting idea - some mechanism for loading & keeping a list of potentially required modules.

That kind of thing -- lazy importing/on-demand functions -- is on my TODO list for ToolSet. If there's a lot of demand for it, I'd be willing to bump it up on my "get around to it" list. I was thinking of exporting a stub function that, if called, would require the real module if it hadn't been loaded already and from then on just directly goto the real function whenever called.

[demerphq]

A minor comment on the call to DDS:

sub rarelyusedfunction
{
        my $self = shift;
        my $cl = Foo::CodeLoader->new();

        $cl->load('DDS');

        my $dds = Data::Dump::Streamer->new;
        $dds->Dump($self)->Out();

}

I would write that as follows:

sub rarelyusedfunction
{
        my $self = shift;
        if ( eval { require Data::Dump::Streamer; 1 } ) {
           return scalar Data::Dump::Streamer::Dump($self)->Out();
        } else {
           warn "**DANGER** Falling back to Data::Dumper as DDS"
                . " isn't available";
           require Data::Dumper;
           return Data::Dumper::Dumper($self)
        }

}

The scalar there is useful, as Out() behaves differently in list and scalar context (something that was probably a mistake in hindsight and which might change in the next release, I've been putting serious consideration into changing the interface somewhat in version 2).

I've never bothered with any Autouse stuff personally. The pattern

if ( eval { require Foo; 1 } ) { .. } else { ... }

is simple enough I think.

[xdg]

Is not having it fatal for core operation or can you work around not having it? If not having it is fatal, then you might as well use it so your module fails immediately. If you can work around it then eval the require so you can manage the case where it's missing.

Also, I would tend to think that almost anything that imports subroutines or other symbols really needs to happen at compile time. There are some exceptions, but those tend to be the special cases.

I think newer versions of Carp are lightweight and require the heavy stuff only if invoked. Generally, that pattern may make sense -- require modules at runtime that you only need in a fatal exception (e.g. a dumper). Once the exception is triggered, the loading time of the module at at that point is pretty irrelevant.

I see it kind of like this (not a complete set of permutations, but a set of guidelines):

• Always used in any execution of the program -- use
• Sometimes used, but has import side effects (e.g. importing functions) -- use
• Sometimes used, import not required, but fatal if not present when needed -- use
• Sometimes used, import not required, can work around it not being available -- require (in an eval)
• Sometimes used, specified at runtime (e.g. by a config file) -- require
• Rarely used, fatal if not present, but module is "big" relative to rest of the code -- require or use (personal preference)
• Rarely used, only in fatal error handling -- require

[tye]

I hardly ever use a module if I'm not wanting to import() from it. Using require at the top of a script or module is not the same as using require from inside some rarely used function. If I use require at the top of a script or module, then the lack of the module will still cause my script to fail "immediately" and before the script "runs", so I see no advantage to avoiding require in favor or always useing.

I prefer to require modules when it suffices because it makes it clear that the module is not importing anything and is not making compile-time changes that impact the compilation of the rest of the code.

I also rarely use a module without giving a specific list of arguments to pass to its import() function. Implicit imports are the enemy of clarity.

[radiantmatrix]

I think it is important to make an informed choice about which of use and require are needed, precicely because which you choose (and where you place requires) has an impact on the percieved performance of your application.

My goal is generally to use modules that will be called early and commonly, but require those called only occasionally. By "commonly" and "occasionally" I mean the answer to the question "in any given run of my app, how likely is it that this particular thing will be called?".

I do this by using require for needed modules at the top of subs that are only occasionally called. As you can imagine, this often gets hard to track. So, as I develop I use modules in each sub, taking advantage of the fact that this will always cause compile-time loads, and won't double-load modules. When I am ready to release a version, I replace certain uses with requires (and calles to import when appropriate), and others I comment out and move to the top of the script. (In fact, I have a commented list of all the requires at the top as well, with an explanation, to help maintainers.)

Now that I've been exposed to autouse, though, I might just autouse modules I'd previously have required, and save the old require for times when I want to avoid calling an import.

[Anonymous Monk]

Does the module need to be imported at compile time (i.e. does it rely on using BEGIN)?

Remove "(i.e. does it rely on using BEGIN)" as it isn't a factor.

[xdg]

The fact that a use creates a BEGIN block can matter if order of compilation is important. That's substantially different from a runtime require and import without the BEGIN. For example: importing a function that overrides a built-in. If requiring and installing a subroutine at runtime, the compiler will already have resolved calls to the built-in functions at compile time before the replacement functions are installed.

[ambrus]

In the newer version of the egrep clone with function name display (a snapshot can be downloaded from http://www.math.bme.hu/~ambrus/pu/cgrep) I require the File::Find module only if a recursive grep is requested.

I just did a quick mesurement to see how much time this conserves. I ran a grepping command on a single small file that runs very fast. When I didn't use recursive grepping, the command ran in 0.49 seconds. When I turned on recursive grepping (which doesn't make much sense as I haven't given any directories only a single file) it ran in 0.78 seconds. As a comparision, the grep program (which is written in C) always did the same search in 0.00 seconds, whether recursive or not.