Namespace and use confusion. "Undefined subroutine errors"

[Wonko the sane]

Recently I came across a problem with namespaces that I do not fully understand.
I dont know enough about namespaces though to understand exactly what is going on here.

I was under the assumption that the same module could be 'used' or required
multiple times, throughout an application and its modules, without a problem.
Even if this created a sort of 'use' loop; one module using another, that in turn uses the first one again.

AB_prod.pm => SYS_mail.pm => AB_prod.pm

I am finding though, that this kind of setup causes problems in some situations,
depending on the order that the modules use statements are listed in the App.

When modules are used in a certain order, I start getting runtime error messages like this one.

#--------------------------------------------------------------------
Undefined subroutine &AB_prod::send_mail called at
/usr/web/wonko/test/lib/AB/AB_prod.pm line 18.
#--------------------------------------------------------------------

Here is an simple test setup that I have made up to show what I am talking about....

In this hypothetical example, the modules used by the Application are
divided up into layers. Each set of layer modules is kept in its own directory.

/u/web/wonko/test/lib/AB/ -- Contains Applied Business rules.
/u/web/wonko/test/lib/SYS/ -- Contains System level functions.

A simple test script that uses an AB and a SYS layer function.

/usr/web/wonko/test/test1.pl
#--------------------------------------------------------------------
#!/usr/local/bin/perl -w
use strict;

use lib qq{/usr/web/wonko/test/lib/SYS};
use lib qq{/usr/web/wonko/test/lib/AB};

use SYS_mail qw( send_mail ); # Move after AB_prod & error goes away

use AB_prod qw( send_notice ); # uses SYS_mail.pm as well.


send_notice();

exit;
#--------------------------------------------------------------------

This module 'uses' the SYS_mail module, just as the test1.pl script does. This in itself does not seem to be a problem.

/usr/web/wonko/test/AB/AB_prod.pm
#--------------------------------------------------------------------
package AB_prod;

use lib qq{/usr/web/wonko/test/lib/SYS};

use SYS_mail qw ( send_mail );

use strict;
use vars qw(@ISA @EXPORT_OK);
use Exporter();
@ISA = qw(Exporter);
@EXPORT_OK = qw( send_notice );

sub send_notice
{
   print qq{Hello from AB_prod::send_notice\n};

   send_mail();
}
1;
#--------------------------------------------------------------------

Here is where the problem develops.
This module uses the'AB_prod.pm' module, which is already using the 'SYS_mail.pm'
'AB_prod' => 'SYS_mail' => 'AB_prod'. This causes a situation where
it looks like the namespace is being modified to where the AB_prod module
cannot find the SYS_mail module again, during runtime.
This situation does not generate any errors and passes fine under a
'perl -cw'. The error only surfaces, when the actual function(SYS_mail::send_mail)
is called at runtime.
It like its entry in the Symbol table got stomped or something.

/usr/web/wonko/test/lib/SYS/SYS_mail.pm
#--------------------------------------------------------------------
package SYS_mail;

use lib qq{/usr/web/wonko/test/lib/AB};

use AB_prod qw( send_notice );

use strict;
use vars qw(@ISA @EXPORT_OK);
use Exporter();
@ISA = qw(Exporter);
@EXPORT_OK = qw( send_mail );

sub send_mail
{
   print qq{Hello from SYS_mail::send_mail\n};
}
1;
#--------------------------------------------------------------------

When the test1.pl script is run, the following runtime error is generated.

#--------------------------------------------------------------------
:!./test1.pl
Hello from AB_prod::send_notice

Undefined subroutine &AB_prod::send_mail called at
/usr/web/wonko/test/lib/AB/AB_prod.pm line 18.
#--------------------------------------------------------------------

Notice that it appears to be looking for the send_mail function in
the wrong name space.

What is very strange, is that changing the order of the AB and SYS use
statements in the test1.pl script, DOES make the error message go away and
everything functions normally.

Is the Symbol table entry for this function actually getting squashed?

Is this type of setup for Modules just unacceptable?

Any help is much apprecited,
Wonko

[ikegami]

The simple answer is don't do that! Modules calling each other or calling themselves are usually a sign of bad design.


Putting aside the validity of your design, let's first find the cause of the problem. Keep in mind that

use Module LIST;

is the same as

require Module;
Module->import LIST if Module->can('import');

while we look, line by line, at the code Perl executes:

test1.pl:2     use strict;
test1.pl:4     use lib qq{/usr/web/wonko/test/lib/SYS};
test1.pl:5     use lib qq{/usr/web/wonko/test/lib/AB};
test1.pl:7     require SYS_mail;

   Start executing SYS_mail.pm

SYS_mail.pm:1  package SYS_mail;
SYS_mail.pm:3  use lib qq{/usr/web/wonko/test/lib/AB};
SYS_mail.pm:5  require AB_prod;

   Start executing AB_prod.pm

AB_prod.pm:1   package AB_prod;
AB_prod.pm:3   use lib qq{/usr/web/wonko/test/lib/SYS};
AB_prod.pm:5   require SYS_mail;

   Does nothing. Already required

AB_prod.pm:5   SYS_mail->import(qw ( send_mail ))
                  if SYS_mail->can('import');

   ERROR!!

The error is that we haven't yet executed

use vars qw(@ISA @EXPORT_OK);
use Exporter();
@ISA = qw(Exporter);
@EXPORT_OK = qw( send_notice );

in SYS_mail.pm, so SYS_mail has no import function, so send_mail doesn't get imported into AB_prod. Similar errors occur at later points in the program.


And now the fix:

To fix this problem (aside from the obvious one of not including yourself directly or indirectly), change your modules to the following:

/usr/web/wonko/test/AB/AB_prod.pm
#--------------------------------------------------------------------
package AB_prod;

use lib qq{/usr/web/wonko/test/lib/SYS};

use strict;
use warnings;

BEGIN {
   our @ISA = qw( Exporter );
   our @EXPORT_OK = qw( send_notice );
   require Exporter;
}

use SYS_mail qw ( send_mail );

sub send_notice
{
   print qq{Hello from AB_prod::send_notice\n};

   send_mail();
}
1;
#--------------------------------------------------------------------

/usr/web/wonko/test/lib/SYS/SYS_mail.pm
#--------------------------------------------------------------------
package SYS_mail;

use lib qq{/usr/web/wonko/test/lib/AB};

use strict;
use warnings;

BEGIN {
   our @ISA = qw( Exporter );
   our @EXPORT_OK = qw( send_mail );
   require Exporter;
}

use AB_prod qw( send_notice );

sub send_mail
{
   print qq{Hello from SYS_mail::send_mail\n};
}
1;
#--------------------------------------------------------------------


Tested:

use strict;
use warnings;

use SYS_mail qw( send_mail );
use AB_prod qw( send_notice );

send_notice();

__END__
output
======
Hello from AB_prod::send_notice
Hello from SYS_mail::send_mail

I also tested it with the two use statements inverted. It gave the same output.


Update: It doesn't matter that sub send_mail { ... } hasn't been reached at the point Exporter->import('send_mail') is called, so I removed the stub functions I added just below the BEGIN in SYS_mail. Same idea for AB_prod.

[Wonko the sane]

Ah! I understand now. Fantastic answer! Thank you very much.

[bart]

The simple answer is don't do that! Modules calling each other or calling themselves are usually a sign of bad design.

... and when that happens, I take it as a sign that you always want both of your source files always to be loaded together. So why put them in separate files? Simply put them together in the same module. Perl does not enforce packages being in the source file as indicated by the package name. It's best to have that package in the source too, as a sanity handle, and to avoid confusion, and otherwise import won't work either; but it's not absolutely required.

See (the source of) XML::Parser for example, where multiple packages are put into the same module — an example where they might just as well, or maybe even better, have used separate files.

[shemp]

This is because of how perl requires & uses. Consider this code:

package Foo;

use thing_to_use;

...

package Bar;

use thing_to_use;

...

When this code executes, in package Foo, thing_to_use is included, and the stuff in thing_to_use is put into the namespace Foo.
Then in package Bar, thing_to_use is specified to be used again, but since perl has already used thing_to_use, it skips over that use line. (otherwise circular references would keep parsing forever - among other issues).
So when you try to use something from thing_to_use in package Bar, it doesnt work right because the functions, etc in thing_to_use were put into the Foo namespace. If you fully qualified that in package Bar it would work in this instance, but not in general, because other programs might not import thing_to_use into package Foo, but some other package that asked for it first.

The solution would be something like this:

use thing_to_use;

package Foo;
...

And then in your code for Bar:

use Bar;

package Bar;
...

This way, thing_to_use is imported into main. But, you'll have to qualify calls into thing_to_use with main:: in both Foo and Bar.