Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

Exported subroutine redefine

by ribasushi (Pilgrim)
on Nov 10, 2007 at 11:41 UTC ( [id://650045] : perlquestion . print w/replies, xml ) Need Help??

ribasushi has asked for the wisdom of the Perl Monks concerning the following question:

Hello fellow monks,

I have a situation in which I have a 3rd party module which exports some subroutines which I need to redefine (lets not delve into a discussion of why this is bad, etc.). Here is an example:
package module; use warnings; use strict; use base qw/Exporter/; our @EXPORT = qw/function/; sub function { print "old\n"; } 1;
package redefine; use warnings; use strict; no warnings qw/redefine/; use module; sub module::function { print "new\n"; } 1;

Now unless 'use redefine' preceeds _any_ 'use module' - everything works as expected. However if I do this:
use module; use redefine; function();
I still get 'old', because the symbol table is still referencing the first version of function().

Is there a way to truly redefine a subroutine, or the only way to do this right is to ensure is the very first one use()d in any circumstances?


Replies are listed 'Best First'.
Re: Exported subroutine redefine
by Joost (Canon) on Nov 10, 2007 at 12:42 UTC
    Probably the cleanest way to achieve this, is to have in Redefine:
    package Redefine; use Module qw(name all not redefined functions here); use base 'Exporter'; our @EXPORT = qw(name all exported functions including those from Modu +le here); sub redefined_function { ... }
    And then replace all 'use Module' statements with 'use Redefine'. Which is at least fairly easy to do and check automatically for most cases, and you won't have to worry about re-re-defining functions, since you're not re-defining functions anymore.

Re: Exported subroutine redefine
by dragonchild (Archbishop) on Nov 10, 2007 at 17:59 UTC
    No-one's actually told you what the problem is. In, you redefine module::function. But, when you call function(), you're calling the exported version. Exporter does a glob assignment which is different from a reference. When you assign to the RHS of a glob assignment, the connection is broken.

    The proper solution is for redefine to be a facade over module. Bring all of module into redefine, then redefine what you want to change, then everywhere you had "use module", change that to "use redefine".

    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
      Thanks for replying, this is what I figured is happening. What you are suggesting is hard to do, see my other reply
Re: Exported subroutine redefine
by shmem (Chancellor) on Nov 10, 2007 at 15:14 UTC
    Redefine the caller's functions imported from elsewhere in a import() sub:
    package redefine; use warnings; use strict; my @redefines = qw( module::function foo::bar ); sub import { my $class = shift; my $caller = caller; no strict 'refs'; no warnings qw/redefine/; for my $sym (@redefines) { "unqualify" subs (my $sub = $sym) =~ s/.*:://; if ( ${$caller.'::'}{$sub} && *{$caller.'::'.$sub}{CODE} eq *$sym{CODE} ) { *{$caller.'::'.$sub} = \&$sub; } *{$sym} = \&$sub; } } sub function { package module; print "new\n"; }; sub bar { package foo; print "in bar\n"; } 1;

    No need to 'use module' in your redefining package (if you 'use redefine' after 'use module', that is). Adding a package declaration to the new functions gives them access to that packages' symbol table (for 'our' vars, other functions etc.)

    update: made code more "general-purpose"

    update 2: added package declarations to subs


    _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                  /\_¯/(q    /
    ----------------------------  \__(m.====·.(_("always off the crowd"))."·
    ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
Re: Exported subroutine redefine
by Somni (Friar) on Nov 10, 2007 at 12:35 UTC
    Short of reaching into the package that imported the subroutine, I'm not seeing a way.

    You could crawl all over the symbol table finding any references to the old Foo::foo and update them. You're already doing something horribly icky, what's a bit more eh?

    Edit: Hah, I meant module::function, sorry. Foo::foo was the name of the function I was using to test things.

      Thanks for answering, this sounds like a good idea. Can you tell me how can I get a list of all currently populated namespaces? %INC contains only namespaces tied to a physical .pm file, and if a file has several package X pragmas I will examine only the first one.

      Btw if you are curious the ickyness comes form the fact that I am fooling around with exception handling, and I had this nice idea of globally redefining Carp::croak() and Carp::confess() to throw an object except of a string (I know I know it is far from best practice to fool around with core modules, but I really liked the idea, an wanted to see if it can actually work). Since Carp is a very central module it is being used all over the place in many other modules, and hence it is
      1) impractical to change all calls to it Carp::x()
      2) impractical to attempt to load my exception handler before any references to Carp.
      The two subroutines that must be redefined are trivial one-liners, thus I figured it should do no harm:
      sub croak { die shortmess @_ } sub confess { die longmess @_ }

      Addition: Why I don't simply overload CORE::die? Because for one it can be overloaded by someone else, and more importantly module authors sometimes examine $@ and finding an object there has a very remote chance of breaking stuff (although I have backwards compatible stringification of '<error> at <file> line <ln>.' On the other hand croak is clearly designed to be seen by the user, who in this case will be expecting an object.
        In the case of Carp specifically there is a far easier way. Use Carp::Heavy. Then redefine Carp::longmess_heavy and Carp::shortmess_heavy.

        Otherwise you'd have to walk the symbol table. Just to get you started on that, print out the keys of %main::. All of the keys ending in :: are subtables you can look for. Beware of infinite loops. (For instance %main::main::main:: is the same as %main::.)

        Strange, why aren't you just setting up your own $SIG{__DIE__} handler? It seems tailor-made to solve your problem for you. Granted, it's a hack, and can get messy, but it's a far better hack than trying to overload another module's subroutines globally.

        A long time back I went down this road, as well. I thought it was a great idea to setup a __DIE__ handler and catch all the errors, turning them into pretty output for a web framework I was designing. I suppose it worked, sorta. It's much cleaner to design an actual exception model, with exception classes thrown by code, and possibly a catch routine that can transform string exceptions into a relevant object.

        If the object in $@ has stringification it shouldn't break anything that's attempting to match against it. The only thing it should "break" is something that's checking to see if it's an object, in which case that code should know what it's doing.

        Could you just redefine Carp::shortmess and Carp::longmess? If not, you can walk the symbol table starting with something like this:
        for (grep /(?<!main)::/, keys %::) { # do stuff... }
Re: Exported subroutine redefine
by Krambambuli (Curate) on Nov 10, 2007 at 12:46 UTC
    I'm not an expert on it, so please don't take my comments other than as comments.

    I think there is a problem with how you mix @EXPORT with the other code that should do the redefine, as it makes your 'old' module::function to become main::function.

    main is the third player in the game, that you seem to ignore. If you use the test as
    use module; use redefine; module::function();
    it will work as wanted, regardless in which order you 'use' the moduls. You might want eighter to not export, or maybe to redefine main::function instead of module::function.

    enjoying Mark Jason Dominus' Higher-Order Perl
Re: Exported subroutine redefine
by ikegami (Patriarch) on Nov 10, 2007 at 17:06 UTC

    You could use a wrapper around the main script

    #!/usr/bin/perl use redefine; use File::Basename qw( dirname ); use File::Spec::Functions qw( rel2abs catfile ); do(catfile(dirname(rel2abs($0)), ''));

    Or use -M

    perl -Mredefine
Re: Exported subroutine redefine
by brian_d_foy (Abbot) on Nov 11, 2007 at 10:27 UTC

    I wrote an all of chapter 10 for Mastering Perl about how to do this (and chapter 4 deals with some fo that Carp stuff tilly mentioned).

    I didn't cover the @EXPORT case, though, so maybe I should add that. As others pointed out, you have to export to the calling namespace to overwrite the functions something else exported. The problem is that now you have to use those modules in the right order or you'll get the wrong function (anyone remember the and modules fighting it out to see whose url() would be the final one defined?).

    See if you can get yourself to the point where you're only using redefine in the code so it loads the module it wants to override, does its work, then expors the final versions of everything. You won't have someone come along and list all of the modules in alphabetical order (yeah, some of us are a bit OCD like that) and screw up everything. :)

    brian d foy <>
    Subscribe to The Perl Review
      I just ended up walking the entire symbol table, which turned out to be a trivial subroutine. Please let me know if I am doing something horribly wrong (the justification can be found in the big picture)
      _redef(); sub _redef { my $parent = shift || '::'; for my $ns (grep /^\w+::/, keys %{$parent}) { $ns = $parent . $ns; _redef($ns) unless $ns eq '::main::'; for my $sub (keys %redef) { *{$ns . $sub} = $redef{$sub} if (exists ${$ns}{$sub}); } } }
        This is really handy (and absolute evil of course). I find that it's a good idea to also check that the function you're replaceing not only has the same name, but is truely a pointer to the same function. Just a matter of comparing that \&{${$ns}{$sub}} == $origsub , where $origsub is a pointer to the function you want to replace.
Re: Exported subroutine redefine
by ribasushi (Pilgrim) on Nov 11, 2007 at 12:17 UTC
    Thank you all who responded with constructive suggestions. Apart from learning a great deal about perl internals, I like the result so much that I figured I'll share it with you. Criticism always welcome!
    The goal was to seamlessly integrate OO exception handling with minimal effort from the user, and make the object smart enough to preserve as much information as possible along the way.

    What this module can do:
    • Exports a subroutine die() which throws an objectified exception.
    • Performs minimal changes to by redefining croak() and confess(), making them throw objects. The nature of croak() is preserved, and it still reports errors from the callers perspective. A symbol table walk is performed in order to find all references to croak() and confess(), in case this module is loaded after some other modules used Carp.
    • Defines a UNIVERSAL::die method, which throws an object containing a reference to the calling object.
    • The object is implemented as an array representing the exception stack. Thus the following:
      eval { eval { Module::stuff(...); # this throws an exception } croak ($@, 'More info'); } die;
      will resut in $@ containing an object with 3 frames, each containing all information pertaining to the specific exception.
    • The exception object has proper stringification, based on information found in the first (deepest) frame, and formatted accordingly to the exception type (die, croak, confess, etc.) A stringification of a more recent frame can be obtained by $@->stringify ($frameno)
    • Every exception frame contains a full trace produced by Carp::longmess, and exception objects have a crude ->dump() method, allowing to examine their internal state.

    The Module
    # package name can be anything, __PACKAGE__ is used throughout the mod +ule # subclassing is impractical and not implemented package PRD::Error; use warnings; use strict; use Carp qw//; use Data::Dumper; use base qw/Exporter/; # probably should write my own import() our @EXPORT = qw/die/; # and have :objectify as a flag to objectify C +arp use overload ( q/""/ => \&stringify, fallback => 1, ); # this die() will be imported into any package that uses us sub die (@) { # find the first caller outside of this package my $fr = 0; while (caller($fr) and __PACKAGE__ eq caller($fr)) { $fr++; } my @caller = caller($fr); my $eframe = { file => $caller[1], line => $caller[2], caller => 'die', }; my @err; if (@_) { #check for a pseudo-object created by UNIVERSAL::die #(recognized by being a hash instead of an array) if (ref $_[0] eq __PACKAGE__ and UNIVERSAL::isa ($_[0], 'HASH' +) ) { my $pseudo = shift; for (keys %$pseudo) { if ($_ eq 'error') { @err = @{$pseudo->{$_}}; } else { $eframe->{$_} = $pseudo->{$_}; } } } #check if we are called as a class method (package->die ()) elsif ($caller[0] eq $_[0] and (@_ > 1 or $@) ) { $eframe->{class} = shift; $eframe->{caller} = 'class'; } } # either remaining @_ or $@ or nothing unless (@err) { @err = @_ ? @_ : ($@ || () ); } my $eobj = []; # check if this is a re-thrown error object if (ref $err[0] eq __PACKAGE__) { $eobj = shift @err; } $eframe->{trace} ||= _trace (@err); if (@err) { $eframe->{error} = \@err; } push @$eobj, $eframe; CORE::die bless ($eobj, __PACKAGE__); } # teach objects how to die sub UNIVERSAL::die { my $obj = shift; # native object if (ref $obj eq __PACKAGE__) { &die ($obj, @_); } # foreign object else { my $pseudo = { caller => 'object', object => $obj, trace => _trace (@_), }; $pseudo->{error} = [ @_ ] if @_; &die (bless $pseudo, __PACKAGE__); } } sub stringify { my $self = shift; my $fr = shift || 0; # no stringification when called by _trace() return $self if (caller(1) and (caller(1))[3] eq __PACKAGE__ . ':: +_trace'); unless ($self->[$fr]) { CORE::die Carp::shortmess ( sprintf ( "Frame index '%s' requested from object with %d frames", $fr, scalar @$self, )); } my $err = ( $self->[$fr]{error} ) ? join '; ', @{$self->[$fr]{error}} : ''; if ($err !~ /\n$/ or grep { $self->[$fr]{caller} eq $_ } qw/confess croak/ ) { $err .= " at $self->[$fr]{file} line $self->[$fr]{line}.\n"; } if ($self->[$fr]{caller} eq 'confess') { $err .= join "\n", ( map { "\t$_" } (splice @{$self->[$fr]{trace}}, 1), '', ); } return $err; } # dumper shortcut sub dump { my $self = shift; return Dumper [ @$self ]; } # objectify exceptions globally { no warnings qw/redefine/; no strict qw/refs/; my %redef = ( croak => sub { my $pseudo = { error => [ @_ ], caller => 'croak', trace => _trace (@_), }; { local $Carp::CarpLevel = 1; ($pseudo->{file}, $pseudo->{line}) = Carp::shortmess ('') =~ /^ \s at \s (.+) \s line \ +s (\d+)/x; } &die (bless $pseudo, __PACKAGE__); }, confess => sub { my $pseudo = { error => [ @_ ], caller => 'confess', trace => _trace (@_), }; &die (bless $pseudo, __PACKAGE__); }, ); _redef(); sub _redef { my $parent = shift || '::'; for my $ns (grep /^\w+::/, keys %{$parent}) { $ns = $parent . $ns; _redef($ns) unless $ns eq '::main::'; for my $sub (keys %redef) { *{$ns . $sub} = $redef{$sub} if (exists ${$ns}{$sub}); } } } } # separate Carp::longmess into logical lines sub _trace { local $Carp::CarpLevel = 1; my @trace; my $mess = Carp::longmess (join '; ', @_); while ($mess =~ /\s* (.+? \s at \s [^\n]+? \s line \s \d+) \s*\n/x +mgs) { push @trace, $1; } return \@trace; } 1;
    Update: Fix stringification problems.
    Update2: Fix the fix :)
      Nice, but... now seeing the big picture and noting that it's not a general "global subroutine override" problem, but just about Carp - what about writing your own version of and including its path in PERL5LIB? Seems much easier and cleaner to me.


      _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                    /\_¯/(q    /
      ----------------------------  \__(m.====·.(_("always off the crowd"))."·
      ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
        Funny but I never thought of that :) So basically bring all the functionality described above plus the subroutines from, call the resulting module and load it from perl5lib while still relying on the original Carp::Heavy. Neat... Well at least I got pretty comfortable with the symbol table along the way :)
Re: Exported subroutine redefine
by TOD (Friar) on Nov 10, 2007 at 12:03 UTC
    make redefine inherit from module, would be my advice.
    masses are the opiate for religion.

      I'm not sure how that would solve the problem. Can you elaborate?

Re: Exported subroutine redefine
by syphilis (Archbishop) on Nov 11, 2007 at 09:00 UTC
    I'm not sure that it satisfies all of your conditions, but the following (using the fully qualified function name) produces the desired output:
    use module; use redefine; module::function();