Small examples of string eval

[spurperl]

Dear monks,

I need to enlighten some newbies re the usage of string eval. I want to show relatively small and simple examples in which string eval saves the day, beating the other approaches. That is, the examples have to be real-life, something that shows "live" how the eval is useful, not toy examples.

It is preferable to see code generation examples (I suspect that most real useful uses of string eval are for on-the-fly code generation).

Your help will be most appreciated. Thanks in advance

[gellyfish]

This is the only real example I could find in my own code, essentially it is a method on a base collection class to cause the collection to appear to be sorted on an arbitrary attribute (or attributes) of the objects in the collection. It uses eval to dynamically create a sort sub from a list of comparison specifications passed as an argument. Okay it doesn't fit the the 'small' criteria but I think it is illustrative of the kind of use string eval is good for:

sub sort
{
   my ( $self, $sort_spec ) = @_;
 
   my $sort_done = 0;
 
   if ( defined $sort_spec )
   {
 
     my @sort_objs;
 
     foreach my $obj ( @{$self->{_data}} )
     {
       my $sv = [];
       push @{$sv}, $obj->factory_index();
 
       foreach my $spec ( @{$sort_spec} )
       {
         my $meth = $spec->{Key};
         push @{$sv}, $obj->$meth();
       }
 
      push @sort_objs, $sv;
 
     }
 
 
     my @comparisons = ();
 
     my $index = 0;
 
     foreach my $spec ( @{$sort_spec} )
     {
       $index++;
       my $comp = '';
       my ($left_arg, $right_arg) = ('$a','$b');
 
       if ( exists $spec->{Direction} and $spec->{Direction} eq '-' )
       {
          ( $left_arg, $right_arg) = ('$b', '$a');
       }
 
       if ( $spec->{Compare} eq 'cmp' || $spec->{Compare} eq '<=>' )
       {
          $comp = "${left_arg}->[$index] $spec->{Compare} ${right_arg}
+->[$index]";
       }
       else
       {
         $comp = "$spec->{Compare}(${left_arg}->[$index],${right_arg}-
+>[$index])";
       }
 
       push @comparisons, $comp;
 
     }
      
     if ( $index )
     {
       my $sub = join ' || ' , @comparisons;
 
       $sub = "sub { $sub  };";
 
       *sortsub = eval $sub;
 
       $self->{_sort_index} = [];
 
       @{$self->{_sort_index}} = map { $_->[0] }
                                 sort sortsub  @sort_objs;
       
       $sort_done = 1;
 
       my $index = 0;
 
       foreach my $obj ( $self->list() )
       {
          $obj->factory_index($index++);
       }
 
     }
   }
 
  return $self->{_sorted} = $sort_done;
 
}

/J\

[brian_d_foy]

You don't need a eval to do this, and it's easier to do without it. You basically want a higher order function that composes other functions, and all of the functions are going to get the same arguments.

Starting from the finished product, you might have a sort subroutine like this, where the return value is the first thing that returns true (which should be -1 or 1).

sub sortsub {
    $a cmp $b ||
    other_function( $a, $b ) ||
    ...
    }

That's the same as writing it as a collection of subroutines that get arguments instead of using globals though:

sub sortsub {
    string_compare( $a, $b ) ||
    other_function( $a, $b ) ||
    ...
    }

But I can rewrite that sortsub to take a list of subroutine references as its arguments. Now my sortsub goes through each subroutine and returns when it finds one that returns a true value. If it finds 0 (meaning that sub thought the elements were equal), it goes onto the next subroutine.

sub sortsub {
    my @subs = @_;

    foreach my $sub ( @subs )
        {
        my $result = $sub->($a, $b);
        return $result if $result;
        }
    }

I can't use this in sort yet because I can't pass it arguments. I can, however, modify it to return a subroutine reference I can use:

sub make_sort_sub {
    my @subs = @_;

    return sub {
        foreach my $sub ( @subs )
            {
            my $result = $sub->($a, $b);
            return $result if $result;
            }
        };
    }

Now, I basically make the reference then use it in my sort block.

my $sort_sub = make_sort_sub( @sub_references );

my @sorted = sort { $sort_sub->() } @stuff;

And here's a full demo, using it with numbers, then strings, then a user-defined sort subroutine. I don't need an eval for any of it.

#!/usr/bin/perl
# pre-defined common sort subroutines
my %Subs = (
    numeric_descending => sub { $b <=> $a },
    numeric_ascending  => sub { $a <=> $b },
    string_descending  => sub { $b cmp $a },
    string_ascending   => sub { $b cmp $a },
    case_insensitive   => sub { "\L$a" cmp "\L$b" },
    );


sub make_sort_sub {
    my @subs = @_;

    return sub {
        foreach my $sub ( @subs )
            {
            my $result = $sub->($a, $b);
            return $result if $result;
            }
        };
    }
    
# numbers
{
my @use_these_subs = map { $Subs{$_} } qw(numeric_descending);
my $sort_sub = make_sort_sub( @use_these_subs );

my @sorted = 
    sort { $sort_sub->() } qw( 1 10 11 100 2 12 21 3 31 300 4 5 6 66 7
+ 71 );
print "@sorted\n";
}

# strings
{
my @use_these_subs = map { $Subs{$_} } qw(case_insensitive string_asce
+nding);
my $sort_sub = make_sort_sub( @use_these_subs );

my @sorted = 
    sort { $sort_sub->() } qw( Fred fred FReD Barney barney Betty BETT
+Y );
print "@sorted\n";
}

# strings by length with user defined subroutine
{
my @use_these_subs = ( sub { length $a <=> length $b } ); 
push @use_these_subs, map { $Subs{$_} } qw(string_ascending);
my $sort_sub = make_sort_sub( @use_these_subs );

my @sorted = 
    sort { $sort_sub->() } qw( Fred fred FReD Barney barney Betty BETT
+Y );
print "@sorted\n";
}

[spurperl]

Thank you, this is what I was looking for. I have a similar experience with string eval, generating subroutines that access certain data fields, using a configuration file. This was done to improve performance, as the generated subroutines were fine-tailored to the data extraction task at hand (with completely non-generic statements like substr($str, 2, 13)).

Unlike what many think, although the string eval is dangerous, it is necessary to efficiently and generically implement certain things that are impossible without it. The terrific "Higher Order Perl" has a lot of examples for smart usage of eval, and I'll look there as well.

[graff]

Teaching newbies to use eval? What a concept... (like playing with knives) -- well, it's sure to be valuable as a learning experience.

The one thing where eval is basically unavoidable is the use of a string variable within the "tr///" operator. It's simple, basic, and has a bunch of obvious uses in "real-life".

(update: forgot to include an example...)

my %charmap = ( a => 'B', c => 'D', ) # ... could come from a config f
+ile

my $old = join '', keys %charmap;
my $new = join '', values %charmap;

while (<>) {
    eval "tr{$old}{$new}";
    if ( $@ ) {
        die "tr{$old}{$new} failed at line $.: $_";
    }
    print;
}

[ikegami]

In my opinion, eval EXPR (as opposed to eval BLOCK) should be used only as a last resort. Your code is a prime example. It breaks as soon as someone places -, } or \ in %charmap. The following does not, and avoids eval EXPR entirely:

while (<>) {
    s/(.)/exists $charmap{$1} ? $charmap{$1} : $1/seg;
    print;
}

(No, the /e is not the same as eval EXPR. There's no dynamic code generation. /ee, on the other hand, would be the same as eval EXPR.)

[spurperl]

By newbies I don't mean people who've learned to program Pascal yesterday, but rather programmers from other languages who don't yet grok the concepts of runtime evaluation and code generation.

Thanks for the example, but it's not what I'm looking for. Your example is very idiomatic to Perl, I'd say it's a patch on a deficiency, because I have no trouble imagining tr/// implemented in such a way that allows the trick without eval.

What I'm looking for is more in the code-generation league. Generate some code with eval, on the fly, and call it when necessary.

[ikegami]

Major code generation is usually used when a language is inept (or not particularly adept) at doing something. Off the top of my head, I can think of three reasons why a language would be inept at doing something

• 1) A language could be inept at doing something because it lacks the specialization that would make it adept. Querying databases (facilatitated by SQL), generating parsers (facilitated by BNF or similar) and generating object accessors would fall in this category.

• 2) A language could be inept at doing something because it lacks the generalization that would make it adept. For example, working with Java Bytecode is more tedious than working in Java, so we do the work in Java and generate the Java Bytecode.

• 3) A language could be inept at doing something because it is primarily text based and/or non-interactive, and a GUI and/or interactive solution would be more adept. For example, GUI builders are often used to build GUI code.

[BrowserUk]

I have several scripts that take numeric arguments from the command line where is is extremely useful to be able to enter expressions instead of simple values.

#! perl -slw
use strict;

our $LIMIT = eval $LIMIT || 'somedefault';

This way I can specify

script -LIMIT=2**33

or

script -LIMIT=8*1024**3

Which beats the crap out of trying to remember

script -LIMIT=8589934592

[brian_d_foy]

You don't need eval for that, even if it makes it easier than parsing the argument and figuring out the value. The downside is that anyone can run arbitrary code using your script. With a tiny bit of programming you can do the math yourself and save yourself the security problem. :)

[BrowserUk]

anyone can run arbitrary code using your script ...

Apart from the fact that only I run the scripts in question, those same bad people you are parnoid about could also type perl -e"the same arbitrary code"

With a tiny bit of programming ...

Show me the code.

[UnderMine]

An example you use every day an never know about it. The dynamic loading of DBD drivers from the DBI source code.

    # --- load the code
    my $driver_class = "DBD::$driver";
    eval qq{package                     # hide from PAUSE
                DBI::_firesafe;         # just in case
            require $driver_class;      # load the driver
    };

UnderMine

[eyepopslikeamosquito]

Most Perl experts discourage the use of string eval. For example:

• merlyn in •Re^2: Dynamically constructed function calls states: Do not resort to eval-string if other means are available. You're firing up the compiler (slower than almost any other solution), and exposing yourself to hard to debug and hard to secure practices.
• TheDamian in Perl Best Practices in Chapter 8 has a guideline titled: Avoid string eval. Damian argues that string eval can be slow; produces run time warnings rather than more desirable compile time warnings; and code that generates other code tends to be harder to maintain. He further advises you to use anonymous subroutines and the CPAN Sub::Installer module when you have a need to create new subroutines that are built around some expression that the user supplies.

Update: In this quiz of the week MJD states:

A good rule of thumb is that unless what you're trying to do is most clearly described as "compile and run arbitrary Perl code", it's probably a mistake to use 'eval' to do it.

[BrowserUk]

But each of them knows when to eshew complicated eval-avoidance-schemes, in favour of simplicity.

merlyn See name_to_page() in CGI::Prototype::Hidden, new() in CGI::Prototype::Mecha, type() in File::Finder, the grammer for Inline::Spew.

TheDamian See import() in Attribute::Handlers, import() in Attribute::Types, install_dispatch() in Class::MultiMethods, initialise() & new() in Class::Std... I stopped here with D-Z to go.

Dominus memoize() in Memoize, in Memoize::AnyDBM_File, fill_in() inText::Template, TIEARRAY() in Tie::File.

And before anyone leaps to these authors defense, there is no need. String eval is a tool. Avoiding it where practical is common sense; eshewing at all costs is illogical.

I'm betting that Perl6 will retain string eval in some form.

[spurperl]

I was just going to reply that MJD's HOP also has a few legitimate uses of string eval. As you said it's a tool, a powerful tool, and as such it should be used with great caution.

I really hope Perl 6 won't drop the string eval, having it IMHO is essential to any dynamic language.

[stvn]

He (TheDamian) further advises you to use anonymous subroutines and the CPAN Sub::Installer module when you have a need to create new subroutines that are built around some expression that the user supplies.

While Sub::Installer is certainly a good idea (insulating the user from the evils of symbol table manipulation, which is at least as dangerous as string eval in the 'wrong' hands). It's code can hardly be considered to contain "Best Practices" IMO. It clearly abuses UNIVERSAL in a way which affects not only the code which uses it, but all other running Perl code. This is the offending code:

package UNIVERSAL;
use base 'Sub::Installer';

This would mean that every Perl object will respond true to the query $object->isa('Sub::Installer'). Again, IMO, that is not a Best Practice, but instead a really really bad idea.

Sorry, I am a little passionate about code which tramples over the Perl 5 object model like this.

But fear not! CPAN will save the day. As an alternative, I offer Sub::Install an excellent module which does not assert it's will upon your entire runtime.

[tilly]

See Class::FlyweightWrapper for an example. And advantage over alternate methods of doing the same thing is that in stack backtraces the proxying code will show up with useful package names.

[ruzam]

I have two places where I use eval all the time. I think eval is the best choice in these situations, maybe someone has a better alternative (that doesn't require non-core modules).

1) retrieving stored hashes
I regularily used Data::Dumper to turn hashes into strings so they can be stored in a database for later retrieval. The string accurately represents the original hash with out my having to do a thing and it's safe to store anyway I have to, flat files, dbm, mysql. When the data is later retrieved, eval is the easiest way to get it back into a useful hash. Something like this:

use Data::Dumper;
$Data::Dumper::Indent = 0; # no spaces tabs or other fluff
$Data::Dumper::Purity = 1; # include code to recreate recursive refere
+nces

sub store_hash {
    my $hash_ref = shift;
    my $saved = Dumper($hash_ref);
    # store $saved somewhere
}

sub get_hash {
    # get $saved from somewhere
    my $VAR1;
    eval {$saved}
    return $VAR1 ? $VAR1 : {};
}


1) wrapping DBI calls to capture errors. Far as I know this is the recommended (only?) way to do this.

sub selectrow_array { my $dbh = shift;
    my $query = shift;
    my @result;
    eval { @result = $dbh->selectrow_array($query) };
    return my_custom_error_handler("selectrow_array $query") if $@;
    return @result;
}

More generically I guess you could say eval is the tool of choice when ever you have to call code beyond your control that may kill your script.

[eyepopslikeamosquito]

    eval { @result = $dbh->selectrow_array($query) };
    return my_custom_error_handler("selectrow_array $query") if $@;

That is an example of block eval, not string eval; the OP was asking about string eval. There is wide agreement that block eval is a good thing in Perl 5 (for exception handling). Overloading eval with two different meanings in Perl 5 was unfortunate and is fixed in Perl 6, where block eval is spelled try.

[aufflick]

Along the lines of other commenters, if your goal is to enlighten non-agile developers about the beauty of a flexible language and run-time environment like perl, eval EXPR is not the thing to do it with (IMHO).

eval EXPR is what I use in php or Sybase stored procedures, to make up for the lack of expressiveness in the language.

While eval EPXR has definite uses (such as the quoted example of dynamically loading packages), probably 90% of the times that eval EXPR is used in Perl code, faster and more safe code could be written to do the same thing using techniques such as those shown in other responses (such as closures/currying etc.).

In fact, if these developers are serious C or Java developers and talk about things like compiler optimisations then they are going to recognise most eval EXPR based Perl code as a weak solution to hard problems. It would be sad for that to be their first Perl experience as Perl offers such better solutions to those problems than does, say, C or Java.

[eyepopslikeamosquito]

A historic example of string eval that springs to mind is Larry's famous rename script. See also Unix rename faq (a version of Larry's script is given at the end) and the Perl Cookbook, recipe 9.9.

And though these may set a poor example for newbies, please notice that Acme::Bleach:

do {eval brighten $shirt; exit} unless dirty $shirt && not dress $shir
+t;

Acme::Buffy:

do {eval unslay $demon; exit} unless evil $demon && not punch $demon;

and Acme::EyeDrops:

eval eval '"'.
                   ("\["^
    '+').        ('['^')')
.('`'|')')      .('`'|'.').
 ('['^'/').    ('{'^'[').''.
   ('\\').   '"'.('`'|"\(").(
  '`'|'%') .('`'|',').('`'|','
  ).('`'| '/').('{'^'[').("\["^
  ',').('`'|'/').('['^')').('`'
   |',').('`'|'$').'\\'.('\\').(
   '`'|'.').'\\'.'"'.';'.('!'^'+'
    ).'"';$:='.'^'~';$~='@'|"\(";
     $^=')'^'[';$/='`'|'.';$,= ((
       (( '('))))^'}';$\= '`'| ((
          '!')  );$:=')'   ^(( (
          '}'   ))); $~=   '*'
          |((  '`')) ;(     $^
          )=(  '+')  ^+     ((
          ((   (     ((     ((
          ((    (    (      (
          ((     (  (       (
          (       '_'       )
          )      )) ))      )
         ))    ))))  )     ))
       ))))               ;#;#

all rely on good ol' string eval to perform their magic. :-)

[jwkrahn]

http://groups.google.com/group/comp.lang.perl.misc/msg/41264e99d9b0dc53?hl=en&

[ikegami]

Check out NestedLoops in Algorithm::Loops. In this case,

use Algorithm::Loops qw( NestedLoops );
use List::Util       qw( reduce sum );

sub mul { return reduce { $a * $b } @_; }

NestedLoops(
   [
      map { [ 0..$n_ra->[$_] ] }
          0..$#$n_ra
   ],
   sub {
      $prob_ra[sum @_] += mul
                          map { $f_raa->[$_]->[$_[$_]] }
                          0..$#_;
   }
);

I suspect that
$n_ra->[X]
is equal to
$#{$f_raa->[X]}
in which case
map { [ 0..$n_ra->[$_] ] } 0..$#$n_ra
can be replaced with
map { [ 0..$#$_ ] } @$f_raa

[DrHyde]

Here's an example I wrote. It's in use in production code. The method that contains this is a "helper" method which lives in its own package and is used in several classes. It mangles some paramaters and then:
• changes package to whatever class it was invoked from;
• calls a method in that class's superclass.
The reason I need to change package halfway through the method is, from perldoc perlobj:

It is important to note that "SUPER" refers to the super-class(es) of the current package

so instead of just doing this:

$class->SUPER::blah($hash_of_data);

I have to do this:

eval "
    package $class;
    \$class->SUPER::blah(\$hash_of_data);
";

Block eval won't cut it here, and while I'm sure plenty of people are going to bleat that this travesty just demonstrates that my design was wrong in the first place (and I'd agree) in the real world one doesn't always know what all the requirements are when one starts writing code. I decided it was better to write a couple of lines of ugly code - and comment what I was doing and why - than to spend literally days re-jigging everything else so that I could then write that tiny snippet of functionality more cleanly.

[DrHyde]

Here (String eval is cool) is a great example. It's not code generation. In fact, all it does is generate a value from another value. However, using string eval makes it really obvious how the two values are related and avoids several lines of opaque bit-banging.