Transform Sequence Problem

[wbirkett]

Thanks to all those who offered help! Your suggestions helped me find a better solution.

use B::Deparse;

# make deparsing object
$deparse = B::Deparse->new("-p", "-sC");

# forward encoding transforms
@fwd = (
    sub {$_[0]/100, ($_[1] + 128)/255, ($_[2] + 128)/255},
    sub {$_[0] * 256/25700, ($_[1] + 128) * 256/65535, ($_[2] + 128) *
+ 256/65535},
    sub {$_[0]/100, ($_[1] + 128) * 256/65535, ($_[2] + 128) * 256/655
+35},
    sub {@_},
    sub {$_[0] + 116 * $_[1]/500, $_[0], $_[0] - 116 * $_[2]/200},
    sub {$_[0]/100 + 116 * $_[1]/50000, $_[0]/100, $_[0]/100 - 116 * $
+_[2]/20000},
    sub {@_},
    sub {$_[0] * 0.4821, $_[1] * 0.5, $_[2] * 0.41245},
    sub {$_[0] * 0.9642, $_[1], $_[2] * 0.8249},
    sub {@_},
    sub {$_[0] * 96.42, $_[1] * 100, $_[2] * 82.49},
);

# reverse encoding transforms
@rev = (
    sub {$_[0] * 100, $_[1] * 255 - 128, $_[2] * 255 - 128},
    sub {$_[0] * 25700/256, $_[1] * 65535/256 - 128, $_[2] * 65535/256
+ - 128},
    sub {$_[0] * 100, $_[1] * 65535/256 - 128, $_[2] * 65535/256 - 128
+},
    sub {@_},
    sub {$_[1], 500 * ($_[0] - $_[1])/116, 200 * ($_[1] - $_[2])/116},
    sub {$_[1] * 100, 50000 * ($_[0] - $_[1])/116, 20000 * ($_[1] - $_
+[2])/116},
    sub {@_},
    sub {$_[0]/0.4821, $_[1]/0.5, $_[2]/0.41245},
    sub {$_[0]/0.9642, $_[1], $_[2]/0.8249},
    sub {@_},
    sub {$_[0]/96.42, $_[1]/100, $_[2]/82.49},
);

# CIELAB transforms
sub _x2L {($_[0] > 216/24389) ? 116 * $_[0]**(1/3) - 16 : $_[0] * 2438
+9/27}
sub _xyz2Lab {my $L = _x2L($_[1]); $L, (_x2L($_[0]) - $L) * 500/116, (
+$L - _x2L($_[2])) * 200/116}
sub _L2x {($_[0] > 8) ? (($_[0] + 16)/116)**3 : $_[0] * 27/24389}
sub _Lab2xyz {_L2x($_[0] + $_[1] * 116/500), _L2x($_[0]), _L2x($_[0] -
+ $_[2] * 116/200)}

# make example transform sequence array
@seq = ($rev[1], \&_Lab2xyz, $fwd[10]);

# make combined transform
# parameter: (array_of_code_references)
# returns: (code_reference)
sub combine {
    
    # copy transform array
    my @t = @_;
    
    # initialize expression
    my $expr = '@_';
    
    # for each transform step
    for my $i (0 .. $#t) {
        
        # add to expression
        $expr = "\$t[$i]->($expr)";
        
    }
    
    # return combined transform
    eval("return(sub {$expr})");
    
}

# combined transform (fast and flexible, but requires initializaton (a
+pprox. 75 μs))
$trans3 = combine(@seq);

# deparse the code reference
$body = $deparse->coderef2text($trans3);

print "\$trans3: $body\n";

# call the transform
@out = &$trans3(@in);

print "@out\n";

The 'combine' function uses 'eval' to create a clean and efficient transform from the component parts. The overhead of this 'combine' step is equal to about 10 transform calls, which is insignificant in my application. ##### original posting ##### I need an efficient way to process a sequence of transforms. For example, here are some individual transforms as code references:

# some code references
$f1 = sub {map {$_ + 1} @_};
$f2 = sub {map {log($_)} @_};
$f3 = sub {map {$_ * 3} @_};

They may be combined into a sequence

 # combined transform
sub trans1 {&$f3(&$f2(&$f1))};

# some data
@data = (1, 2, 3);

# call the transform
@trans = trans1(@data);

print "@trans\n";

The sequence could have fewer or more steps, so I would like to make an array of code references:

# make a sequence array
@seq = ($f1, $f2, $f3);

# combined transform
sub trans2 {
    
    # for each code reference
    for (@seq) {
        
        # transform data
        @_ = &$_;
        
    }
    
    # return
    return(@_);
    
}

# call the transform
@trans = trans2(@data);

print "@trans\n";

This works okay, but I wonder if there is a better and faster way.

[hdb]

Using an array of code references and closures:

use strict;
use warnings;

sub fgenerator {
# some code references
        my @f = (
                sub {map {$_ + 1} @_},
                sub {map {log($_)} @_},
                sub {map {$_ * 3} @_},
                );
        my @idx = @_;
        return sub {
                @_ = $f[$_]->(@_) for @idx;
                return @_;
        };
}

# create transform based on indices of function
# read:  first apply f0, then f1, then f2
my $trans1 = fgenerator( 0, 1, 2 );
my $trans2 = fgenerator( 0, 0, 0 );

# some data
my @data = (1, 2, 3);

# call the transforms
my @trans = $trans1->( @data );
print "@trans\n";
@trans = $trans2->( @data );
print "@trans\n";

[hdb]

A few minor changes I like:

• Using a hash instead an array of code refs allows to give names to the transformations.
• A bit of error checking can be added, unknown transformations can be ignored and warned about.
• Instead of passing the indices into the closure, it is more efficient to pass the code references into it. This will generate no additional overhead from the niceties added to the generator function.
It could look like this:

use strict;
use warnings;

sub fgenerator {
        my %f = (
                add_one => sub {map {$_ + 1} @_},
                log     => sub {map {log($_)} @_},
                times_3 => sub {map {$_ * 3} @_},
                );
        my @t = @f{ grep { exists $f{$_} } @_ }; # ignore unknown name
+s
        warn "Don't know sub(s) ".join ", ",(grep { !exists $f{$_} } @
+_) if @t < @_;
        return sub { @_ = $_->(@_) for @t; return @_; };
}

my $trans1 = fgenerator qw( add_one log times_3 );
my $trans2 = fgenerator qw( add_one add_one add_one times_3 fun );

my @data = (1, 2, 3);
my @trans = $trans1->( @data );
print "@trans\n";
@trans = $trans2->( @data );
print "@trans\n";

[hdb]

Something was missing, now I know:

think_of_a_number => sub {map {rand $_} @_},

[LanX]

> This works okay, but I wonder if there is a better and faster way.

Like always... it depends what you wanna do and how flexible you need to be!

If you are calling a fixed trans() very often, you could consider eval to generate it with chained map statements.

something like

 eval 'sub trans {  map {$_ + 1} map {log($_)} map {$_ * 3} @_ }'

You'll need to hold the map-codes as an array of strings.

But your functional approach is more flexible (eg. for debugging) ! =)

edit

you could avoid 3 loops by applying the transformations directly in one loop

 eval 'sub trans {  map { log($_ * 3) + 1} @_ }'

again this can be constructed as strings (s///ubstituting $_) and evaled ....

...OR functionally with

 sub trans {  map { $f1->($f2->($f3->($_))) } @_ }

update

or better (untested)

 
sub trans {
  for my $val (@_) {
     $val = $_->( $val) for @transforms;
  }
  return @_;
}

@transform needs to be in the closure, maybe consider passing it as arr-ref as first argument.

 $a_transforms =  shift

[roboticus]

LanX:

I was thinking along the same lines--I coded up a a little something this morning, but never got around to writing a response. It looks like the code I put together goes well with your comments, so without further ado:

#!/usr/bin/perl
use strict; use warnings; use autodie;
use Benchmark qw[ cmpthese ];

# operate on an array
my $fa_inc = sub { map {$_ + 1} @_ };
my $fa_log = sub { map {log $_} @_ };
my $fa_x3  = sub { map {$_ * 3} @_ };

# edit an array in place
my $fea_inc = sub { map {$_ = $_ + 1} @_ };
my $fea_log = sub { map {$_ = log $_} @_ };
my $fea_x3  = sub { map {$_ = $_ * 3} @_ };

# operate on a scalar
my $fs_inc = sub { 1 + shift };
my $fs_log = sub { log shift };
my $fs_x3  = sub { 3 * shift };

# operate on a scalar by reference
my $fsr_inc = sub { $_[0] + 1 };
my $fsr_log = sub { log $_[0] };
my $fsr_x3  = sub { $_[0] * 3 };

# edit a scalar in place
my $fser_inc = sub { $_[0] = $_[0] + 1 };
my $fser_log = sub { $_[0] = log $_[0] };
my $fser_x3  = sub { $_[0] = $_[0] * 3 };

my (@l1, @l2, @l3);
push @l1, 100*rand for 0 .. 1000;

sub t_fa {
    $fa_inc->($fa_log->($fa_x3->(@_)))
}

sub t_fea {
    @l3 = @_;
    $fea_inc->($fea_log->($fea_x3->(@l3)))
}

sub t_fs {
    map { $fs_inc->($fs_log->($fs_x3->($_))) } @_
}

sub t_fsr {
    map { $fsr_inc->($fsr_log->($fsr_x3->($_))) } @_
}

sub t_fser {
    @l3 = @_;
    map { $fser_inc->($fser_log->($fser_x3->($_))) } @l3;
}

# verify that they all have the same result
my (@t, @u);
@t = t_fa(@l1);

@u = t_fea(@l1);
cmp_array('fea', \@t, \@u);

@u = t_fs(@l1);
cmp_array('fea', \@t, \@u);

@u = t_fsr(@l1);
cmp_array('fea', \@t, \@u);

@u = t_fser(@l1);
cmp_array('fea', \@t, \@u);

sub cmp_array {
    my ($funcname, $ra1, $ra2) = @_;
    my ($la1, $la2) = ($#$ra1, $#$ra2);
    die "$funcname: Mismatched length! $la1 != $la2\n" 
      unless $la1 == $la2;
    for (0 .. $la1) {
        die "$funcname: Mismatch $_: $ra1->[$_] != $ra2->[$_]\n" 
          if $ra1->[$_] != $ra2->[$_];
    }
}

cmpthese -1, {
    fa   => q[ @l2 = t_fa(@l1)   ],
    fea  => q[ @l2 = t_fea(@l1)  ],
    fs   => q[ @l2 = t_fs(@l1)   ],
    fsr  => q[ @l2 = t_fsr(@l1)  ],
    fser => q[ @l2 = t_fser(@l1) ],
};

I didn't explicitly test the case you mentioned (combining the functions into a single one). I was more interested in array operations (fa, fea) vs composing scalar functions and applying them to an array.

In order to make them return the same result without destroying the original array, the "edit in place" functions (fea, fser) perform an array copy. If the original array isn't necessary, you can make it a little faster still--perhaps enough to have the rankings shuffle a little. I was surprised that fs was faster than fser. (I expected that the shift might be a bit more expensive the array indirection.)

$ perl chain_transform.pl
         Rate  fea   fa fser  fsr   fs
fea  199110/s   --  -8% -60% -68% -69%
fa   216392/s   9%   -- -57% -66% -67%
fser 499367/s 151% 131%   -- -21% -23%
fsr  628278/s 216% 190%  26%   --  -4%
fs   651987/s 227% 201%  31%   4%   --

...roboticus

When your only tool is a hammer, all problems look like your thumb.

[LanX]

Thanx for testing it out.

Though I'm not sure what you tried to achieve in 'fea_*()'

In sub { map {$_ = $_ + 1} @_ }; you are changing the array alias in place but effectively only returning the result from map.

So whats the intended benefit?

[BillKSmith]

I like your solution. You may save some code using a recursion, but it would require more memory and probably be slower. It is a matter of opinion which would be clearer.

[BillKSmith]

The recursion was harder than I expected (but just as short)!

use strict;
use warnings;
my @data = (1, 2, 3);
$, = "\n";
print trans1(\@data,2);

BEGIN{
    my $seq =[
        sub {map {$_ + 1}  @_},
        sub {map {log($_)} @_},
        sub {map {$_ * 3}  @_},
    ];
    sub trans1 {
        my ($x, $n) = @_;
        return $seq->[ 0]->(@$x) if $n == 0;
        return $seq->[$n]->( trans1($x, $n-1, $seq) );
    }
}

[Anonymous Monk]

dominus's Higher-Order Perl, the chapter about iterators?

use Iterator_Utils qw/:all/;

my @data = (1, 2, 3);

my $data_iter = list_iterator(@data);


$f1 = sub { $_ + 1 };
$f2 = sub { log($_) };
$f3 = sub { $_ * 3 };

my $iter = $data_iter;
for my $func ($f1, $f2, $f3) {
    $iter = imap($func, $iter);
}

while (my $item = NEXTVAL($iter)) {
    print $item, "\n";
}

(Iterator_Utils.pm available at his web site)

# output
2.07944154167984
3.29583686600433
4.15888308335967