Perl Monk, Perl Meditation PerlMonks

### Transform Sequence Problem

by wbirkett (Novice)
 on Jun 19, 2013 at 11:45 UTC Need Help??
wbirkett has asked for the wisdom of the Perl Monks concerning the following question:

Thanks to all those who offered help! Your suggestions helped me find a better solution.

```
use B::Deparse;

# make deparsing object
\$deparse = B::Deparse->new("-p", "-sC");

# forward encoding transforms
@fwd = (
sub {\$_[0]/100, (\$_[1] + 128)/255, (\$_[2] + 128)/255},
sub {\$_[0] * 256/25700, (\$_[1] + 128) * 256/65535, (\$_[2] + 128) *
+ 256/65535},
sub {\$_[0]/100, (\$_[1] + 128) * 256/65535, (\$_[2] + 128) * 256/655
+35},
sub {@_},
sub {\$_[0] + 116 * \$_[1]/500, \$_[0], \$_[0] - 116 * \$_[2]/200},
sub {\$_[0]/100 + 116 * \$_[1]/50000, \$_[0]/100, \$_[0]/100 - 116 * \$
+_[2]/20000},
sub {@_},
sub {\$_[0] * 0.4821, \$_[1] * 0.5, \$_[2] * 0.41245},
sub {\$_[0] * 0.9642, \$_[1], \$_[2] * 0.8249},
sub {@_},
sub {\$_[0] * 96.42, \$_[1] * 100, \$_[2] * 82.49},
);

# reverse encoding transforms
@rev = (
sub {\$_[0] * 100, \$_[1] * 255 - 128, \$_[2] * 255 - 128},
sub {\$_[0] * 25700/256, \$_[1] * 65535/256 - 128, \$_[2] * 65535/256
+ - 128},
sub {\$_[0] * 100, \$_[1] * 65535/256 - 128, \$_[2] * 65535/256 - 128
+},
sub {@_},
sub {\$_[1], 500 * (\$_[0] - \$_[1])/116, 200 * (\$_[1] - \$_[2])/116},
sub {\$_[1] * 100, 50000 * (\$_[0] - \$_[1])/116, 20000 * (\$_[1] - \$_
+[2])/116},
sub {@_},
sub {\$_[0]/0.4821, \$_[1]/0.5, \$_[2]/0.41245},
sub {\$_[0]/0.9642, \$_[1], \$_[2]/0.8249},
sub {@_},
sub {\$_[0]/96.42, \$_[1]/100, \$_[2]/82.49},
);

# CIELAB transforms
sub _x2L {(\$_[0] > 216/24389) ? 116 * \$_[0]**(1/3) - 16 : \$_[0] * 2438
+9/27}
sub _xyz2Lab {my \$L = _x2L(\$_[1]); \$L, (_x2L(\$_[0]) - \$L) * 500/116, (
+\$L - _x2L(\$_[2])) * 200/116}
sub _L2x {(\$_[0] > 8) ? ((\$_[0] + 16)/116)**3 : \$_[0] * 27/24389}
sub _Lab2xyz {_L2x(\$_[0] + \$_[1] * 116/500), _L2x(\$_[0]), _L2x(\$_[0] -
+ \$_[2] * 116/200)}

# make example transform sequence array
@seq = (\$rev[1], \&_Lab2xyz, \$fwd[10]);

# make combined transform
# parameter: (array_of_code_references)
# returns: (code_reference)
sub combine {

# copy transform array
my @t = @_;

# initialize expression
my \$expr = '@_';

# for each transform step
for my \$i (0 .. \$#t) {

\$expr = "\\$t[\$i]->(\$expr)";

}

# return combined transform
eval("return(sub {\$expr})");

}

# combined transform (fast and flexible, but requires initializaton (a
+pprox. 75 &#956;s))
\$trans3 = combine(@seq);

# deparse the code reference
\$body = \$deparse->coderef2text(\$trans3);

print "\\$trans3: \$body\n";

# call the transform
@out = &\$trans3(@in);

print "@out\n";

The 'combine' function uses 'eval' to create a clean and efficient transform from the component parts. The overhead of this 'combine' step is equal to about 10 transform calls, which is insignificant in my application. ##### original posting ##### I need an efficient way to process a sequence of transforms. For example, here are some individual transforms as code references:

```# some code references
\$f1 = sub {map {\$_ + 1} @_};
\$f2 = sub {map {log(\$_)} @_};
\$f3 = sub {map {\$_ * 3} @_};

They may be combined into a sequence

``` # combined transform
sub trans1 {&\$f3(&\$f2(&\$f1))};

# some data
@data = (1, 2, 3);

# call the transform
@trans = trans1(@data);

print "@trans\n";

The sequence could have fewer or more steps, so I would like to make an array of code references:

```# make a sequence array
@seq = (\$f1, \$f2, \$f3);

# combined transform
sub trans2 {

# for each code reference
for (@seq) {

# transform data
@_ = &\$_;

}

# return
return(@_);

}

# call the transform
@trans = trans2(@data);

print "@trans\n";

This works okay, but I wonder if there is a better and faster way.

Replies are listed 'Best First'.
Re: Transform Sequence Problem
by hdb (Monsignor) on Jun 19, 2013 at 12:09 UTC

Using an array of code references and closures:

```use strict;
use warnings;

sub fgenerator {
# some code references
my @f = (
sub {map {\$_ + 1} @_},
sub {map {log(\$_)} @_},
sub {map {\$_ * 3} @_},
);
my @idx = @_;
return sub {
@_ = \$f[\$_]->(@_) for @idx;
return @_;
};
}

# create transform based on indices of function
# read:  first apply f0, then f1, then f2
my \$trans1 = fgenerator( 0, 1, 2 );
my \$trans2 = fgenerator( 0, 0, 0 );

# some data
my @data = (1, 2, 3);

# call the transforms
my @trans = \$trans1->( @data );
print "@trans\n";
@trans = \$trans2->( @data );
print "@trans\n";

A few minor changes I like:

• Using a hash instead an array of code refs allows to give names to the transformations.
• A bit of error checking can be added, unknown transformations can be ignored and warned about.
• Instead of passing the indices into the closure, it is more efficient to pass the code references into it. This will generate no additional overhead from the niceties added to the generator function.
It could look like this:

```use strict;
use warnings;

sub fgenerator {
my %f = (
add_one => sub {map {\$_ + 1} @_},
log     => sub {map {log(\$_)} @_},
times_3 => sub {map {\$_ * 3} @_},
);
my @t = @f{ grep { exists \$f{\$_} } @_ }; # ignore unknown name
+s
warn "Don't know sub(s) ".join ", ",(grep { !exists \$f{\$_} } @
+_) if @t < @_;
return sub { @_ = \$_->(@_) for @t; return @_; };
}

my \$trans1 = fgenerator qw( add_one log times_3 );

my @data = (1, 2, 3);
my @trans = \$trans1->( @data );
print "@trans\n";
@trans = \$trans2->( @data );
print "@trans\n";

Something was missing, now I know:

```think_of_a_number => sub {map {rand \$_} @_},
Re: Transform Sequence Problem
by LanX (Bishop) on Jun 19, 2013 at 12:55 UTC
> This works okay, but I wonder if there is a better and faster way.

Like always... it depends what you wanna do and how flexible you need to be!

If you are calling a fixed trans() very often, you could consider eval to generate it with chained map statements.

something like

eval 'sub trans {  map {\$_ + 1} map {log(\$_)} map {\$_ * 3} @_ }'

You'll need to hold the map-codes as an array of strings.

But your functional approach is more flexible (eg. for debugging) ! =)

##### edit

you could avoid 3 loops by applying the transformations directly in one loop

eval 'sub trans {  map { log(\$_ * 3) + 1} @_ }'

again this can be constructed as strings (s///ubstituting \$_) and evaled ....

...OR functionally with

sub trans {  map { \$f1->(\$f2->(\$f3->(\$_))) } @_ }

##### update

or better (untested)

```
sub trans {
for my \$val (@_) {
\$val = \$_->( \$val) for @transforms;
}
return @_;
}

@transform needs to be in the closure, maybe consider passing it as arr-ref as first argument.

\$a_transforms =  shift

Cheers Rolf

( addicted to the Perl Programming Language)

LanX:

I was thinking along the same lines--I coded up a a little something this morning, but never got around to writing a response. It looks like the code I put together goes well with your comments, so without further ado:

```#!/usr/bin/perl
use strict; use warnings; use autodie;
use Benchmark qw[ cmpthese ];

# operate on an array
my \$fa_inc = sub { map {\$_ + 1} @_ };
my \$fa_log = sub { map {log \$_} @_ };
my \$fa_x3  = sub { map {\$_ * 3} @_ };

# edit an array in place
my \$fea_inc = sub { map {\$_ = \$_ + 1} @_ };
my \$fea_log = sub { map {\$_ = log \$_} @_ };
my \$fea_x3  = sub { map {\$_ = \$_ * 3} @_ };

# operate on a scalar
my \$fs_inc = sub { 1 + shift };
my \$fs_log = sub { log shift };
my \$fs_x3  = sub { 3 * shift };

# operate on a scalar by reference
my \$fsr_inc = sub { \$_[0] + 1 };
my \$fsr_log = sub { log \$_[0] };
my \$fsr_x3  = sub { \$_[0] * 3 };

# edit a scalar in place
my \$fser_inc = sub { \$_[0] = \$_[0] + 1 };
my \$fser_log = sub { \$_[0] = log \$_[0] };
my \$fser_x3  = sub { \$_[0] = \$_[0] * 3 };

my (@l1, @l2, @l3);
push @l1, 100*rand for 0 .. 1000;

sub t_fa {
\$fa_inc->(\$fa_log->(\$fa_x3->(@_)))
}

sub t_fea {
@l3 = @_;
\$fea_inc->(\$fea_log->(\$fea_x3->(@l3)))
}

sub t_fs {
map { \$fs_inc->(\$fs_log->(\$fs_x3->(\$_))) } @_
}

sub t_fsr {
map { \$fsr_inc->(\$fsr_log->(\$fsr_x3->(\$_))) } @_
}

sub t_fser {
@l3 = @_;
map { \$fser_inc->(\$fser_log->(\$fser_x3->(\$_))) } @l3;
}

# verify that they all have the same result
my (@t, @u);
@t = t_fa(@l1);

@u = t_fea(@l1);
cmp_array('fea', \@t, \@u);

@u = t_fs(@l1);
cmp_array('fea', \@t, \@u);

@u = t_fsr(@l1);
cmp_array('fea', \@t, \@u);

@u = t_fser(@l1);
cmp_array('fea', \@t, \@u);

sub cmp_array {
my (\$funcname, \$ra1, \$ra2) = @_;
my (\$la1, \$la2) = (\$#\$ra1, \$#\$ra2);
die "\$funcname: Mismatched length! \$la1 != \$la2\n"
unless \$la1 == \$la2;
for (0 .. \$la1) {
die "\$funcname: Mismatch \$_: \$ra1->[\$_] != \$ra2->[\$_]\n"
if \$ra1->[\$_] != \$ra2->[\$_];
}
}

cmpthese -1, {
fa   => q[ @l2 = t_fa(@l1)   ],
fea  => q[ @l2 = t_fea(@l1)  ],
fs   => q[ @l2 = t_fs(@l1)   ],
fsr  => q[ @l2 = t_fsr(@l1)  ],
fser => q[ @l2 = t_fser(@l1) ],
};

I didn't explicitly test the case you mentioned (combining the functions into a single one). I was more interested in array operations (fa, fea) vs composing scalar functions and applying them to an array.

In order to make them return the same result without destroying the original array, the "edit in place" functions (fea, fser) perform an array copy. If the original array isn't necessary, you can make it a little faster still--perhaps enough to have the rankings shuffle a little. I was surprised that fs was faster than fser. (I expected that the shift might be a bit more expensive the array indirection.)

```\$ perl chain_transform.pl
Rate  fea   fa fser  fsr   fs
fea  199110/s   --  -8% -60% -68% -69%
fa   216392/s   9%   -- -57% -66% -67%
fser 499367/s 151% 131%   -- -21% -23%
fsr  628278/s 216% 190%  26%   --  -4%
fs   651987/s 227% 201%  31%   4%   --

...roboticus

When your only tool is a hammer, all problems look like your thumb.

Thanx for testing it out.

Though I'm not sure what you tried to achieve in 'fea_*()'

In sub { map {\$_ = \$_ + 1} @_ }; you are changing the array alias in place but effectively only returning the result from map.

So whats the intended benefit?

Cheers Rolf

( addicted to the Perl Programming Language)

Re: Transform Sequence Problem
by BillKSmith (Parson) on Jun 19, 2013 at 12:32 UTC
I like your solution. You may save some code using a recursion, but it would require more memory and probably be slower. It is a matter of opinion which would be clearer.
Bill
The recursion was harder than I expected (but just as short)!
```use strict;
use warnings;
my @data = (1, 2, 3);
\$, = "\n";
print trans1(\@data,2);

BEGIN{
my \$seq =[
sub {map {\$_ + 1}  @_},
sub {map {log(\$_)} @_},
sub {map {\$_ * 3}  @_},
];
sub trans1 {
my (\$x, \$n) = @_;
return \$seq->[ 0]->(@\$x) if \$n == 0;
return \$seq->[\$n]->( trans1(\$x, \$n-1, \$seq) );
}
}
Bill
Re: Transform Sequence Problem
by Anonymous Monk on Jun 19, 2013 at 19:46 UTC
dominus's Higher-Order Perl, the chapter about iterators?
```use Iterator_Utils qw/:all/;

my @data = (1, 2, 3);

my \$data_iter = list_iterator(@data);

\$f1 = sub { \$_ + 1 };
\$f2 = sub { log(\$_) };
\$f3 = sub { \$_ * 3 };

my \$iter = \$data_iter;
for my \$func (\$f1, \$f2, \$f3) {
\$iter = imap(\$func, \$iter);
}

while (my \$item = NEXTVAL(\$iter)) {
print \$item, "\n";
}
(Iterator_Utils.pm available at his web site)
```# output
2.07944154167984
3.29583686600433
4.15888308335967

Create A New User
Node Status?
node history
Node Type: perlquestion [id://1039758]
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (3)
As of 2018-08-14 23:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
Voting Booth?
Asked to put a square peg in a round hole, I would:

Results (155 votes). Check out past polls.

Notices?