Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Transform Sequence Problem

by wbirkett (Novice)
on Jun 19, 2013 at 11:45 UTC ( #1039758=perlquestion: print w/ replies, xml ) Need Help??
wbirkett has asked for the wisdom of the Perl Monks concerning the following question:

Thanks to all those who offered help! Your suggestions helped me find a better solution.

use B::Deparse; # make deparsing object $deparse = B::Deparse->new("-p", "-sC"); # forward encoding transforms @fwd = ( sub {$_[0]/100, ($_[1] + 128)/255, ($_[2] + 128)/255}, sub {$_[0] * 256/25700, ($_[1] + 128) * 256/65535, ($_[2] + 128) * + 256/65535}, sub {$_[0]/100, ($_[1] + 128) * 256/65535, ($_[2] + 128) * 256/655 +35}, sub {@_}, sub {$_[0] + 116 * $_[1]/500, $_[0], $_[0] - 116 * $_[2]/200}, sub {$_[0]/100 + 116 * $_[1]/50000, $_[0]/100, $_[0]/100 - 116 * $ +_[2]/20000}, sub {@_}, sub {$_[0] * 0.4821, $_[1] * 0.5, $_[2] * 0.41245}, sub {$_[0] * 0.9642, $_[1], $_[2] * 0.8249}, sub {@_}, sub {$_[0] * 96.42, $_[1] * 100, $_[2] * 82.49}, ); # reverse encoding transforms @rev = ( sub {$_[0] * 100, $_[1] * 255 - 128, $_[2] * 255 - 128}, sub {$_[0] * 25700/256, $_[1] * 65535/256 - 128, $_[2] * 65535/256 + - 128}, sub {$_[0] * 100, $_[1] * 65535/256 - 128, $_[2] * 65535/256 - 128 +}, sub {@_}, sub {$_[1], 500 * ($_[0] - $_[1])/116, 200 * ($_[1] - $_[2])/116}, sub {$_[1] * 100, 50000 * ($_[0] - $_[1])/116, 20000 * ($_[1] - $_ +[2])/116}, sub {@_}, sub {$_[0]/0.4821, $_[1]/0.5, $_[2]/0.41245}, sub {$_[0]/0.9642, $_[1], $_[2]/0.8249}, sub {@_}, sub {$_[0]/96.42, $_[1]/100, $_[2]/82.49}, ); # CIELAB transforms sub _x2L {($_[0] > 216/24389) ? 116 * $_[0]**(1/3) - 16 : $_[0] * 2438 +9/27} sub _xyz2Lab {my $L = _x2L($_[1]); $L, (_x2L($_[0]) - $L) * 500/116, ( +$L - _x2L($_[2])) * 200/116} sub _L2x {($_[0] > 8) ? (($_[0] + 16)/116)**3 : $_[0] * 27/24389} sub _Lab2xyz {_L2x($_[0] + $_[1] * 116/500), _L2x($_[0]), _L2x($_[0] - + $_[2] * 116/200)} # make example transform sequence array @seq = ($rev[1], \&_Lab2xyz, $fwd[10]); # make combined transform # parameter: (array_of_code_references) # returns: (code_reference) sub combine { # copy transform array my @t = @_; # initialize expression my $expr = '@_'; # for each transform step for my $i (0 .. $#t) { # add to expression $expr = "\$t[$i]->($expr)"; } # return combined transform eval("return(sub {$expr})"); } # combined transform (fast and flexible, but requires initializaton (a +pprox. 75 μs)) $trans3 = combine(@seq); # deparse the code reference $body = $deparse->coderef2text($trans3); print "\$trans3: $body\n"; # call the transform @out = &$trans3(@in); print "@out\n";

The 'combine' function uses 'eval' to create a clean and efficient transform from the component parts. The overhead of this 'combine' step is equal to about 10 transform calls, which is insignificant in my application. ##### original posting ##### I need an efficient way to process a sequence of transforms. For example, here are some individual transforms as code references:

# some code references $f1 = sub {map {$_ + 1} @_}; $f2 = sub {map {log($_)} @_}; $f3 = sub {map {$_ * 3} @_};

They may be combined into a sequence

# combined transform sub trans1 {&$f3(&$f2(&$f1))}; # some data @data = (1, 2, 3); # call the transform @trans = trans1(@data); print "@trans\n";

The sequence could have fewer or more steps, so I would like to make an array of code references:

# make a sequence array @seq = ($f1, $f2, $f3); # combined transform sub trans2 { # for each code reference for (@seq) { # transform data @_ = &$_; } # return return(@_); } # call the transform @trans = trans2(@data); print "@trans\n";

This works okay, but I wonder if there is a better and faster way.

Comment on Transform Sequence Problem
Select or Download Code
Re: Transform Sequence Problem
by hdb (Prior) on Jun 19, 2013 at 12:09 UTC

    Using an array of code references and closures:

    use strict; use warnings; sub fgenerator { # some code references my @f = ( sub {map {$_ + 1} @_}, sub {map {log($_)} @_}, sub {map {$_ * 3} @_}, ); my @idx = @_; return sub { @_ = $f[$_]->(@_) for @idx; return @_; }; } # create transform based on indices of function # read: first apply f0, then f1, then f2 my $trans1 = fgenerator( 0, 1, 2 ); my $trans2 = fgenerator( 0, 0, 0 ); # some data my @data = (1, 2, 3); # call the transforms my @trans = $trans1->( @data ); print "@trans\n"; @trans = $trans2->( @data ); print "@trans\n";

      A few minor changes I like:

      • Using a hash instead an array of code refs allows to give names to the transformations.
      • A bit of error checking can be added, unknown transformations can be ignored and warned about.
      • Instead of passing the indices into the closure, it is more efficient to pass the code references into it. This will generate no additional overhead from the niceties added to the generator function.
      It could look like this:

      use strict; use warnings; sub fgenerator { my %f = ( add_one => sub {map {$_ + 1} @_}, log => sub {map {log($_)} @_}, times_3 => sub {map {$_ * 3} @_}, ); my @t = @f{ grep { exists $f{$_} } @_ }; # ignore unknown name +s warn "Don't know sub(s) ".join ", ",(grep { !exists $f{$_} } @ +_) if @t < @_; return sub { @_ = $_->(@_) for @t; return @_; }; } my $trans1 = fgenerator qw( add_one log times_3 ); my $trans2 = fgenerator qw( add_one add_one add_one times_3 fun ); my @data = (1, 2, 3); my @trans = $trans1->( @data ); print "@trans\n"; @trans = $trans2->( @data ); print "@trans\n";

        Something was missing, now I know:

        think_of_a_number => sub {map {rand $_} @_},
Re: Transform Sequence Problem
by BillKSmith (Chaplain) on Jun 19, 2013 at 12:32 UTC
    I like your solution. You may save some code using a recursion, but it would require more memory and probably be slower. It is a matter of opinion which would be clearer.
    Bill
      The recursion was harder than I expected (but just as short)!
      use strict; use warnings; my @data = (1, 2, 3); $, = "\n"; print trans1(\@data,2); BEGIN{ my $seq =[ sub {map {$_ + 1} @_}, sub {map {log($_)} @_}, sub {map {$_ * 3} @_}, ]; sub trans1 { my ($x, $n) = @_; return $seq->[ 0]->(@$x) if $n == 0; return $seq->[$n]->( trans1($x, $n-1, $seq) ); } }
      Bill
Re: Transform Sequence Problem
by LanX (Canon) on Jun 19, 2013 at 12:55 UTC
    > This works okay, but I wonder if there is a better and faster way.

    Like always... it depends what you wanna do and how flexible you need to be!

    If you are calling a fixed trans() very often, you could consider eval to generate it with chained map statements.

    something like

     eval 'sub trans {  map {$_ + 1} map {log($_)} map {$_ * 3} @_ }'

    You'll need to hold the map-codes as an array of strings.

    But your functional approach is more flexible (eg. for debugging) ! =)

    edit

    you could avoid 3 loops by applying the transformations directly in one loop

     eval 'sub trans {  map { log($_ * 3) + 1} @_ }'

    again this can be constructed as strings (s///ubstituting $_) and evaled ....

    ...OR functionally with

     sub trans {  map { $f1->($f2->($f3->($_))) } @_ }

    update

    or better (untested)

    sub trans { for my $val (@_) { $val = $_->( $val) for @transforms; } return @_; }

    @transform needs to be in the closure, maybe consider passing it as arr-ref as first argument.

     $a_transforms =  shift  

    Cheers Rolf

    ( addicted to the Perl Programming Language)

      LanX:

      I was thinking along the same lines--I coded up a a little something this morning, but never got around to writing a response. It looks like the code I put together goes well with your comments, so without further ado:

      #!/usr/bin/perl use strict; use warnings; use autodie; use Benchmark qw[ cmpthese ]; # operate on an array my $fa_inc = sub { map {$_ + 1} @_ }; my $fa_log = sub { map {log $_} @_ }; my $fa_x3 = sub { map {$_ * 3} @_ }; # edit an array in place my $fea_inc = sub { map {$_ = $_ + 1} @_ }; my $fea_log = sub { map {$_ = log $_} @_ }; my $fea_x3 = sub { map {$_ = $_ * 3} @_ }; # operate on a scalar my $fs_inc = sub { 1 + shift }; my $fs_log = sub { log shift }; my $fs_x3 = sub { 3 * shift }; # operate on a scalar by reference my $fsr_inc = sub { $_[0] + 1 }; my $fsr_log = sub { log $_[0] }; my $fsr_x3 = sub { $_[0] * 3 }; # edit a scalar in place my $fser_inc = sub { $_[0] = $_[0] + 1 }; my $fser_log = sub { $_[0] = log $_[0] }; my $fser_x3 = sub { $_[0] = $_[0] * 3 }; my (@l1, @l2, @l3); push @l1, 100*rand for 0 .. 1000; sub t_fa { $fa_inc->($fa_log->($fa_x3->(@_))) } sub t_fea { @l3 = @_; $fea_inc->($fea_log->($fea_x3->(@l3))) } sub t_fs { map { $fs_inc->($fs_log->($fs_x3->($_))) } @_ } sub t_fsr { map { $fsr_inc->($fsr_log->($fsr_x3->($_))) } @_ } sub t_fser { @l3 = @_; map { $fser_inc->($fser_log->($fser_x3->($_))) } @l3; } # verify that they all have the same result my (@t, @u); @t = t_fa(@l1); @u = t_fea(@l1); cmp_array('fea', \@t, \@u); @u = t_fs(@l1); cmp_array('fea', \@t, \@u); @u = t_fsr(@l1); cmp_array('fea', \@t, \@u); @u = t_fser(@l1); cmp_array('fea', \@t, \@u); sub cmp_array { my ($funcname, $ra1, $ra2) = @_; my ($la1, $la2) = ($#$ra1, $#$ra2); die "$funcname: Mismatched length! $la1 != $la2\n" unless $la1 == $la2; for (0 .. $la1) { die "$funcname: Mismatch $_: $ra1->[$_] != $ra2->[$_]\n" if $ra1->[$_] != $ra2->[$_]; } } cmpthese -1, { fa => q[ @l2 = t_fa(@l1) ], fea => q[ @l2 = t_fea(@l1) ], fs => q[ @l2 = t_fs(@l1) ], fsr => q[ @l2 = t_fsr(@l1) ], fser => q[ @l2 = t_fser(@l1) ], };

      I didn't explicitly test the case you mentioned (combining the functions into a single one). I was more interested in array operations (fa, fea) vs composing scalar functions and applying them to an array.

      In order to make them return the same result without destroying the original array, the "edit in place" functions (fea, fser) perform an array copy. If the original array isn't necessary, you can make it a little faster still--perhaps enough to have the rankings shuffle a little. I was surprised that fs was faster than fser. (I expected that the shift might be a bit more expensive the array indirection.)

      $ perl chain_transform.pl Rate fea fa fser fsr fs fea 199110/s -- -8% -60% -68% -69% fa 216392/s 9% -- -57% -66% -67% fser 499367/s 151% 131% -- -21% -23% fsr 628278/s 216% 190% 26% -- -4% fs 651987/s 227% 201% 31% 4% --

      ...roboticus

      When your only tool is a hammer, all problems look like your thumb.

        Thanx for testing it out.

        Though I'm not sure what you tried to achieve in 'fea_*()'

        In sub { map {$_ = $_ + 1} @_ }; you are changing the array alias in place but effectively only returning the result from map.

        So whats the intended benefit?

        Cheers Rolf

        ( addicted to the Perl Programming Language)

Re: Transform Sequence Problem
by Anonymous Monk on Jun 19, 2013 at 19:46 UTC
    dominus's Higher-Order Perl, the chapter about iterators?
    use Iterator_Utils qw/:all/; my @data = (1, 2, 3); my $data_iter = list_iterator(@data); $f1 = sub { $_ + 1 }; $f2 = sub { log($_) }; $f3 = sub { $_ * 3 }; my $iter = $data_iter; for my $func ($f1, $f2, $f3) { $iter = imap($func, $iter); } while (my $item = NEXTVAL($iter)) { print $item, "\n"; }
    (Iterator_Utils.pm available at his web site)
    # output 2.07944154167984 3.29583686600433 4.15888308335967

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1039758]
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (4)
As of 2014-12-29 03:46 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (184 votes), past polls