Re^3: how to unpack a C struct with length array preceding data array

[BrowserUk]

If anyone can explain what the quoted text means (specifically the integer arguments) or have a simpler example, I would appreciate that.

Re:"For "unpack", an internal stack of integer arguments unpacked so far is used. You write "/"sequence-item and the repeat count is obtained by popping off the last element from the stack."

It means that the template character before the '/', must be one that unpacks an integer. That value is then used as the repeat count for the template character after the '/'.

So, if you use a template of 'C/A' the 'C' will unpack the first character of the string as a number and then its value will determine how many characters of the string are used by the 'A'.

Eg. Below, the C unpacks the first character "\x05", which is then used as the repeat count for the 'A', resulting the next 5 characters being unpacked.

say unpack "C/A", "\x05hellofred";;
hello

Note. The reference to "the stack" refers to Perl's argument stack. As unpack processes each element of its template, the values extracted from the input string are pushed onto that stack, so that when the function returns, they are returned to the caller.

When the '/' is encountered, the last value pushed onto the stack is popped off again and used as the repeat count for the next template character. Hence, in the example above, the value '5' generated by the 'C' is not returned to the caller.

The <t1>/<t2> combination (where t1 can be any template char that results in a numeric value) is very useful for streaming communications protocols where you frequently have length-prefixed data items: <len1><data1><len2><data2>.

Unfortunately, there is no sensible way to use it for your struct format where you have <len1><len2><data1><data2>. (That is, I don't consider the absolute positioning solution above a "sensible" approach as it is not really extensible to more than 2 len/data pairs.

It will almost certainly always be quicker and cleaner to do that in two steps:

my( $len1,  $len2  ) = unpack "SS", $packed;
my( $data1, $data2 ) = unpack "x[SS] A$len1 A$len2", $packed;

[AnomalousMonk]

... the absolute positioning solution ... is not really extensible to more than 2 len/data pairs.

I disagree, with a potential caveat dependent on the exact meaning of the word 'really'. I think I have shown below that an absolute positioning solution can easily (for some definition of 'easy') be generalized to any number of data items using any data length 'type'.

But not all things that are easy are wise, and I continue to agree with you and ig that a two-step approach is almost certainly best.

[BrowserUk]

This is what I mean by "not really extensible":

#! perl -slw
use strict;
use Benchmark qw[ cmpthese ];


our $I //= -1;
our $N //= 4;
my @ls =  map int( 2+rand 10 ), 1 .. $N;
our $str = pack 'S*', @ls;
$str .= 'x' x ($_-1) . ' ' for @ls;

cmpthese $I, {
    a => q[
# line 1 "a"
        my $templ = "S$N" . join( ' ', map{
            "\@0 x[S$_] S \@0" .
            "S$_ x[S${ \( $N-$_ )}]" .
            '/x' x $_ .
            '/a'
        } 0 .. $N-1 );
        my( @d ) = unpack $templ, $str;
        shift @d for 1 .. $N;
        $I == 1 and print 'a ', join '|', @d;
    ],
    b => q[
# line 1 "b"
        my @l = unpack "S$N", $str;
        my $templ = "x[S$N]" . join( ' a', '', @l );
        my @d = unpack $templ, $str;
        $I == 1 and print 'b ', join '|', @d;
    ],
};

__END__
C:\test>junk -I=-1 -N=1
      Rate    a    b
a 128551/s   -- -59%
b 312873/s 143%   --

C:\test>junk -I=-1 -N=10
     Rate    a    b
a 19086/s   -- -67%
b 57153/s 199%   --

C:\test>junk -I=-1 -N=100
    Rate    a    b
a  894/s   -- -89%
b 7979/s 793%   --

You can clean that up by not returning the counts -- which are redundant -- but it is still doing a shit load of extra work in order to avoid two calls to unpack.