Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Unpacking variable length bit fields

by puterboy (Scribe)
on Jun 18, 2012 at 20:24 UTC ( #976891=perlquestion: print w/ replies, xml ) Need Help??
puterboy has asked for the wisdom of the Perl Monks concerning the following question:

I have a bitstream containging 3 length fields and the corresponding 3 fields

The bitstream is layed out as follows:

<length field 1 in bits (1 byte)> <field 1> <length field 2 in bits (1 byte)> <field 2> <length field 3 in bits (1 byte)> <field 3>

I am hoping that there is some clever way to do this with unpack but I don't know how to get the field length while also unpacking without iteratively calling pack.

Comment on Unpacking variable length bit fields
Re: Unpacking variable length bit fields
by Limbic~Region (Chancellor) on Jun 18, 2012 at 20:36 UTC
    puterboy,
    Assuming this data is in a string, I would do something like:
    my $skip = 0; while ($skip < length($str)) { my ($f1_len, $f2_len, $f3_len) = unpack("x${skip}aaa", $str); $skip += 3; my $field1 = unpack("x${skip}a$f1_len", $str); $skip += $f1_len; # etc }

    This is like using seek which is exactly what you were trying to avoid (iteratively calling pack) but if wrapped in the proper abstraction can be very clean.

    Update: After reading BrowserUk's response and re-reading your post, I realize I incorrectly assumed you had multiple records in the string - each one with variable length fields. Please ignore me.

    Cheers - L~R

Re: Unpacking variable length bit fields
by BrowserUk (Pope) on Jun 18, 2012 at 20:58 UTC

    Assuming your bitfields are all multiples of 8, then using unpack's length/type nomenclature will work.

    $bitstream = "\x18\xf0\xff\x0f\x10\xaa\x55\x20\xde\xad\xbe\xef";; ( $f1, $f2, $f3 ) = unpack 'C/b C/b c/B', $bitstream;; print for $f1, $f2, $f3;; 000011111111111111110000 0101010110101010 11011110101011011011111011101111
    You'll need to use the correct variations of 'c' or 'C' (probably the latter) and b/B (which will depend upon the endianess of the originating hardware. I've used a mixture here to demonstrate the affect of some of the combinations.

    Note also that the output is ascii-ised binary. That is, each bit of the input is translated to a byte containing ascii '0' or ascii '1'.

    Other variations are possible. For example, you might want to retain the bits as binary encoded fields and then manipulate them with vec. But you'd need to explain more about what they are.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?

      This appears to be EXACTLY what I needed. Thanks!
      Oops, I spoke to soon. The field length specifies the number of bits (multiple of 8) but I want to extract the fields themselves as bytes not as bit strings of 1/0's.

      My problems is that using something like: unpack('C/C'...) would unpack C bytes for each filed rather than C bits for each field which is what I want. So I really would like to divide by 8 but I'm not sure how to do that. I imagine I could unpack as a bit string of 1/0's then split, then repack but that seems klugey.

      It's a screwy format, but I don't have control over it....

        The the easiest way is to convert the ascii-ised binary back to real numbers:

        $bitstream = "\x18\xf0\xff\x0f\x10\xaa\x55\x20\xde\xad\xbe\xef";; ( $f1, $f2, $f3 ) = map unpack( 'L', pack( 'b32', $_ )), unpack 'C/b C +/b C/b', $bitstream;; print for $f1, $f2, $f3;; 1048560 21930 4022250974

        This assumes maximum 32-bit values. You could use Q if you are on a 64-bit platform and Perl and need that.

        By converting them all to the maximum sized integer the platform can handle you save having to make decisions and they'd end up that way anyway.

        You might need to adjust the L (or the 'b32' to 'B32' or 'B64') for the endianness of the originating platform.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?

Re: Unpacking variable length bit fields
by ikegami (Pope) on Jun 18, 2012 at 21:27 UTC
    If your lengths aren't necessarily multiples of 8,
    my @fields; my $bits = unpack('B*', $input); while ($bits) { die if length($bits) < 8; my $num_bits = unpack('C', pack('B*', substr($bits, 0, 8, ''))); my $field = substr($bits, 0, $num_bits, ''); push @fields, $field; }

    You can probably do this more efficiently using bit twiddling, but this is simpler.

    If you want the result as a number rather than a string of "1"s and "0"s, you can use the following:

    use Config qw( ); use constant UV_BITS => $Config::Config{uvsize} * 8; die if $num_bits > UV_BITS; push @fields, pack('B*', substr(("0" x UV_BITS).$field, -UV_BITS));

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://976891]
Approved by sweetblood
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (6)
As of 2014-09-19 06:04 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (131 votes), past polls