Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

unpack - array takes too many bytes

by jjap (Monk)
on Apr 15, 2011 at 04:00 UTC ( #899555=perlquestion: print w/ replies, xml ) Need Help??
jjap has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,

Having settled this issue of unpacking the "records" part of a binary file, I now tackle its "header" (or part of its header).

($name, $onebyte, @twobytes, @fourbytes, $back2onebyte, @trailing) = unpack( "Z16 C1 C2 C4 C1 x400 C*", $data );

The @twobytes array swallows all remaining bytes of the file, not just the 2 is am seeking. The next variables remain of course undefined. Any hints on why that happens?

For a secondary question (once I can solve the first issue), I'll mention I have over 50 variables of varying size in that header. Am I on a valid approach by appending them in the above statement? That will end up being one looong statement...

Again I am very thankful for any hints as well as for your indulgence in case I am missing the obvious

Update
Thanks for very very informative answers, and BrowserUK's suggested use of constant works really nicely! ++ too all!! Update2: enum mentioned later could be a fabulous time saver as well, but I could not find a way to use it with anything else than scalars.

Comment on unpack - array takes too many bytes
Select or Download Code
Re: unpack - array takes too many bytes
by Eliya (Vicar) on Apr 15, 2011 at 04:29 UTC
    The @twobytes array swallows all remaining bytes of the file, not just the 2 is am seeking.

    You can use array slices to prevent the array from swallowing up more than you want:

    my $data = "ABCDEFG"; my ($one, @two, @four); (@two[0..1], @four[0..3], $one) = unpack "C2 C4 C", $data; print "@two | @four | $one"; # 65 66 | 67 68 69 70 | 71

    But note that assigning to a slice does not clear any other elements that might have been in the array.

      Wonderful!

      I guess my misunderstanding was to expected the digit in C2 to limit the assignment to 2 bytes, which it clearly does not.

      Actually replacing C2 by C1 in your code like so:
      (@two[0..1], @four[0..3], $one) = unpack "C1 C4 C", $data; print "@two | @four | $one"; # 65 66 | 67 68 69 70 |
      And warns for uninitialized likely undef. I am really missing the purpose of what the digit does!?
        I am really missing the purpose of what the digit does!?

        Essentially, the unpack just returns a list of values that you have to assign in some meaningful way.

        The digits in the template are just telling unpack how to parse the input data. No provisions are made to assign "sub-lists" to separate arrays, or some such.  In other words, in the example "C2 C4 C" has the same effect as saying "C7".  But note that in other cases, such as "A2 A4 A", where A2 etc. would return a single element, there would a difference to saying "A7".

Re: unpack - array takes too many bytes
by BrowserUk (Pope) on Apr 15, 2011 at 05:00 UTC
    Any hints on why that happens?

    Because that is how array assignment works in Perl. The array expands to accommodate as many items as you assign to it. This is usually seen as a good thing as you'll discover as your experience of Perl grows.

    But also, think about how could perl know you only want two values to be assigned to the first array and four to the second? Perl doesn't understand the English in your variable names :)

    have over 50 variables of varying size in that header. Am I on a valid approach by appending them in the above statement? That will end up being one looong statement...

    Simply put, No. That would not be convenient for you at all. Even if you used the array slice assignment trick: it would be very messy to read & write; difficult to maintain; and very easy to get wrong.

    The simplest thing is to assign all fields to a single array and define constant names to access them:

    use constant { NAME => 0, ONBYTE => 1, TWOBYTE => 2, FOURBYTE => 3, TWOBYTE2 => 4, TRAILING => 5, }; my @fields = unpack( "Z16 C1 C2 C4 C1 x400 C*", $data ); print $fields[ NAME ]; print $fields[ FOURBYTE ]; print for @fields[ TRAILING .. $#fields ];

    Hopefully with better names :)

    But realise, that if you want to unpack two bytes as a single entity, the format C2 will not work. It will return 2 single bytes. You probably want something more like:

    my @fields = unpack( "Z16 C S L C x400 C*", $data );

    That is:1 unsigned byte, 1 unsigned short, 1 unsigned long, another unsigned byte; skip 400 bytes and the unpack what's left as bytes.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: unpack - array takes too many bytes
by BrowserUk (Pope) on Apr 15, 2011 at 13:05 UTC
    BrowserUK's suggested use of constant works really nicely!

    You (and I!) should also look at the enum pragma to simplify declaring such incremental constants:

    use enum qw[ NAME ONEBYTE TWOBYTE FOURBYTE TWOBYTE2 TRAILING ];

    Makes such things much cleaner.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: unpack - array takes too many bytes
by ikegami (Pope) on Apr 15, 2011 at 16:12 UTC

    There's only one thing a function can return: a list of scalars, so as far as the assignment is concerned,

    my ($name, $onebyte, @twobytes, @fourbytes, $back2onebyte, @trailing) = unpack( "Z16 C1 C2 C4 C1 x400 C*", $data);

    is the same thing as

    my ($name, $onebyte, @twobytes, @fourbytes, $back2onebyte, @trailing) = ($z, $c1, $c2_0, $c2_1, $c4_0, $c4_1, $c4_2, $c4_3, ...);

    How can it possibly know it should assign two (no more, no less) of those elements to @twobytes? It can't.

    So what should Perl do when there's an array on the LHS of an assignment? Well, it could throw an error, but instead, it simply assigns everything else to the array. At least that makes (..., @trailing) = useful.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://899555]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (4)
As of 2014-09-22 04:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (178 votes), past polls