The reason for the slow-down is that the first time you are calling
unpack in scalar (well really void) context so it is only extracting one field, while the second time you are extracting all of the fields. So Perl is doing a lot more work.
Now I have seen a couple of signs that unpack might not as fast as it could be. But I would need to look at it closely to figure out why. (Or even if that is true.)
In any case I wouldn't worry about it. This isn't running interactively, is it? If not then wait until you are done and see if it is fast enough...
