unpack'ing "64-bit quantities"

[isync]

Being a novice when it comes to binary structures I would like to ask the monks for a quick answer on how to correctly unpack() "64-bit quantities" into more or less human readable integers.

I am reading out raw file times with CLI's getfattr from NTFS via ntfs-3g and would like to get them into human-readable form...

[BrowserUk]

That really depends upon what format (within Perl), you are getting them in? Eg. Pure binary bytes; hex encoded etc.

And whether you're running a 64-bit integer capable version of Perl? Eg. Some 32-bit builds and all 64-bit builds will support the Q and q unpack formats specifically for handling this.

If you're using a 32-bit perl without 64-bit integers, then you can unpack the values as two 32-bit integers and then multiply them out to obtain an up to 53-bit integer value stored in a 64-bit real, which covers many uses. See Re: Hexadecimal to binary for one example of handling 128-bit values.

With a clearer description--and preferably a couple of samples--you'll get better responses.

[Xilman]

For portability reasons, I'd unpack them into two 32-bit variables and then process them further. This will work regardless of whether you have a 32-bit or 64-bit Perl.

What you do after that depends on what you mean by "more or less human readable". I happen to find hexadecimal more or less readable, in which case displaying first the more significant and then the less significant 32 bit values is acceptable.

If you wish to display them as decimal integers, you can either roll your own code in a (fairly obvious) manner or exploit the capabilities of the Math::BigInt module.

Paul

[salva]

Math::Int64 provides functions for packing and unpacking 64 bit integers.

[isync]

Probing a ntfs-3g mounted volume, the CLI command getfattr -n system.ntfs_times file gives me:

system.ntfs_times=0sUMKxfHwEyQFMaayBfwTJAXa37TWB98oB7hGD3t6jyQE=

and according to man the leading 0s indicates it is base64 encoded. Likewise getfattr -e hex -n system.ntfs_times file gives hex output:

system.ntfs_times=0x50c2b17c7c04c9014c69ac817f04c90176b7ed3581f7ca01ee1183dedea3c901

And from there, even your verbose comments, do not help me much. I've used unpack() in the past but more in trial and error... How am I supposed to extract 4 epoch values from this string?

[BrowserUk]

Less verbose:

#! perl -slw
use strict;
use constant {
    MSEpoch_2_UnixEpoch => 116444736000000000, ## corrected
};

my $in = '0x50c2b17c7c04c9014c69ac817f04c90176b7ed3581f7ca01ee1183dede
+a3c901';

my @hexTimes = unpack 'xx(A16)4', $in;
my @binTimes = map{ unpack 'Q', pack 'H16', $_ } @hexTimes;
my @unixTimes = map{ ( $_ - MSEpoch_2_UnixEpoch ) /1e7 } @binTimes;

print scalar localtime( $_ ) for @unixTimes;

__END__
[13:52:05.69]C:\test>845551.pl
Fri Aug 22 17:28:27 2008
Fri Aug 22 17:50:03 2008
Wed May 19 18:29:26 2010
Fri Mar 13 13:23:16 2009

[almut]

MSEpoch_2_UnixEpoch => 116444700000000000, ## Corrected (I think)

Not entirely sure, but I think it should be

116444736000000000 == (int( (1970 - 1601) * 365.25 ) - 3) * 24 * 60 * 
+60 * 1e7

(your magic value seems to be one hour off)

[isync]

Of course, "verbose" (copious) was meant in a positive way!

Thanks again...

[isync]

Next problem: I am on 32bit and unable to find the substitute for unpack 'Q'...

# my @binTimes = map{ unpack 'Q', pack 'H16', $_ } @hexTimes;
my @binTimes = map{ Math::Int64::uint64_to_number(pack 'H16', $_) } @h
+exTimes; # wrong!

Related posts: Ubuntu bug report, another Perlmonks thread

[BrowserUk]

Doing this with a 32-bit only Perl is a tad more complex, but works. You break the 64-bit number into 2 32-bit values and then multiply them out. You do have to pay attention to endianess though.

#! perl -slw
use strict;
use constant {
    MSEpoch_2_UnixEpoch => 116444736000000000,
};

my $in = '0x50c2b17c7c04c9014c69ac817f04c90176b7ed3581f7ca01ee1183dede
+a3c901';

my @hexTimes = unpack 'xx(A16)4', $in;

my @binTimes = map{
    my( $lo, $hi ) = unpack 'VV', pack 'H8H8', unpack 'A8A8', $_;
    $hi * 2**32 + $lo
} @hexTimes;

my @unixTimes = map{ ( $_ - MSEpoch_2_UnixEpoch ) /1e7 } @binTimes;

print scalar localtime( $_ ) for @unixTimes;

__END__
[13:52:05.69]C:\test>845551.pl
Fri Aug 22 17:28:27 2008
Fri Aug 22 17:50:03 2008
Wed May 19 18:29:26 2010
Fri Mar 13 13:23:16 2009

[isync]

Works! Would've never figured that one out myself... Thanks a bunch!