Hex to decimal resulting in integer overflow

[Anonymous Monk]

Oh Wise Ones, I have stumbled across an issue I have some difficulties with.
I've been tasked with modifying an existing file that contains id's in hex format(below is actual example). Now I need to generate additional id's by an increment of 1.

$hex="24d0e803adb7";
$dec=hex($hex);

I get the following:
Integer overflow in hexadecimal number at

Can someone give me a pointer as to how to get around this issue?

Thanks,
Wisdom Seeker

[ikegami]

Your build of Perl cannot hold numbers that big (48 bits).

• You could recompile your Perl to support 64-bit numbers.
• You could use a module that provides arbitrary precision number (like Math::BigInt).
• You could use a module that provides access to large native integers (like Math::Int64 and Math::Int128).
• You could even store it in a floating point number without loss.

  my $num = hex(substr(("0"x12).$hex, -12, 4)) * 2**32
      + hex(substr(("0"x12).$hex, -8));
  

Update: Added mention on Math::BigInt, Math::Int64 and Math::Int128.
Update: Fixed bug. (0x12 is not the same as "0"x12!)

[ikegami]

Turns out that hex will actually use floating point numbers if required. The warning doesn't indicate outright failure as Isuspected; it indicate a potentional loss of precision. But it you're numbers are no more than 53 bits in size, there is no problem and you can silence the warning.

[kennethk]

I can't replicate your issue, but searching Google turns up something very similar. What version are you using? I get good behavior on v5.10.1 built for MSWin32-x86-multi-thread and v5.10.1 (*) built for x86_64-linux-gnu-thread-multi.

Update:On v5.8.9 built for MSWin32-x86-multi-thread I can get the warning you are citing. Specifically, perl -e "print hex('24d0e803adb7')" outputs

Integer overflow in hexadecimal number at -e line 1.
40479664352695

This can actually be suppressed with a no warnings declaration, though I'd make sure that the math is still being respected:

{
    no warnings;
    $hex="24d0e803adb7"; 
    $dec=hex($hex);
}

[wind]

Just build your own function to increment theid's. Won't have to worry about any out of bounds issues then.

use strict;

my $hex = "24d0e803adb7";

for (0..16) {
    $hex = inc_hex($hex);
    print $hex, "\n";
}

sub inc_hex {
    my @a = reverse map {hex} split '', $_[0];
    $a[my $i = 0]++;
    while ($a[$i] == 16) {
        $a[$i] = 0;
        $a[++$i]++;
    }
    return join '', map {sprintf "%x", $_} reverse @a;
}

output

24d0e803adb8
24d0e803adb9
24d0e803adba
24d0e803adbb
24d0e803adbc
24d0e803adbd
24d0e803adbe
24d0e803adbf
24d0e803adc0
24d0e803adc1
24d0e803adc2
24d0e803adc3
24d0e803adc4
24d0e803adc5
24d0e803adc6
24d0e803adc7
24d0e803adc8

[tye]

You can also do that with a single regex substitution.

BEGIN {
    my %n= ( 0..9, 'a'..'f', 1..9, 'a'..'f', 10 );

    sub incHex {
        s{
            ( [0-9a-fA-F] ) # Any hex digit followed by
            ( [fF]* )\b     # (always) trailing (optional) 'f's
        }{
            $n{ lc $1 } . '0' x length($2);
        }gex
            for  @_;
    }
}

[wind]

Cool, thanks for sharing. I was actually looking for that solution, but just got a tooth extracted today and the brain just wasn't working right.

The 'g' modifier isn't actually necessary there, and adding 'i' lets you remove the upper case entries from the character classes. Of course, the hash lookup is as fast as you're ever going to get, but I'd still probably choose to do the functional solution.

sub incHex {
    s{
        ( [0-9a-f] ) # Any hex digit followed by
        ( f* )\b     # (always) trailing (optional) 'f's
    }{
        sprintf("%x", 1 + hex $1) . '0' x length($2);
    }iex
        for  @_;
}

Very nice though

[Anonymous Monk]

Thanks everyone,
This was a great help!!!