http://www.perlmonks.org?node_id=964793

mdej has asked for the wisdom of the Perl Monks concerning the following question:

Hi Perl Guru's

I am stuck with the following: I have a large Hex string (32 Characters) which I want to bitwise AND with a large Integer. I am using the function 'hex' to convert the Hex string to an Integer. However since the number is so large Perl gives an error 'Integer overflow in hexadecimal number'. I am already using the bignum library.

Does somebody know how I can get the Integer from the large Hex number or perhaps an easier way to bitwise two large numbers?

Thanks

Replies are listed 'Best First'.
Re: Bitwise AND with large numbers
by BrowserUk (Patriarch) on Apr 12, 2012 at 19:46 UTC

    Whilst Math::BigInt and similar arbitrary precision modules work for this, they leave a lot to be desired when it comes to performing fixed-precision math. As well as being dog slow, they don't round or truncate as fixed-precision does, and that can lead to the need for lots of extra work to 'fix' intermediate values in calculations.

    If you have a lot of 128-bit integers (16-bytes/32-hex bytes) to manipulate, and need fixed precision semantics, then look at salva's Math::Int128 it is efficient and stable and provides the correct semantics for bitwise manipulations of 128-bit digests, hashes and similar.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?

Re: Bitwise AND with large numbers
by davido (Cardinal) on Apr 12, 2012 at 18:46 UTC

    It seems that really you are just dealing with big integers. The hex representation is only a detail. Math::BigInt handles the details of representation for you, provides a bitwise 'and' operation, named band:

    use Math::BigInt; my $operand = '0x123456789abcdef0123456789abcdef0'; my $mask = '0x100000000f00000000000000000000f0'; print Math::BigInt->new($mask)->band($operand)->as_hex(), "\n";

    Math::BigInt also overloads the '&' operator, so you can do things like this:

    my $anded = Math::BigInt->new($mask) & $operand; print $anded->as_hex(), "\n";

    Dave

Re: Bitwise AND with large numbers
by Eliya (Vicar) on Apr 12, 2012 at 17:50 UTC

    You can do bitwise AND with strings, too (in which case there's no size limitation).

    So, convert your hex string and your number to binary strings using pack (e.g. pack "H*", $hexstring for the hex string), pad the shorter one to the length of the longer one (with \0 or \xff, depending on the desired semantics), AND (&) both strings, and convert the result back to whatever you like (e.g. hex string).

    The details somewhat depend on what exactly you need, so if you provide sample inputs and desired result, someone will surely come up with actual code...

    Otherwise, if you want to stick with Math::Big*, there is from_hex().

      Hi,

      Thank you for your quick reactions ;-)
      Using Math::BigInt or pack I unfortunately still get an integer overflow. I'm probably doing something wrong but don't see what. Sorry if this is to much of a newbie question.

      A little more context: The hexstring comes from a switch. I want to use the hexstring to determine if a switchport is member of a specific VLAN. The code I have so far is:
      my $switchPortNumber = 2; # 32 Character hex string from switch my $hexstring = hex '40000000000000000000000000000000'; my $mask = (0x80000000000000000000000000000000 >> $switchPortNumber); my $result = ($hexstring & $mask) > 0;
      In this case only the first port is a member and the second port is NOT a member. Therefore the $result should be 0

      If possible I would like to stick to the default Perl modules.
      Thanks again for all the help so far.

        This works fine for me:

        use Math::BigInt; my $switchPortNumber = 2; # 32 Character hex string from switch my $hexstring = Math::BigInt->new('0x40000000000000000000000000000000' +); my $mask = (Math::BigInt->new('0x80000000000000000000000000000000') >> + $switchPortNumber); my $result = ($hexstring & $mask) > 0;

        Another approach would be to turn it into an ASCII bitstring, in which case you can use a simple substr() instead of the AND masking

        my $bitstring = unpack('B*', pack('H*', '40000000000000000000000000000 +000')); my $result = substr($bitstring, $switchPortNumber, 1) eq '1';
Re: Bitwise AND with large numbers
by JavaFan (Canon) on Apr 12, 2012 at 18:23 UTC
    If you want to perform a bitwise AND with a Perl integer, the first 16 or 24 "characters" of your hex string will disappear anyway, as your integer will be limited to 32 or 64 bits. So, your result is going to be:
    hex(substr($hex, -8)) & $integer; # Perl with 32-bit integers hex(substr($hex, -16)) & $integer; # Perl with 64-bit integers