Re^2: Perl XS portable uint32_t

[tachyon-II]

Hello again almut. I would rather not just compile in the & MASK32 if it is not needed. It is easy enough to set a define in the Makefile.PL if use64bitint is set:

    DEFINE => ($Config{use64bitint} ? '-DUSING_64_BIT_INT' : ''),

Then in the C you can use this to provide 2 versions of the MIX macro. I may well be wrong (can't test) but it seems to me that none of the & MASK32 are required in the main jhash code, provided you use a macro setup as shown below. The a b c ints will probably overflow into the low bits of the 64 bit space but because all the operations are addition the low order bits will be the valid representation in 32 bit space. Could you give this a try:

 
/* Need to constrain U32 to only 32 bits on 64 bit systems
 * For efficiency we only use the & 0xffffffff if required
 */

#define USING_64_BIT_INT /* save messing with Makefile.PL define */

#if defined(USING_64_BIT_INT)
#define MIX(a,b,c) \
{ \
  a &= 0xffffffff; b &= 0xffffffff; c &= 0xffffffff; \
  a -= b; a -= c; a ^= (c>>13); a &= 0xffffffff; \
  b -= c; b -= a; b ^= (a<<8);  b &= 0xffffffff; \
  c -= a; c -= b; c ^= (b>>13); c &= 0xffffffff; \
  a -= b; a -= c; a ^= (c>>12); a &= 0xffffffff; \
  b -= c; b -= a; b ^= (a<<16); b &= 0xffffffff; \
  c -= a; c -= b; c ^= (b>>5);  c &= 0xffffffff; \
  a -= b; a -= c; a ^= (c>>3);  a &= 0xffffffff; \
  b -= c; b -= a; b ^= (a<<10); b &= 0xffffffff; \
  c -= a; c -= b; c ^= (b>>15); c &= 0xffffffff; \
}
#else
#define MIX(a,b,c) \
{ \
  a -= b; a -= c; a ^= (c>>13); \
  b -= c; b -= a; b ^= (a<<8);  \
  c -= a; c -= b; c ^= (b>>13); \
  a -= b; a -= c; a ^= (c>>12); \
  b -= c; b -= a; b ^= (a<<16); \
  c -= a; c -= b; c ^= (b>>5);  \
  a -= b; a -= c; a ^= (c>>3);  \
  b -= c; b -= a; b ^= (a<<10); \
  c -= a; c -= b; c ^= (b>>15); \
}
#endif

/* rest of code unchanged with no masking */

[almut]

I would rather not just compile in the & MASK32 if it is not needed.

That's not what I was trying to say :)   Rather, I meant to imply two things: (a) it might be easier to just do the 32-bit masking yourself, in case you can't easily figure out what the respective magic non-standard __I32_ulong__t incantation is to get the proper 32-bit type on that yet unknown weird 64-bit platform/compiler combo, (b) the performance penalty of doing so is not as big as one might expect.

But you're right, the amount of masking statements actually required may be simplified (I must admit, I hadn't put too much thought into the details here). In particular, masking a, b, c centrally at the beginning of the macro, saves you from having to do so at the end of various other code paths that you do come along...  Interestingly though, there's not much gain in performance from the reduced masking — I now get on average:

      Rate  u32  u64
u32 56.2/s   -- -12%
u64 63.7/s  13%   --

(under otherwise identical conditions)

Anyhow, I tested your simplified code suggestion, and it's working fine (at least on x86_64-linux).

Thanks for adding another useful module to CPAN!

[tachyon-II]

Thanks for testing that. I uploaded version 0.05 a couple of days ago that implemented the dual macro, but did not set the define(s) in the Makefile.PL (I forgot!). The defines used are from Digest::MD5 which have quite a complex makefile setup. Anyway I also changed from using long to perl's U32. Now although the macro must still be the old simple macro it seems to pass on the 64 bit systems it was failing on before (4 of them).

To test the theory that although perl warns you that you can't depend on U32 being exactly 32 bits wide I uploaded Test::U32 - so far it has not detected a failure case. I will be interesting to know.

[syphilis]

It is easy enough to set a define in the Makefile.PL if use64bitint is set

Just be a little cautious with $Config{use64bitint}. It doesn't always tell you what you want/need to know. On 32-bit systems where perl is built with -Duse64bitint, the 'long' and 'int' sizes can (and generally do, I believe) remain at 4 bytes.

I think I've also seen perls built with -Dusemorebits (the equivalent of building with -Duse64bitint && -Duselongdouble) that have neither use64bitint nor uselongdouble defined.

And finally, it would be possible to have 64-bit longs and ints in play without having built with use64bitint support (ie when 64 bits is the size of the long/int on the particular compiler being used).

There are probably other aspects to consider as well. (See the INSTALL file that ships with the perl source for a more authoritative account.)

Cheers,
Rob