Beefy Boxes and Bandwidth Generously Provided by pair Networks
Come for the quick hacks, stay for the epiphanies.
 
PerlMonks  

Storable- long integer size

by sectokia (Pilgrim)
on Oct 21, 2015 at 10:41 UTC ( [id://1145536]=perlquestion: print w/replies, xml ) Need Help??

sectokia has asked for the wisdom of the Perl Monks concerning the following question:

Hi wise monks,

I have just changed from OSX to Win10, and storable retrieve is failing with "long integer size is not compatible". I have tried both the 32bit and 64bit versions of perl on windows.

Any ideas? Everything I have googled and perldoc only talks about 'Byte order not compatible'.

I still have the OSX system running, so hopefully there is some way I can store it differently to be compatible? I am using perl 5.22.0

Replies are listed 'Best First'.
Re: Storable- long integer size
by Corion (Patriarch) on Oct 21, 2015 at 10:50 UTC

    Using Storable to transfer data between Perl installations is not really advisable in my opinion. You need to make sure that the Perl and Storable versions are identical and also that the flags used to compile Perl are identical. You also need to make sure to use nstore* instead of store everywhere (even though that shouldn't matter when moving between the same CPU platform). Maybe your OSX Perl is compiled with 64bit long and the Windows Perl is not? Or maybe it's the other way around?

    You can find out where the differences between your Perl versions lie by comparing the output of perl -V in both environments.

    If you mostly need a one-time migration, consider using Data::Dumper to transfer the data from the old environment to the new environment. If the file size is too large, Sereal also might be byte-order independent.

    Personally, I would look at using SQLite or SQL as a transfer mechanism, or JSON. JSON doesn't like to export blessed structures though.

    Having an export and import feature is also a good approach for having good backups, in the case that the original environment was lost and cannot be restored as easily as a similar but crucially different environment.

      Maybe your OSX Perl is compiled with 64bit long and the Windows Perl is not?

      I think that quite likely explains it.
      On Windows, the "long" is always 32bit. Therefore, on a 64bit integer build of Windows perl, the IV type will always be something other than "long".

      However, I don't know enough about Storable to know whether that should trip things up.
      I was thinking that, so long as the IV sizes were the same on both perls (and endianness was not an issue), then maybe it shouldn't matter if the respective $Config{ivtype} values don't match ?
      (Clarification welcome.)

      Cheers,
      Rob
        linux-x64 $ perl -V:'.*size|byte.*' | sort byteorder='12345678'; charsize='1'; d_chsize='undef'; d_malloc_good_size='undef'; d_malloc_size='undef'; doublesize='8'; fpossize='16'; gidsize='4'; i16size='2'; i32size='4'; i64size='8'; i8size='1'; intsize='4'; ivsize='8'; longdblsize='16'; longlongsize='8'; longsize='8'; lseeksize='8'; nvsize='16'; ptrsize='8'; shortsize='2'; sig_size='69'; sizesize='8'; st_ino_size='8'; u16size='2'; u32size='4'; u64size='8'; u8size='1'; uidsize='4'; uvsize='8'; $
        hp-ux ia64 $ perl -V:'.*size|byte.*' | sort byteorder='87654321'; charsize='1'; d_chsize='undef'; d_malloc_good_size='undef'; d_malloc_size='undef'; doublesize='8'; fpossize='8'; gidsize='4'; i16size='2'; i32size='4'; i64size='8'; i8size='1'; intsize='4'; ivsize='8'; longdblsize='16'; longlongsize='8'; longsize='8'; lseeksize='8'; nvsize='16'; ptrsize='8'; shortsize='2'; sig_size='49'; sizesize='8'; u16size='2'; u32size='4'; u64size='8'; u8size='1'; uidsize='4'; uvsize='8'; $

        The quotation will be different on Windows. An IV has not only length shown by ivsize, but also an order, as shown by byteorder. Both have to match for Storable to work. I also would promote the use of Sereal instead, as that was written with portability in mind.

        You can also keep using Storable if what you store are strings from pack where you define the format yourself. A long would then be packed using "l>", and unpacked with the same signature: the ">" defines the endianness


        Enjoy, Have FUN! H.Merijn
Re: Storable- long integer size
by BrowserUk (Patriarch) on Oct 21, 2015 at 10:55 UTC

    Suggestion: Run a script under OSX that retrieves the data structure and dumps it to a file using Data::Dumper. Copy that file to Windows and eval it back to a data structure and then use Storable to freeze it back to a binary file.

    That will work around any incompatibilities and get you back to a working system that has all the benefits of Storable.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Storable- long integer size
by Your Mother (Archbishop) on Oct 21, 2015 at 12:26 UTC

    What BrowserUK said except just dump it to JSON with JSON::XS and keep it there. It's just as fast as Storable in most cases and it's always human readable and portable between machines and languages.

        5.6 support! Wow. Good to know about. Thank you.

Re: Storable- long integer size
by Anonymous Monk on Oct 21, 2015 at 14:05 UTC
    Storable is evil. It doesn't consistently thaw what it froze, even on the same machine. JSON (or YAML) is about the same size when compressed, and it is human readable.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1145536]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others rifling through the Monastery: (3)
As of 2024-04-24 05:45 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found