Creating Storable objects from the C

[zwon]

Hi Monks!

We need to pass some data structures from C program to Perl. Structures may be rather complex. XML is good way to describe these structures, but it's too slow for this task, so I'm trying to find another way. One of my thoughts is about Storable, if we could create Storable object in C program it may solve our problem. So I'm wondering if there's some library which I could use from the C program to create Storable.

A Merry Christmas to All!

[BrowserUk]

You're looking at the wrong tool. Storable serialises Perl data structures. Even if a C-callable Storable library existed, you'd have to convert your C-structure into a Perl data structure first before you could use it.

Most relative simple C-structs--ie. those that do not contain pointers--are most easily and efficiently accessed from Perl using unpack. If the structure uses pointers, then serialising it to some text format, is probably far easier than converting it to Perl's internal format.

[tilly]

Writing a Storable file from C does not sound like an easy solution to me. Here are alternate ways to solve this problem. I am assuming throughout that there is some C library you have that provides an interface into this data structure. If it doesn't exist yet, you should be able to build one fairly easily.
1. Use a standard data exchange format such as XML or YAML. As you have discovered, this can be slow.
2. Come up with a format for the data and use pack/unpack to pass data between C and Perl (great if it works but it won't always work).
3. Write an XS interface for a C library that has the complete specification for the data structure then access your C library directly from Perl. This is a lot of work but should be very fast.
4. Use swig to generate an interface into your C library. This actually builds an XS interface. I don't hear about people doing this much for Perl, and in my one experience with such an interface we found some performance silliness that we cleaned up, but it may save you some work.
5. Use Inline::C in Perl to access your data. This is simpler to set up than the other versions, is fast (except the first time you call it), but requires a C compiler and some (fairly minimal) C knowledge from your Perl developers.
From your question I suspect that you're using Storable. Be warned that the direct C solutions will not necessarily play well with Storable if your Perl developers are using that. The solution to that is to build STORABLE_freeze and STORABLE_thaw hooks that serialize and deserialize your C data structures into a string that Storable can use.

[zwon]

Thanks, I suspected that there's no possibility to create Storable from C, but just wondered if it is. Yes, XML and YAML are too slow, but ikegamy just pointed me to JSON and it looks better, so maybe I will use it. Also I thought about XS, but this will require to design some format to transmit data and implement encoder/decoder, so I fear it would be just another *ML, maybe even not faster ;).

[ikegami]

There are already C libraries for JSON and YAML, I believe. Implementing Storable in C sounds like a lot of unnecessary work if it hasn't been done already.

[Your Mother]

Just adding to this JSON::XS was faster than Storable in the last benchmark I saw/ran. Benchmarks can be misleading (size, data, versions) but even if it's not faster it's probably roughly as fast and if there are C libs available it would be easy to see if it worked well for you.

[diotalevi]

In the last benchmark I ran, Storable was faster than JSON::XS, fwiw.