Re^7: Looking for alternative for IPC::Shareable (or increase size)

[marioroy]

Hi, jcb

I updated the examples to output the frozen length. MCE::Channel does not enable compression when using Sereal.

> 1 billion chars (2^30 1073741824)

Storable
    runtime : 2.106 seconds
    memory  : 4,102 megabytes

Sereal
    runtime : 1.549 seconds
    memory  : 3,078 megabytes
  
Sereal with compression enabled
    runtime : 2.170 seconds
    memory  : 2,104 megabytes

Thank you, DomX and jcb. ++ PerlMonks for this forum.

Humble regards, Mario

[jcb]

It seems obvious that the version using Storable is holding an extra copy of the data somewhere compared to the version using Sereal. The compression in Sereal is working reasonably well, reducing 1G of highly compressible input to about 52M stored.

I still wonder if there really is a meaningful difference here — it should be equally possible to compress a frozen Storable image using any of the Compress:: modules to save memory or disk or network time, although it does appear that Sereal may have a slightly more efficient implementation (avoiding that extra copy of the data) that could be considered for adaptation to improve Storable as well. Maybe submit an enhancement request at the bug tracker for Storable?