If you are content with not having proper Perl data structures but a raw blob of memory, a good approach to sharing that data among processes (and threads) could be File::Map. This gives you a scalar region in which you can read/write shared data using substr.
If this falls into "serializing/unserializing everything all the time" territory for you, then there is little you can do about it. Perl data structures are reference counted and you will have to do some very hacky and specialized things to share a Perl data structure between processes. I can imagine that you might be able to create "shared" scalars that point to the shared region via newSV_pv(..., 0). This would tell the Perl memory manager that you want to maintain the storage space allocated to the scalar yourself. But writing to that scalar needs to be done through a special routine that makes sure the sizes still fit - you can't grow or shrink the length of the scalar for example.