-dlc

Thanks, dchetlin!

The mechanics for doing it with Data::Dumper are the following:

  my %h = (key1=>'value1',
        key2=>{key3=>'value3'});
 
  my $stringy = Dumper \%h;
  {
    no strict 'vars';
    %h = %{eval($stringy)};  
    die"$@" if $@;
  }
[download]

That eval there takes 0.25 seconds on my 17,000 elements hash. This is much better than the 0.47s for uudecoding and thawing of the same hash, but still five times more than the 0.05s of thawing that I would get if that peculiar $Application object worked right.

I just read the Data::Dumper docs. Indent has no effect on the correctness of the output. Only how readable it is to the human eye.

Purity and Terse are more related to allowing nested seralized data to be reconstructed. See the section "Configuration Variables or Methods" for the gory details.

Er ... what I said was:
• As a bonus, it will probably be more compact than Storable (as long as you set $Data::Dumper::Indent to 0).

I didn't mention correctness, I mentioned compactness -- with $Indent set, you get an extremely compressed string that will more often than not be smaller than Storable's binary encoding of the same data.

The drawback is of course that Storable will always be faster.

-dlc

uuencoding seems like a bit of overkill in order to drop the 8th bit. Here is a much simpler method (well, it is conceptually simpler but Perl's built-in uuencoding makes it harder to code in Perl) that might be much faster. It most likely (depending on the concentration of "8-bit characters") will write more bytes but that might not be much of a problem in comparison to the CPU usage of uuencoding. Then again, encoding perl.exe my way takes a ton fewer bytes than uuencoding does.

The concept is to pick two 7-bit characters, $quote7 and $quote8. $quote7 is used to "quote" these two characters. $quote8 is used to "quote" any "8-bit characters". So you replace any 8-bit character with $quote8 followed by that character with the 8th bit stripped. You preceed any occurrances of $quote7 or $quote8 with $quote7. I hope it is obvious how you reverse the process.

Here are the conversion routines wrapped in a test program (and it actually works):

#!/usr/bin/perl -w
use strict;

{
    my( $quote7, $quote8, %quote, %unquote );
    BEGIN {
        $quote7= pack "C", 0x7e;        # Any 7-bit char.
        $quote8= pack "C", 0x7f;        # Any _other_ 7-bit char.
        @quote{ $quote7, $quote8 }=
            ( $quote7.$quote7, $quote7.$quote8 );
        @quote{  map  { pack "C", $_ }  0x80..0xff  }=
                 map  { $quote8 . pack "C", $_ }  0..0x7f;
        %unquote= reverse %quote;
    }
    sub strip8 {
        my( $bin )= @_;
        $bin =~ s#([$quote7$quote8\x80-\xff])#$quote{$1}#go;
        return $bin;
    }
    sub restore8 {
        my( $str )= @_;
        $str =~ s#([$quote7$quote8].)#$unquote{$1}#gos;
        return $str;
    }
}

die "Usage: $0 file\n"   if  1 != @ARGV;
open IN, "<$ARGV[0]"  or  die "Can't read $ARGV[0]: $!\n";
binmode IN;
undef $/;
my $in= <IN>;
my $out= strip8( $in );
my $end= restore8( $out );
die "Well, that didn't work!"   if  $in ne $end;
[download]

I used the hashes in a quest for execution speed but since I didn't run benchmarks I can't guarentee that other methods aren't tons faster.

After running a modified version of tye's code (below) I was able to further push the thawing time for my 17K elements hash to 0.15s. This and the time it takes to move memory from $Application gives me a decent 0.26s total, which is 10x better than going to the database and well enough for our expected site traffic.

One big and a respectful bow towards tye!

{
  my( $quote7, $quote8, $zero, $one, %quote, %unquote );
  BEGIN {
    $quote7= pack "C", 0x7e;        # Any 7-bit char.
    $quote8= pack "C", 0x7f;        # Any _other_ 7-bit char.
    $zero = pack "C", 0x00; 
    $one = pack "C", 0x01; 
    @quote{ $quote7, $quote8, $zero }=
      ( $quote7.$quote7, $quote7.$quote8, $quote7.$one );
    %unquote= reverse %quote;
  }
  sub strip8 {
    my( $bin )= @_;
    $bin =~ s#([$quote7$quote8$zero])#$quote{$1}#go;
    return $bin;
  }
  sub restore8 {
    my( $str )= @_;
    $str =~ s#([$quote7$quote8].)#$unquote{$1}#gos;
    return $str;
  }
}
[download]