Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??
Many thanks for all the responses, especially I0 for the brilliant snippet of code and bart for the clear explanation of the benefits of pack with the "w" template. All of the responses were valuable; they really improved my understanding. I have implemented a solution along these lines and now I only need about 26% as much memory as I did with the ascii dataset. I am delighted with this solution!

With this solution, I expect to save about 1.5 million numbers in about 2 megabytes of storage. Wow!

Here is the proof of concept code that I ended up with:

use Storable; use File::Flat; my @to_save; my $nrows=10000; my $rowmax=300; for (0..$nrows) { my @r; $r[0]=int(rand(20)); for (1..int(rand($rowmax))) { push @r, $r[$_-1] + 1 + int(rand(200)); } push @to_save, join ',',@r; } my $in=""; foreach( @to_save ) { $in .= $_."\n"; my @a = sort {$a<=>$b} split/,/; my $d=0; for( @a ){ ($d,$_)=($_,$_-$d); } $_ = pack'w*',@a; } open(TEXT, ">array.txt") or die "Can't open output file"; print TEXT $in; close TEXT; store (\@to_save, 'array.sto'); my $to_retrieve= retrieve ('array.sto'); my $out=""; foreach( @$to_retrieve ) { my @a = unpack'w*',$_; my $c = 0; foreach my $n (@a) { $c += $n; $out .= $c.","; } chop $out; $out .= "\n"; } if ($in eq $out) { print "OK\n" } my $sz_in= File::Flat->fileSize('array.txt') or die "Can't size array.txt"; my $sz_out= File::Flat->fileSize('array.sto') or die "Can't size array.sto"; print "Final size = ",int(100*$sz_out/$sz_in+0.5),"% of input\n";
It should work perfectly the first time! - toma

In reply to Compressing a set of integers - Solution by toma
in thread Compressing a set of integers by toma

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others romping around the Monastery: (5)
As of 2024-05-18 15:55 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found