comment on

Would it help to convert your decimal values to a higher base, e.g., base 64? Consider the following:

use Modern::Perl;
use Math::BaseCnv;

my $original = <<END;
17077.161362,-595,2619,-15202
17077.174405,601,2115,-14859
17077.187416,-816,1894,-15539
17077.200402,210,1790,-15284
17077.213445,1560,1721,-15457
17077.226432,1408,1483,-15834
17077.239474,947,1161,-17531
17077.252517,689,1396,-17515
17077.265504,-861,1206,-18238
17077.278547,-493,1369,-17951
17077.291589,850,1294,-17360
17077.304576,1110,1215,-17271
17077.317619,326,1055,-13948
17077.330662,1980,1083,-16239
17077.343648,1295,1057,-16154
17077.356691,-1302,1207,-16277
17077.369734,-1986,1216,-16537
17077.382721,-134,1108,-17409
17077.395763,-739,1059,-16734
17077.408774,-2262,1006,-16690
17077.421761,-807,1174,-17046
17077.434803,-591,1081,-17615
17077.447846,-2080,-5757,-21463
17077.460833,-209,-17608,-19409
END

my @original = split ' ', $original;

say for @{ b10to64( \@original ) };

sub b10to64 {
    my $original = $_[0];
    chomp @$original;

    for ( my $i = 0 ; $i < @$original ; $i++ ) {
        my @entries = split ',', $$original[$i];
        my ( $whole, $decimal ) = split '\.', $entries[0];
        $entries[0] = cnv( $whole, 10, 64 ) . '.' . cnv( $decimal, 10,
+ 64 );
        $entries[$_] = cnv( $entries[$_], 10, 64 ) for 1 .. 3;
        $$original[$i] = ( join ',', @entries );
    }

    $original;
}
[download]

Output:

4Ar.dPI,-9J,ex,-3jY
4Ar.gb5,9P,X3,-3eB
4Ar.jmO,-Cm,Tc,-3op
4Ar.mxI,3I,R.,-3kq
4Ar.q75,OO,Qv,-3nX
4Ar.tI0,M0,NB,-3tQ
4Ar.wTo,Ep,I9,-4Hx
4Ar.zfb,An,Lq,-4Hh
4Ar.10qW,-DT,Is,-4S.
4Ar.140J,-7j,LP,-4OV
4Ar.17C5,DI,KE,-4FG
4Ar.1AN0,HM,I_,-4Dt
4Ar.1DYp,56,GV,-3Py
4Ar.1Gkc,Uy,Gx,-3zl
4Ar.1JvW,KF,GX,-3yQ
4Ar.1N5J,-KM,It,-3.L
4Ar.1QH6,-V2,J0,-42P
4Ar.1TS1,-26,HK,-4G1
4Ar.1Wdp,-BZ,GZ,-45U
4Ar.1Zp6,-ZM,Fk,-44o
4Ar.1c.1,-Cd,IM,-4AM
4Ar.1g9p,-9F,Gv,-4JF
4Ar.1jLc,-WW,-1Pz,-5FN
4Ar.1mWX,-3H,-4J8,-4lH
[download]

Conversion artifact:

4Ar.1c.1,-Cd,IM,-4AM
      ^
      |
      + - split on first '.' when converting back to base 10
[download]

This yielded a 30% reduction size of your data in accel.csv, which may help with your final data size after packing and compression.

Update: Evidently, this would not help...

In reply to Re: Specialized data compression by Kenosis
in thread Specialized data compression by wanna_code_perl

Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
Want more info? How to link or How to display code and escape characters are good places to start.


We don't bite newbies here... much
	PerlMonks