XP is just a number PerlMonks

### Re: Specialized data compression

by salva (Abbot)
 on Sep 11, 2012 at 09:28 UTC ( #992967=note: print w/replies, xml ) Need Help??

in reply to Specialized data compression

The x,y,z acceleration values in your sample file are highly correlated on time, so using delta values for them should also improve their compressibility.

Then, you can divide the x,y,z deltas by the time deltas in order to get the derivate of the acceleration vector in order to get even more normalized values. The problem is that at this point you are using floats instead of integers, but unless you want absolute precision this should be a minor issue. Just select a resolution that fits your problem and discretize them accordingly.

Another possibility is to express the x,y,z deltas (or derivates) in the coordinate system defined by the Frenet-Serret frame. That would probably produce numbers that are nearer to zero and so, more compressible.

Replies are listed 'Best First'.
Re^2: Specialized data compression
by wanna_code_perl (Pilgrim) on Sep 16, 2012 at 16:37 UTC

These suggestions were helpful. Thank you! Using deltas for x,y,z did help quite a bit.

The time delta derivative approach was interesting to think about, but would basically break the lossless compression I have going. At the moment, at least, I need to be able to reconstruct the original input verbatim, and the only way to do that is with fixed point arithmetic.

The one concession I was able to safely (verifiably) make is quantizing the time deltas that are close to the sensor frequency. The clock has a large (+/- 15%!) but predictable jitter pattern I was able to isolate and then basically RLE the result.

With the latest optimizations, I'm sitting at about 9:1 compression, which is acceptable. I was shooting for 10:1, but that last 33 KiB won't come easy.

Create A New User
Node Status?
node history
Node Type: note [id://992967]
help
Chatterbox?
 [Corion]: So Hyperboria is an envrypted, decentralized IPv6 mesh network. I'm interested and want to look how good coverage is in my area. But the API for querying Hyperboria is only reachable over Hyperboria itself. Thanks but no thanks.

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (3)
As of 2017-06-23 07:25 GMT
Sections?
Information?
Find Nodes?
Leftovers?
Voting Booth?
How many monitors do you use while coding?

Results (536 votes). Check out past polls.