Re^3: When should a wheel be reinvented

[demerphq]

BTW, Huffman coding is a lousy

I think I know why you say that, but Id like to know more...

Also, it can be fun: A Golf question maybe? Huffman with a twist, New Huffman Obfus, A Brainteaser -- Huffman with a twist, New Huffman Obfus, Huffman encoder, Huffman decoder

Actually I solved the "with a twist" problems long ago, and in linear time iirc, but for some reason I never got around to following up on them. Maybe ill dig out the solution and post it now that you have reminded me.

[ysth]

Huffman coding (with a translation table tailored to the data) guarantees the least number of generated bits possible while encoding each character (or whatever the input unit is) separately. But in most data, there is some correlation of each character to the ones around it that can be used to achieve better compression.

[demerphq]

Right. Also transmitting the table is inefficient as well (although there is always adaptive huffman encoding, which i never got around to implementing). But you may find LZW Demystified and RFC: Compress::LZW interesting. :-)

Actually in my experiments my implementation is more efficient than straight LZW because it doesnt lose bits by packing things into words. But that also means its not binary compatible which makes it more or less useless.