Re^3: Unpacking and converting

[andal]

Well. Probably you are looking the wrong way. If your programm directly connects to DB using DBI or alike, then DBI takes care of all necessary optimizations. Your attempts to convert strings to numbers just make system less efficient. If your programm simply passes data from one server to another, where another program picks it up, then again it makes sense simply take the text, zip it (or bzip2 it :) and then copy to remote. Probably rsync with option -z would be best for this.

The point is. Conversion of perl variable from text to number does not add any compactness.

The other point is. Iterating over elements of array using foreach is faster.

[dwalin]

Probably you are looking the wrong way.

Yes, I came to the same conclusion. In my case, I will be able to use SSH with compression and this indeed solves all my pains; somehow it wasn't the first choice for me. I was dumbstruck probably, as it seems obvious in hindsight.

For consistency sake though, I'd like to mention that conversion from text to number does indeed add compactness to serialized data. Consider this:

use Storable qw(freeze);

my @a = '0001'..'1000';
my $foo = freeze \@a;
$_ += 0 for @a;
my $bar = freeze \@a;
print "before: ", length $foo, ", after: ", length $bar, "\n";

before: 6016, after: 4635

I fail to understand why so many people are insistent on ignoring the obvious. Granted, today's fast and abundant hardware resources may have spoiled us but there are situations yet when every byte counts. Make the dataset in above example three orders of magnitude larger and the difference becomes quite distinct.

Regarding array iteration, I feel this discussion was beneficial for me as it cleared some murky points. I wish all my questions were answered so productively in future. :)

Regards,
Alex.

[andal]

For consistency sake though, I'd like to mention that conversion from text to number does indeed add compactness to serialized data. I fail to understand why so many people are insistent on ignoring the obvious.

You fail to understand it because you don't understand what is "serialized data" and what is "conversion from text to number". Serialized data is something stored in sequential area (file, memory chunk etc.) The "conversion" is something that is done at run time to provide context specific information. Perl does the conversion internally and automatically and you normally don't need to think about it. Conversion to or from numbers does not necessary save any RAM, since you don't know if the perl has discarded the string buffer, or keeps it around for the sake of speed optimization when the string is needed again.

The "serialization" is needed when you copy the data from perl into file (for example). In this case, to save space you may use function "pack". Here's the piece of code that stores '00000001' as single character in the file.

print FILE pack('C', '00000001');

Again, perl converts from string to number automatically and stores in the buffer single byte with value 1.

I hope this clarifies for you why explicit conversion from string to number is a nonsense in general. Though it might be needed in certain cases, but definitely not for decreasing RAM usage :)