Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Re^2: SQLite and Storable: is "Text" okay?

by ikegami (Pope)
on Oct 06, 2010 at 15:53 UTC ( #863825=note: print w/replies, xml ) Need Help??

in reply to Re: SQLite and Storable: is "Text" okay?
in thread SQLite and Storable: is "Text" okay?

If you're using SQLite 3, the TEXT type stores data UTF encoded

I don't see SQLite performing any encoding conversion, so that's surely false. It stores whatever bytes it received, whether it's encoded using UTF-8 or not.

The difference will surface when someone tries to perform text operations (e.g. sorting) on the field.

BLOB is definitely more appropriate here, seeing as the data is binary, but it's my understanding that the type of a column doesn't matter in SQLite if all you do with the column is store and fetch.

If it does do encoding conversions, you need to stay away from TEXT for Storable data. You'll potentially waste space and time.

I would still pick TEXT, because it describes the data the best.

Storable returns binary data. BLOB should be picked because it describes the data best.

Replies are listed 'Best First'.
Re^3: SQLite and Storable: is "Text" okay?
by assemble (Friar) on Oct 06, 2010 at 17:11 UTC

    The way that I read the Storable doc, it seemed to indicate that it stored data in UTF8. Looking at it again, it only indicates that strings are stored that way. In light of that fact, BLOB would definitely be the right choice.

    It looks like SQLite has started to actually make different data types matter recently. I started using it in version 2, when it said "everything is stored as string anyway". The version 3 docs don't make any such claims, but the main differences do appear to be with non store-fetch operations.

      The way I read it, SQLite values are like Perl scalars. Stored in a convenient format, and coerced appropriately when required (by comparisons). The storage format is based on the value itself, whereas the coercion is based on the column type.

      it only indicates that strings are stored that way.

      Whatever it indicates, it's doesn't do that. It stores the string as it appears in the scalar.

      $ perl -MStorable=freeze -e' utf8::upgrade( $_="abc\x{80}def" ); print freeze(\$_); ' | od -c 0000000 004 \a 004 1 2 3 4 004 004 004 \b 027 \b a b +c 0000020 302 200 d e f 0000025 $ perl -MStorable=freeze -e' utf8::downgrade( $_="abc\x{80}def" ); print freeze(\$_); ' | od -c 0000000 004 \a 004 1 2 3 4 004 004 004 \b \n \a a b +c 0000020 200 d e f 0000024

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://863825]
[1nickt]: Hm, that is, how to prevent Perl from turning 1.0 into 1? I feel sure this must be an faq, but am reading perlnum and not finding the answer ...
[Lady_Aleena]: 1nickt, why does it matter in this case?
[1nickt]: Hm, perhaps this? "Operators which expect an integer force the argument into the integer format."
[1nickt]: Can this be? So print
[1nickt]: ... "expects an integer" ?

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (12)
As of 2017-05-24 18:35 GMT
Find Nodes?
    Voting Booth?