Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Re: DBIx returns question marks

by tinita (Parson)
on Aug 07, 2011 at 15:34 UTC ( #919095=note: print w/ replies, xml ) Need Help??


in reply to DBIx/DBI returns question marks

In addition to the other suggestions you might want to try:

use Devel::Peek; Dump $object->value;

You'll see the exact content and if the variable has the utf8 flag set or not.
Also, how do you check the database content? Is mysqldump showing the same for both tables?


Comment on Re: DBIx returns question marks
Download Code
Re^2: DBIx returns question marks
by stepamil (Acolyte) on Aug 07, 2011 at 16:36 UTC

    Hi All,

    thanks for excellent ideas I'll start working on them right now (in a slow sunday night way). Meanwhile, I've discovered that when I do this on dev machine:

    show variables like "%character%"; show variables like "%collation%";

    I get this:

    character_set_client utf8 character_set_connection utf8 character_set_database utf8 character_set_filesystem binary character_set_results utf8 character_set_server utf8 character_set_system utf8 character_sets_dir /usr/share/mysql/charsets/ ---------- collation_connection utf8_general_ci collation_database utf8_general_ci collation_server utf8_general_ci

    And on my staging machine is this:

    character_set_client utf8 character_set_connection utf8 character_set_database latin1 character_set_filesystem binary character_set_results utf8 character_set_server utf8 character_set_system utf8 character_sets_dir /usr/share/mysql/charsets/ ------------------ collation_connection utf8_general_ci collation_database latin1_swedish_ci collation_server utf8_general_ci

    That cannot be good. I'll play during the night and post my results in the morning.

    Thanks a lot, Stepamil

      Hi, me again,

      Thanks Corion for giving me this list. This one's a keeper :) I went through it and here's what I have:

      I've connected staging machine to use dev DB so both scripts are using the same DB. I've also removed DBIx from equation because I have the same behaviour on DBI, so I'll change the thread title shortly.

      Corion: Check the data format in your script as it writes to its output. Is it properly encoded from Unicode to the target encoding? Compare the hexdumps.

      Me: Dev -> var is UTF8, endoded:\x{7f16}\x{53f7} hex:e7bc96e58fb7, Stg -> var is not UTF8, var:??, hex:3f3f

      Corion: Check the data format in your script as it reads the data from the database. Is it utf8 everywhere? Compare the hexdumps.

      Me: No, the ?? variable is not UTF8 when I get it from staging machine

      Corion: Check the data format in your database in all tables. Are the tables/columns declared to be utf8/Unicode everywhere? Compare the hexdumps.

      Me: I'm using the same database, but different machine. Dev is using local DB and staging is using dev's DB. So that should be covered.

      Corion: Check the data format when it is written from your script to the database. Do they write utf8/Unicode everywhere? Compare the hexdumps.

      Me: Same as previous answer - same DB, different machine

      Corion: Check the data format from where your script gets its data. Is it properly decoded from the source to utf8 everywhere? Compare the hexdumps.

      Me: Also covered by previous answer

      Well, fighting with this continues. My more experienced colleague is coming from holiday this afternoon. He'll be fresh pair of eyes on this issue.

      Stepamil

        Are you sure that the libmysql versions between the two machines are the same? It seems to me that the two machines read different things from the database, which means that they either have problems due to one going over the network and the other being local, or due to being otherwise different versions.

        The issue is fixed!!!

        I've went through the thread once again and checked DBD::mysql. On dev was 4.018 version, but on stg it was 3.x. After the update everything worked perfectly. I just cannot belive that I somehow (while checking) missed to notice this.

        Thanks a lot for your help. It has been a tricky one for me, and, apart from learning the cause to this, I learned extra on utf8 difficulties.

        Stepamil

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://919095]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (10)
As of 2014-12-22 18:28 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (126 votes), past polls