Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re: PgPP invalid byte seqnence for encoding UTF8

by McA (Curate)
on Mar 24, 2013 at 14:38 UTC ( #1025149=note: print w/ replies, xml ) Need Help??


in reply to PgPP invalid byte seqnence for encoding UTF8

Hi,

what does the psql driver expect as input? A UTF-8 encoded string?

When you do the following:

#!/usr/bin/perl use warnings; use strict; use Data::Dumper; use JSON; my $json = q({"my": "En d\u00e9mocratie, on a le droit d'avoir tort"}) +; my $perl = decode_json $json; my $flag_utf8 = utf8::is_utf8($perl->{'my'}) ? 1 : 0; print "is unicode: $flag_utf8\n"; print Dumper($perl), "\n";
you see, that after decoding JSON you get probably strings which are marked as unicode. As soon as you are using "output" of these strings (a database is one of the many output channels) you have to asked which encoding is expected. That means: If psql expects UTF-8 encoding you have to encode the string appropriately with Encode::encode('UTF-8', $string);

McA


Comment on Re: PgPP invalid byte seqnence for encoding UTF8
Select or Download Code
Re^2: PgPP invalid byte seqnence for encoding UTF8
by cormanaz (Chaplain) on Mar 24, 2013 at 16:22 UTC
    That did take care of the encoding error tho I don't understand why because I thought Perl was natively UTF-8. Anyway it works.

    But now when I try to use that value to update a record like so:

    my $sth = $dbh->prepare("update userinfo set description=? where uid=$ +uid"); $sth->execute(encode('UTF-8',$userdata->[$i]->{description})) || die $ +sth->errstr;
    I get DBD::PgPP::st execute failed: ERROR:  array value must start with "{" or dimension information at character 33

    This is confusing as the string is En démocratie, on a le droit d'avoir tort and character 33 is either an "a" or a "v" depending on whether it's counting form zero.

      Hi,

      I'm pretty sure it has nothing to do with encoding but with the way the related columns are defined. Please, show use your table definition of userinfo.

      McA

        Here it is

        Table "public.user info" Column | Type | Modifiers | Storage | Description ------------------+------------------------------- +-----------+----------+------------- uid | integer | not null | plain | screen_name | character varying | not null | extended | user_name | character varying | not null | extended | location | character varying | | extended | user_language | character varying | | extended | favourites_count | integer | | plain | followers_count | integer | | plain | friends_count | integer | | plain | status_count | integer | | plain | listed_count | integer | | plain | created_at | character varying | | exten ded | time_zone | character varying | | exten ded | user_url | character varying | | exten ded | description | text[] | | exten ded | added_on | timestamp without time zone[] | | exten ded | Indexes: "pk_user" PRIMARY KEY, btree (uid) Rules: my_table_on_duplicate_ignore AS ON INSERT TO userinfo WHERE (EXISTS ( SELECT 1 FROM userinfo WHERE userinfo.uid = new.uid)) DO INSTEAD NOTHING Has OIDs: no
        Maybe the problem is the brackets after the text type? Those were inserted when I created the column with pgadminIII.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1025149]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (15)
As of 2014-08-27 19:53 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (251 votes), past polls