Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options

Comment on

( #3333=superdoc: print w/replies, xml ) Need Help??
DBIx::Log4perl sees the data before it is sent to the database by DBD::Pg so you cannot rely on what you see in its output as DBD::Pg can change the data.

In theory, yes, but practically Log4perl was logging as expected a line before the SQL statement and the output of DBIx makes sense for the problem I described. If I encode the data to WIN1252 before passing it to DBI the logged output of DBIx change as well, therefore my strong guess that DBIx is logging what gets send to the database.

Are you binding the data as parameters when it is inserted?

No, the application simply creates a SQL string by concatenating different UTF-8 Perlstrings together, pushes that with a "do" and the whole string formed by SQL command and values gets encoded to UTF-8.

What is your postgres client charset set to

It's automatically detected as WIN1252 when the problem occurs, which makes perfectly sense on Windows and the Linux I tested. Another Ubuntu server automatically detects it as UTF-8 instead, because it has an UTF-8 locale. If I manually change it to UTF-8 everything works as expected, the database server properly encodes UTF-8 bytes to WIN1252 characters in the target database.

and what is the encoding of the data you pass to DBD::Pg when it fails

Valid UTF-8 Perlstrings with UTF-8 flag turned on.

and do you have pg_enable_utf8 on?

No, because from my understanding it is deprecated and only used in reading from the database, not writing to.

In reply to Re^4: DBD::Pg encodes Perlstring to UTF-8 bytes instead of WIN1252 regardless client encoding by Pickwick
in thread DBD::Pg encodes Perlstring to UTF-8 bytes instead of WIN1252 regardless client encoding by Pickwick

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and all is quiet...

    How do I use this? | Other CB clients
    Other Users?
    Others wandering the Monastery: (4)
    As of 2018-02-26 04:33 GMT
    Find Nodes?
      Voting Booth?
      When it is dark outside I am happiest to see ...

      Results (316 votes). Check out past polls.