comment on

It is more difficult to explain this in ODBC. The rt poster was running on Windows (I mention that because things differ slightly on UNIX e.g. locales). In ODBC there is the ANSI API and the Wide API - the latter supports UCS2 - these affect calls like SQLPrepare where you can pass SQL in unicode encoded in UCS2 . Then when binding to a column to fetch data you must name the type of column at bind time and in his case as the column was varchar it was bound as a SQL_CHAR (one byte = one chr). There is no way (in ODBC) to say I'm binding it as SQL_CHAR but can you return it to me in somesort of encoding or character-set (unless perhaps you change your SQL and that is DB dependent). However, if his column had been nvarchar DBD::ODBC would have bound it as SQL_WCHAR which is a wide UCS2 encoded chr and all is well.

Each database client lib, ODBC driver etc has different ways of defining the local character-set and there is nothing in ODBC to say what it is. In addition, the ODBC specification does not work properly with variable length character encodings e.g. UTF8 as some of the APIs (SQLGetData) use the filling of the provided buffer to indicate something and that could mean chopping a UTF8 encoded chr off part way through.

Even if you could request the bound column data to be returned in a particular chrset or encoding (and you can as you say in some cases) what is DBD::ODBC to do with it as it has no idea what that chrset or encoding is unless it performs DB-specific SQL on every query to query the possibly per-column chrset.

If the bound data is not returned UCS2 encoded unicode chrs then DBD::ODBC cannot guess anything and it is up to the Perl script. However, DBD::ODBC already has a flag for the data returned from the db is UTF-8 encoded (for some derivative of Postgres) and it decodes it - so I thought I could combine this and the rt to allow the script to specify an encoding.

Add to that the fact that the unicode additions to ODBC are not even in THE spec as passed to X/Open - they are a Microsoft thing added afterwards.

In reply to Re^4: How to call Encode::decode from Perl XS by mje
in thread How to call Encode::decode from Perl XS by mje

Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
Want more info? How to link or How to display code and escape characters are good places to start.


"be consistent"
	PerlMonks