http://www.perlmonks.org?node_id=539093

eXile has asked for the wisdom of the Perl Monks concerning the following question:

esteemed monks,

a mysql insert does some silent conversion of values it inserts into a row, like inserting a string > 255 chars into a column that only takes 255 chars, will insert the first 255 chars. I'd like to know how to detect it when that happens. I've tried looking in various DBI attributes, and mysql specific info, but couldn't find anything that warns me about when this truncating happens:

#!/usr/bin/env perl use strict; use DBI; my $dbh = DBI->connect('a','b','c',{RaiseError => 1, PrintWarn => 1, P +rintError => 1}); my $val = 'x' x 254 . 'y' x 50; my $result; $result=$dbh->do("insert into requestattribute values ('',150,'test123 +','$val')"); print "result: '$result'\n"; print "mysql_info: ". $dbh->{'mysql_info'} . "\n"; print "mysql_errno: ". $dbh->{'mysql_errno'} . "\n"; __DATA__ result: '1' mysql_info: mysql_errno: 0
I've written wrappers around this in the past (ie. fetch values from database and compare these to what was originally meant to be inserted), but there must be a better way. Another way I could handle this is to compare rows values with the table specification beforehand, but both methods involve me doing work that the database should be able to do much better (since it is doing the conversion).

Am I missing something that allows me to easily get some kind of error/warning when this truncating happens? If not, how do other people handle this?

Replies are listed 'Best First'.
Re: DBI mysql truncation best practice?
by Melly (Chaplain) on Mar 24, 2006 at 20:50 UTC
Re: DBI mysql truncation best practice?
by perrin (Chancellor) on Mar 24, 2006 at 21:48 UTC
    Use MySQL 5 and turn on strict SQL mode.
Re: DBI mysql truncation best practice?
by CountZero (Bishop) on Mar 24, 2006 at 22:25 UTC
    Fom the MySQL 5.x docs:
    If you assign a value to a CHAR or VARCHAR column that exceeds the column's maximum length, the value is truncated to fit. If the truncated characters are not spaces, a warning is generated. For truncation of non-space characters, you can cause an error to occur (rather than a warning) and suppress insertion of the value by using strict SQL mode. See Section 5.2.5, “The Server SQL Mode”. (11.4.1. The CHAR and VARCHAR Types)

    CountZero

    "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

Re: DBI mysql truncation best practice?
by nmerriweather (Friar) on Mar 25, 2006 at 00:10 UTC
    As stated above, strict mode is the only way for it to happen (mysql 5 only)

    <quote>"but both methods involve me doing work that the database should be able to do much better (since it is doing the conversion"</quote> that's why i switched to postgres. mysql4 has no way to stop that from happening, and throws no warnings. mysql5 has the strict mode, but it can be overridden unless you're a beast about security policy on the server ( which is annoying, as mysql starts with a very loose policy, and makes you tighten most every option )

    the error messages from mysql are fairly worthless too - they're mostly error codes with a generic descriptor. its impossible to bugfix without googling their online documentation. on the other hand, postgres throws verbose errors with detailed descriptions.

    a lot of people love mysql, i used to be one of them. then 3gbs of data got destroyed by truncation, bad foreign key support, and i went crazy trying to decipher the server messages. now i'm in the postgres camp and completely love it.

    if you're on mysql4, consider switching to postgres (since you'd have to install a new db anyways). its got a steeper learning curve, but its totally worth it.

    as far as the speed differences between the two - unless you're getting slashdotted , you shouldn't really notice anything.