Re: Perl and Postgresql: Invalid byte sequence for encoding "UTF8"

[ides]

I ran into this recently myself with a project to insert E-mail messages into a PostgreSQL database. I solved it like so:

 
use utf8; 
use Encode; 
my $possibly_bad_utf8_data = get_data(); 
my $good_data = encode( "UTF-8", $possibly_bad_utf8_data );

The capitalization of the UTF-8 in the call to encode() is important, it tells encode to be "strict" about the UTF-8. It turns out use utf8; isn't 100% strict and hence can't be inserted into a strict mode PostgreSQL database.

Hope this helps.

[StoneTable]

Remarkable, thanks!

I had tried using Encode before, but missed the "UTF-8" bit apparently. It's working perfectly now.

[Anonymous Monk]

It is NOT the capitalization that is needed. Encode is case-insensitive for the encoding. It is the hyphen that makes the difference, see this example:

use Encode qw(resolve_alias);

my @aliases = ('utf-8', 'UTF-8', 'utf8',);

for my $alias ( @aliases ) { 
my $canonical_name  = Encode::resolve_alias($alias);
print "$alias \t has canonical name $canonical_name\n";
}