Re: Bizarre Dancer encoding behavior (dancer assumes unicode

[Anonymous Monk]

Hmm, backticks ie qx// return bytes not unicode ... you have to decode the bytes you get from backticks to get unicode that dancer can return

Dancer assumes you're returning unicode, so it encodes the bytes ... thus double-encoding

Solution is simple as perlunitut: Unicode in Perl#I/O flow (the actual 5 minute tutorial) teaches, decode external data, then dancer will encode it for you

The client , note how the bytes match the server output

#!/usr/bin/perl --
use strict; use warnings;
use WWW::Mechanize;
my $ua = WWW::Mechanize->new;
for my $url(qw[ http://localhost:3000/unicode http://localhost:3000/by
+tes http://localhost:3000/unibyte   ]){
    $ua->get($url);
    DD($ua->res->as_string);
}
sub DD { use Data::Dumper; print Data::Dumper->new([@_])->Useqq(1)->Du
+mp, "\n"; }
__END__
$VAR1 = "HTTP/1.0 200 OK\nServer: Perl Dancer 1.3118\nContent-Length: 
+6\nContent-Type: text/html; charset=UTF-8\nClient-Date: Mon, 21 Jul 2
+014 02:56:39 GMT\nClient-Peer: 127.0.0.1:3000\nClient-Response-Num: 1
+\nX-Powered-By: Perl Dancer 1.3118\n\n\320\224\320\224\320\242\n";

$VAR1 = "HTTP/1.0 200 OK\nServer: Perl Dancer 1.3118\nContent-Length: 
+12\nContent-Type: text/html; charset=UTF-8\nClient-Date: Mon, 21 Jul 
+2014 02:56:39 GMT\nClient-Peer: 127.0.0.1:3000\nClient-Response-Num: 
+1\nX-Powered-By: Perl Dancer 1.3118\n\n\303\220\302\224\303\220\302\2
+24\303\220\302\242\n";

$VAR1 = "HTTP/1.0 200 OK\nServer: Perl Dancer 1.3118\nContent-Length: 
+6\nContent-Type: text/html; charset=UTF-8\nClient-Date: Mon, 21 Jul 2
+014 02:56:39 GMT\nClient-Peer: 127.0.0.1:3000\nClient-Response-Num: 1
+\nX-Powered-By: Perl Dancer 1.3118\n\n\320\224\320\224\320\242\n";

The server , note the DDumper of the bytes, search for it in the client output

#!/usr/bin/perl --
use utf8;
use Dancer;
use Encode qw/ encode decode /;

sub DD { use Data::Dumper; print Data::Dumper->new([@_])->Useqq(1)->Du
+mp, "\n"; }

config->{charset} = 'UTF-8';

my $unicode = "\x{414}\x{414}\x{422}"; ## q{ДДТ};
my $bytes   = encode('UTF-8', $unicode);

DD( $unicode, $bytes, encode('UTF-8', $bytes) );

get '/unicode' => sub { return $unicode  };
get '/bytes'   => sub { return $bytes };
get '/unibyte' => sub { return decode('UTF-8', $bytes ); };

dance;
__END__
$VAR1 = "\x{414}\x{414}\x{422}";
$VAR2 = "\320\224\320\224\320\242";
$VAR3 = "\303\220\302\224\303\220\302\224\303\220\302\242";

>> Dancer 1.3118 server 300 listening on http://0.0.0.0:3000
== Entering the development dance floor ...
Terminating on signal SIGINT(2)