use utf8;
sub ff {
  die if utf8::is_utf8($_[0]);
}
ff(asd => 1);
[download]

I'd expect both of the following calls to output the same thing, so I think it's a bug.

use utf8;
sub ff {
  print utf8::is_utf8($_[0]) ? 1 : 0, "\n";
}
ff(asd => 1);  # 1
ff("asd");     # 0
[download]

Tested in 5.8.8 and 5.10.0.

After some investigation, barewords in general (anything parsed by force_word) are affected.

use utf8;
sub ff { print utf8::is_utf8($_[0]) ? 1 : 0, "\n"; }

ff("asd");               # 0
ff('asd');               # 0
ff(qw(asd));             # 0

ff(asd => 1);            # 1
{ no strict; ff(asd); }  # 1
ff(-asd);                # 1
ff(asd::);               # 1
[download]

I think function names (including built-ins) and keywords are also affected.

Gotta go. No time to devise a fix atm.

encoding produces the opposite result.

#use encoding 'UTF-8';  # or 'utf8'
#use utf8;

use Encode qw( is_utf8 );
sub ff { print is_utf8($_[0]) ? 1 : 0, "\n"; }

                         # none enco utf8 both
                         # ---- ---- ---- ----
ff("asd");               # 0    1    0    1
ff('asd');               # 0    1    0    1
ff(qw(asd));             # 0    1    0    1

ff(asd => 1);            # 0    0    1    1
{ no strict; ff(asd); }  # 0    0    1    1
ff(-asd);                # 0    0    1    1
ff(asd::);               # 0    0    1    1
[download]

It cost us a lot of blood and sweat to debug why some perfectly ASCII strings would suddenly get the flag.

Does this mean you were using the utf8 flag to determine whether or not a string contains wide characters? That is not what the flag is for, and you shouldn't be using it that way. To test for wide characters in a string, use a regex:

     if ( /[^[:ascii:]]/ ) { ... }

# which is equivalent to

     if ( /[^\x00-\x7f]/ ) { ... }
[download]

Yes, it was a wrong way to do that -- an attempt based on wrong guess that perl would not set utf8 flag on ASCII strings.
--kap

use strict;
use warnings;
my %hash = ( asd => 1 );

sub ff {
  print utf8::is_utf8($_[0]) ? 1 : 0, "\n";
}

eval {
    use utf8;
    ff(%hash); # now prints 0
};
[download]

I suppose it's reasonable that perl encodes barewords as utf8 if you use utf8 even if only ascii characters are involved.

Update: from the utf8 documentation:

The "use utf8" pragma tells the Perl parser to allow UTF-8 in the pro-
gram text in the current lexical scope (allow UTF-EBCDIC on EBCDIC
based platforms). ...

Do not use this pragma for anything else than telling Perl that your
script is written in UTF-8. The utility functions described below are
useful for their own purposes, but they are not really part of the
"pragmatic" effect.
[download]

We actually use lots of non-ASCII strings so we need use utf8.
--kap

Out of curiosity, why does the flag bother you?

We are in the process of converting a huge production codebase from 5.6 + Unicode::String to 5.8. This is as painful as it can get and we log this flag in a lot of places just to find codepaths that need attention.
--kap

Does anyone know what the performance penalty of utf8 hash keys -- even if they contain only ASCII chars -- is?