http://www.perlmonks.org?node_id=11101489

Lady_Aleena has asked for the wisdom of the Perl Monks concerning the following question:

Hello everyone. I am having a problem with the character Ĉ in the string Ĉon Flux not being captured by the first_alpha subroutine below. A space character is being returned. This problem arose when I recently re-encoded my files from Windows-1252 to UTF-8. I am baffled. As always, I appreciate all the help I get.

sub first_alpha { my $alpha = shift; $alpha = ucfirst($alpha) if $alpha =~ /^\l./; $alpha =~ s/\s*\b(A|a|An|an|The|the)(_|\s)//xi; if ($alpha =~ /^\d/) { $alpha = '#'; } elsif ($alpha !~ /^\p{uppercase}/) { $alpha = '!'; } else { $alpha =~ s/^(.)(\w|\W)+/$1/; } return $alpha; }

Even substr('Ĉon Flux', 0, 1) returns a space character.

The weird thing that I found is that the string Ĉon Flux is returned when I ran the data from the file through my make_hash subroutine then ran that hash through my alpha_hash subroutine, both in the same module as first_alpha. (You can see the full module here.)

sub make_hash { my %opt = @_; my $file = $opt{'file'} && ref($opt{'file'}) eq 'ARRAY' ? data_file( +@{$opt{'file'}}) : $opt{'file'}; open(my $fh, '<', $file) || die "Can not open $file $!"; my @headings = $opt{'headings'} ? @{$opt{'headings'}} : ('heading'); my %hash; while (my $line = <$fh>) { chomp $line; die "This file is not for Util::Data! Stopped $!" if $line =~ /no +Util::Data/i; my @values = split(/\|/,$line); my $key = scalar @headings > 1 ? $values[0] : shift @values; my $n = 0; for my $r_heading (@headings) { if (defined($values[$n]) && length($values[$n]) > 0) { my $split = $r_heading =~ /\+$/ ? 1 : 0; (my $heading = $r_heading) =~ s/\+$//; my $value = $split == 1 ? [map { $_ =~ s/^ //; $_ } split(/;/, +$values[$n])] : $values[$n]; if (scalar @headings > 1) { $hash{$key}{$heading} = $value; } else { $hash{$key} = $value; } } $n++; } } return \%hash; } sub alpha_hash { my ($org_list, $opt) = @_; my %alpha_hash; for my $org_value (keys %{$org_list}) { my $alpha = !$opt->{article} ? first_alpha($org_value) : substr($o +rg_value, 0, 1); $alpha_hash{$alpha}{$org_value} = $org_list->{$org_value}; } return \%alpha_hash; }

The following is truncated output from alpha_hash.

'A' => [ 'Alphas', 'Arrow', 'Ash vs. Evil Dead' ], '&#65533;' => [ # in my terminal, there is just a blank spac +e between the quotes. 'Ĉon Flux (1991)' ], 'I' => [ 'I Spy (1965)', 'The Invisible Man (2000)' ],

As I said earlier, any and all help is appreciated.

No matter how hysterical I get, my problems are not time sensitive. So, relax, have a cookie and a very nice day!
Lady Aleena