utf8 problems with Pod-ProjectDocs

[LanX]

Hi

I'm not sure if it's a bug or if I'm missing something...

My employer uses Pod::ProjectDocs for rendering CPAN like HTML for our modules.

But whatever I do German umlauts in UTF8 are rendered wrongly!

the HTML source shows

<p>
<p>ä</p>

<p>ö</p>

<p>ü</p>


</div>

while the POD has

=pod

=encoding utf8

ä

ö

ü

=cut

I've checked the meta-tag it shows content="text/html; charset=UTF-8" />

Adding or leaving =encoding utf8 doesn't help, the only result is a warning:

Unrecognized command 'encoding' skipped at /home/rolf/projects/cgi-bin
+/GLOBAL/DB/gedit_utf.pm line 1

Any ideas?

[remiah]

I couldn't fully trace this, nor understand all. If you are going to have some trial, 4 lines of dirty patch to ProjectDocs::Parse.pm seems to me work.

1. decode $self->{buffer} to character. Add 2 lines

use Encode qw/decode/;
$self->{buffer}=decode('UTF-8', $self->{buffer});

just after the line

  
$self->{buffer} = join "\n", qq[<div class="pod">], $self->{buffer}, "
+</div>";

,which is in the "end_pos" sub.

2. encode $self->{buffer} back. Add 2 lines

use Encode(qw/encode/);
$self->{buffer}=encode('UTF-8', $self->{buffer});

just after

$self->SUPER::parse_from_file(@_);

,which is in "parse_from_file" sub.

before SUPER::parse_from_file, it has no troble. after SUPER::parse_from_file, it is troubled. "end_pos" is callback from Pod::Parser.

[LanX]

Thanks!

I recommended your patch to the maintainer of our installation. =)

[Anonymous Monk]

I think you got it the wrong way round: encode(...) in end_pod and decode(...) in parse_from_file works for me.

[remiah]

Hello

I checked my test at that time, and I find no problems with Pod::ProjectDocs with no patch.
I wonder whether this is fix of ProjectDocs, or Pod::Parser or my dream ???
If you post your test case, I would like to check it. Mine was like this, and again, I see no problem now...

test.pl

 
#!/usr/bin/perl -w
use strict;
use warnings;
use utf8;
use lib './Pod-ProjectDocs-0.40/lib';
use Pod::ProjectDocs;
my $pd = Pod::ProjectDocs->new(
    outroot => './out',
    libroot => './lib',
    title   => 'プロジェクト',
    lan =>'UTF-8',
);
print "before gen\n";
$pd->gen();

And target script in "lib"

 
#!/usr/bin/perl


=pod

=head1 For POD TEST (ä)

ä

ö

ü

=cut

regards

[Anonymous Monk]

It is a bug, Pod::ProjectDocs doesn't account for encoding, and it doesn't binmode input files

[sundialsvc4]

Well, I admit that I might be shooting in the dark here, but ...

In perldoc perlpod I read the following caveat:

And don’t forget, when using any command, that the command lasts up until the end of its paragraph, not its line.   So in the examples below, you can see that every command needs the blank line after it, to end its paragraph.

(... which sure looks to me like one of those “doh-h-h!!” things that I am all too-familiar with.)

I also notice from the documentation for Pod-ProjectDocs that the charset can also be specified there as a parameter.

Now, my big question is (and it is one that I don’t know the answer to):   does the UTF-8 character set support the German umlaut?

Your quotation of the meta-tag ... well, I shall merely assume that there is a closing-quote after text/html.   But, assuming this to be the case, I see that it does specify UTF-8, which rather makes me suspect that this encoding does not contain the character you seek.   And if that be the case, then this is the real reason why the display is turning out wrongly.

[afoken]

Now, my big question is (and it is one that I don’t know the answer to): does the UTF-8 character set support the German umlaut?

Sure it does. Unicode contains all characters available in ISO-8859-1, and ISO-8859-1 contains umlauts. I think it is quite safe to assume that Unicode supports nearly all written languages currently in use, and some ancient ones.

Alexander