Crossed posted on StackOverflow.

"It is considered polite to inform about crossposting so people not attending both sites do not waste their time solving a problem already closed at the other end of the internet."

My apologies. I failed to see your notice at the bottom of your posting...

The OP did note that she cross-posted.

Indeed. I failed to note her note. Corrected. Thanks...

Running your code produces the following console output on my machine (Windows 7, ActivePerl):

0 τι κάνετε;
1 πάρτε το ή αφ&#942
+;στε το
2 שלום חברים
3 abc לא כןכן efg
4 מתי ולאן This is it
5 מעכשיו לעכ&#15
+13;יו 
6 Σήμερα είναι 
+Τρίτη
7 Θέλω να φάω
8 τι κάνετε;
9 שורה מס' 5
[download]

The tied-file's contents:

τι κάνετε;
πάρτε το ή αφή&
+#963;τε το
שלום חברים
abc לא כןכן efg
מתי ולאן This is it
מעכשיו לעכ&#1513
+;יו 
Σήμερα είναι &#
+932;ρίτη
Θέλω να φάω
τι κάνετε;
שורה מס' 5
[download]

I didn't get the errors you're reporting.

On a separate topic, Tie::File works well with smaller files, but performance significantly slows with files > 10M, in case you may be working with such.

Kenosis: this is very interesting.

1. Is there a quick way to convert back the PerlMonks output, so that Unicode/UTF8 editors can show it in the original encoding?

2. It seems that in your setup (apparently, the difference is ActivePerl vs. StrawberryPerl/DWIMPerl that I am using), the problem does not occur. Could the cause be the Tie::File code? StrawberryPerl uses $VERSION = "0.97_02"; Which version is ActivePerl using?

1. I don't know about these things... Yet...
2. ActivePerl uses $VERSION = "0.98" for Tie::File. Perhaps upgrading the Module is worth a try (or tie).

Update: how do I make the PerlMonks web site show the foreign fonts, instead of the HEX?

You can't, sorry

The good news is you don't really need to :) simply post proper Dumps ;)

perl -MData::Dump -MFile::Slurp -e " dd scalar read_file shift, { qw/ binmode :raw / }; " input.txt > input_txt_as_perl.pl

I don't think it is supported. The option you used is called discipline and that is perl 5.6 talk , that feature is now called PerlIO layers

Also, that feature is not documented in Tie::File ; And, in 5.6, that feature only supported :raw and :crlf, as :encoding did not exist ; You should file a bug report (did I say that?)

But you can work around this limitation by doing your own encoding/decoding. This seems to work

#!/usr/bin/perl --
use Path::Class;
use constant THISFILE => file( __FILE__ )->absolute->stringify;
use constant THISDIR  => file( THISFILE )->dir->stringify;
use strict; use warnings;
use Tie::File;
use Fcntl 'O_RDWR', 'O_CREAT';
use Encode;
    
chdir THISDIR or die Fudge( 'chdir', THISDIR );

Main( @ARGV );
exit( 0 );

sub Main {
    binmode STDOUT;
    my $FileName = "".file( THISFILE . '.tf.utf8.txt' );
    tie
        my(@Tied),
        'Tie::File',
        $FileName,
        recsep => "\x0D\x0A",
        mode => O_RDWR | O_CREAT,
    or die Fudge( tie => $FileName );
    
    if( @Tied > 2 ){
        print 'size ', int @Tied, "\n";
        print map { decode('UTF-8', $_ ); "($_)" } @Tied;
    } else {
        push @Tied, encode('UTF-8', chr $_) for 272 .. 30000;
    }
    untie @Tied;
}


sub Fudge {
    use Errno();
    join qq/\n/,
      "Error @_",
      map { "  $_" } int( $! ) . q/ / . $!,
      int( $^E ) . q/ / . $^E,
      grep( { $!{$_} } keys %! ),
      q/ /;
}

__END__
[download]

So you might subclass it like this (untested)

package Tie::File::UTF8;
use parent qw[ Tie::File ];
sub TIEARRAY {
    my $class = shift;
    $class->SUPER::new(
        @_,
        recsep => "\x0D\x0A",
        discipline => ':raw',
    );
}
sub FETCH {
    my( $self, $n ) = @_;
    my $rec = $self->SUPER::FETCH( $n );
    $rec = decode( 'UTF-8', $rec );
    $rec;
}
sub STORE {
    my( $self, $n, $rec ) = @_;
    $rec = encode( 'UTF-8', $rec );
    $self->SUPER::FETCH( $n, $rec );
}
[download]

I don't know about more exotic encodings , but it seems UTF-8 could be supported easily

Hello.

I thought getter, setter method will avoid this trouble. Below was my trial. MyTieFile.pm


Read more... (1290 Bytes)
And test script
Read more... (975 Bytes)

And I saw your post. So, overriding STORE,FETCH seems more elegant. I tried your untested code but it seems producing same warnings message for utf8 characters.

Why???

This seems to be a known problem because there is stackoverflow thread and I see the name "ikegami"

I hope "ikegami" or someone explains this problem , a little more...

And I saw your post. So, overriding STORE,FETCH seems more elegant. I tried your untested code but it seems producing same warnings message for utf8 characters.

Why???

I don't know , I didn't test it :)

remiah: I studied your module and test script: you've done a very good job - it's working. Thank you for that.

But, what this effectively does (as UNK noted in his answer here: http://stackoverflow.com/questions/13209474/ ), is re-encoding the data before inserting it into the tied array and the tied file; so the array does not contain Unicode data in internal Perl representation, but instead simply contains the imported UTF-8 strings.

Now in my project, I am doing regex comparisons and substitutions against the tied array; so if I go this route, I'll have to re-decode the array element before any processing, and re-encode it again.

What do you think?

Many thanks for your well-thought-out answer.

Helen