Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Re: How can I create an UTF8 encoded txt file contains strings like "aaaa"?

by graff (Chancellor)
on Sep 05, 2007 at 03:57 UTC ( #637055=note: print w/replies, xml ) Need Help??


in reply to How can I create an UTF8 encoded txt file contains strings like "aaaa"?

If "visual studio" (presumably a Micro$oft product) happens to be "formally dependent" on using a "Micro$oft-Sanctioned" notion of utf8-encoded input file format, then you may need to ensure that the file begins with a "byte-order-mark" (BOM) character (U+FEFF) -- see whether this helps:
open( OUT, ">:utf8", "a.txt" ) or die "a.out: $!"; print OUT "\x{feff}aaaa\n"; close OUT;
For some reason, M$ apps seem to have adopted the use of a file-initial BOM character to signal that a "plain-text" data file contains utf8-encoded unicode characters. If a file contains utf8 wide characters without the initial BOM, apps like wordpad, etc, will misinterpret the wide characters as something else. And maybe "visual studio" is insisting that a file be "marked as containing utf8" even when it doen't need to include wide characters...

(Of course, the BOM was originally intended to be of use only in UTF16-encoded unicode data files, to indicate the "endian-ness" (byte-order) of the 16-bit data, and it shouldn't really be needed at all in a utf8-encoded file, because utf8 is not affected by big-endian vs. little-endian byte-order. But a number of applications -- particularly M$ apps that are able to handle plain-text files along with their rogues-gallery of "application-specific file formats" -- have inexplicably come to depend on a utf8-encoded BOM at the start of the file, acting like a sort of "magic number" to let them know that they are looking at a utf8-encoded file.)

  • Comment on Re: How can I create an UTF8 encoded txt file contains strings like "aaaa"?
  • Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://637055]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (7)
As of 2016-10-01 21:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    How many different varieties (color, size, etc) of socks do you have in your sock drawer?






    Results (8 votes). Check out past polls.