http://www.perlmonks.org?node_id=919137


in reply to difficulty reading csv file

It would be helpful to your readers to know what your original CSV looks like it before you process it, and what "looks fine" really means, however I can say that it sure appears to be HTML entity codes for CJK unified ideographs. From this I presume "looks fine" probably means "is made up of mixed English and Chinese text". Whatever you use to display the contents of this file must be able to decode this HTML, so many basic text editors will probably get it wrong. So the browser you're using probably looks fine when I copy a chunk from your example without code tags:

礀漀最愀 洀愀琀 crescent moon yoga mat

At this point I don't know where you want to go with this. If you only want the English, it is relatively simple to strip out the HTML entity codes, but if you still want it to be bilingual you will need to keep them and display only through a device that understands them. Fortunately for me I'm blissfully unaware of the wonders of Unicode, but one place to start may be HTML::Entities. I can't offer more than that, but I'm sure many others here can if you can be more specific about your needs and intentions.

Hope this helps!

--
I'd like to be able to assign to an luser