http://www.perlmonks.org?node_id=1021935


in reply to How to remove wide chars (ex: 年 or &#x6216) from a text file?

I agree with davido - your question is too vague. What I see in the OP are hexadecimal entity references for Chinese characters. Maybe you have entity references to other non-ASCII characters as well? If you just want to delete all such entity references:
s/\&#x\w+;//g;
(Note that the number of hex digits per character may vary.) In the case of the OP data sample, that regex leaves 7 lines that are either empty or contain just hyphens and/or spaces.
  • Comment on Re: How to remove wide chars (ex: 年 or &#x6216) from a text file?
  • Download Code