Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re: Unaccenting characters

by Laurent_R (Parson)
on Aug 28, 2013 at 22:17 UTC ( #1051324=note: print w/ replies, xml ) Need Help??


in reply to Unaccenting characters

$str =~ s/(^\x00-\x7F)/$table{'$1'} || '?'/ge;

Are you planning to write a line like that for each of your accented letters? But then, what is the point of the %table hash? You might as well hard code everything (this is not what I am recommending).

Depending on how your file is really encoded, the tr/// function might be much easier to use and probably faster. Something like this (to be completed):

$str =~ tr//aaaceeee/;

There are a number of cases where this simple tr/// function works well. If not, well, then the Unicode modules described by others.

There is a last point, though, which I can see as a problem, and which is in fact the main reason why I am posting here. "Unaccenting" letters may be less trivial than you may think. In French, all letters with an accent can be "unaccented" by just taking the same letter without the accent, this is the common way of doing things that when accents are not available. But in German, is usually rendered by ae, by oe and by ue. Similarly, I would tend to believe that the Scandinavian or other languages which have an '' probably don't translate it into an 'a'. This is just a warning that, depending on what exactly you are trying to do, your contemplated solution might be a bit simplistic.

Update: crossed out the first paragraph following choroba's comment. I had misread the regex.


Comment on Re: Unaccenting characters
Download Code
Re^2: Unaccenting characters
by choroba (Abbot) on Aug 28, 2013 at 22:33 UTC
    Are you planning to write a line like that for each of your accented letters?
    Meditate about the substitution a bit more. Remember negative character classes. There is nothing more to be written.
    لսႽ ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ

      Right, I looked at it too quickly and missed the point.

Re^2: Unaccenting characters
by mwhiting (Beadle) on Aug 29, 2013 at 16:35 UTC
         Depending on how your file is really encoded, the tr/// function might be much easier to use and probably faster. Something like this (to be completed):

         $str =~ tr//aaaceeee/;

    I originally used the tr function like you suggested. Why it didn't work so well, I'm not sure. I had:

    $_ =~ tr//aaaaaceeeeiiiinooooouuuuy/;
    but it produced a two character result, similar to the ?? in my original question above. However, the first output character was an 'a' regardless of what the result was supposed to be, and then an unprintable character placeholder which i can't duplicate here. Like the above code, it recognized which character to replace, but then didn't do it right.

    Thanks for the tip on German/Scandanavian etc languages where they don't translate 1:1, although I'm not so worried about that, at least not at the moment. :)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1051324]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (8)
As of 2014-12-28 13:49 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (181 votes), past polls