One reminder, artificial intelligence as whole is a failure, although I don't deny there are some limited success in very limited number of particular areas. Machine translation is one of those things we wanted to archieve with artificial intelligence, and failed. Now you are talking about something probably even bigger than machine translation.
in reply to detecting the language of a word?
This has nothing to do with Perl, no language, no tool, no project so far archieved this, and during the recent years, the fever is dying down.
There are lots of foundamental problems, just give two examples:
- When to look up the dictionaries? Say you have a stream of words, majority English with some German, you see a word, is this a German or English? What if it exists in both languages? will you then also check grammar to make a determination, see which fits better in the overall sentence? It is getting bigger, is it?
- You see a word, and suspect it is not English, will you look it up in a German dictionary, or a Spanish dictionary?