Beefy Boxes and Bandwidth Generously Provided by pair Networks DiBona
XP is just a number
 
PerlMonks  

Re: detecting the language of a word?

by BrowserUk (Pope)
on Dec 06, 2002 at 19:37 UTC ( #218133=note: print w/ replies, xml ) Need Help??


in reply to detecting the language of a word?

Depending upon the number of foriegn words your looking at, it mght be better to run through your files verifying the words against the dictionary for the predominant language, and flag any that do not show up.

You could write the name of the files to a "pages to check" file, and wrap the words in something glaringly obvious (like the hated <blink> tags :). Then you (or your native language editor person) could look at the suspect words in context and make a decision based on that. Of course that won't help you with words like your example that have meanings in several different languages.

Probably the best way to deal with that is to also flag any words that show up in more than one language dictionary.

I think that if performance is anything of an issue, then you should probably avoid storing your dictionaries in a SQL database. However, using the DBI interface to one of the flat-file databases you can achieve some pretty amazing performance as was prooved to me by grantm in this thread Fast wordlist lookup for game.


Okay you lot, get your wings on the left, halos on the right. It's one size fits all, and "No!", you can't have a different color.
Pick up your cloud down the end and "Yes" if you get allocated a grey one they are a bit damp under foot, but someone has to get them.
Get used to the wings fast cos its an 8 hour day...unless the Govenor calls for a cyclone or hurricane, in which case 16 hour shifts are mandatory.
Just be grateful that you arrived just as the tornado season finished. Them buggers are real work.


Comment on Re: detecting the language of a word?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://218133]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (9)
As of 2014-04-17 02:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (437 votes), past polls