Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re^4: Database Search Format Engine

by creamygoodness (Curate)
on Jun 18, 2009 at 04:21 UTC ( #772631=note: print w/replies, xml ) Need Help??


in reply to Re^3: Database Search Format Engine
in thread Database Search Format Engine

The stable branch of KinoSearch (0.165) doesn't handle UTF-8 properly. You need the dev branch for that (0.20_01 and above). For Asian languages, you absolutely need UTF-8, or support for native encodings like Shift-JIS.

Tokenizing is also quite a challenge for Asian languages, particularly Japanese, and KinoSearch doesn't have a dedicated CJK tokenizer class or anything like that. It's on the todo list, but not very high -- I'm more concerned with making sure that the framework will allow others to write high-performance KSx extensions than with writing everything myself.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://772631]
help
Chatterbox?
[LanX]: we should certainly avoid Amsterdam! ! !
[marto]: kidding. The area in question isn't great. I'd no idea these facilities existed in that part of town.
LanX vile place ...
[LanX]: well, me was kidding!
[marto]: One of the main train stations is right across the road, so handy from that perspective, and there are some nice hotels in the area, but it's not great after dark, or sometimes during the day to be honest
[LanX]: not easy to find good and affordable venues ...
[marto]: hmm, there are several universities and colleges handy that I'd have thought would offer such services, given this is out of term time, I've no experience of paying for venues :)
[LanX]: OK ...
[LanX]: I have no idea how Mark scouted, giving that he lives in London ... but I think he mentioned his wife being Scottish
[marto]: ah, so this is payback :P

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (11)
As of 2017-12-14 15:48 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    What programming language do you hate the most?




















    Results (396 votes). Check out past polls.

    Notices?