Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Re^5: Using the unicode61 tokenizer in DBD::SQLite

by Anonymous Monk
on Jan 07, 2014 at 10:20 UTC ( #1069642=note: print w/ replies, xml ) Need Help??


in reply to Re^4: Using the unicode61 tokenizer in DBD::SQLite
in thread Using the unicode61 tokenizer in DBD::SQLite

I seem to have missed that bit. Are you saying that the unicode61 tokenizer works out of the box in 1.41_03?

I don't know; If the stuff I copy/pasted means unicode61 is working, then I guess it is.


Comment on Re^5: Using the unicode61 tokenizer in DBD::SQLite
Re^6: Using the unicode61 tokenizer in DBD::SQLite
by elef (Friar) on Jan 07, 2014 at 17:27 UTC
    I got 1.41_03 installed using the cpan shell with install ISHIGAKI/DBD-SQLite-1.41_03.tar.gz.
    Still no dice:
    use strict; use warnings; use DBI; my $dbh = DBI->connect( "dbi:SQLite:dbname=test.db", "", "", { # RaiseError => 1, # we'll hand +le the errors ourselves sqlite_unicode => 1, } ) or die "Can't connect to database: $DBI::err +str"; my $sqliteversion = $dbh->{sqlite_version}; print "\nSQLite version: $ +sqliteversion\n"; $dbh->do("DROP TABLE IF EXISTS data") or print "drop table failed"; $dbh->do("CREATE VIRTUAL TABLE data USING fts4 (col1 TEXT, col2 TEXT, +tokenize=unicode61)") or print "failed to create fts4 table";

    Output:
    SQLite version: 3.8.0.2 DBD::SQLite::db do failed: unknown tokenizer: unicode61 at e:\uni612.p +l line 22. failed to create fts4 table

      I got 1.41_03 installed using

      Previously I said I think recompile with perl Makefile.PL DEFINE=-DSQLITE_ENABLE_FTS4_UNICODE61 and give it a shot , so using the way you did doesn't magically give Makefile.PL thos extra arguments, so I'm not surprised nothing changed for you

        Well, if you read back your post you'll see that it wasn't clear at all, which is why I asked if it works out of the box in 1.41_03. By 'out of the box' I meant 'out of the box'. It wouldn't have been that surprising as 1.41_03 has a newer version of sqlite than 1.40 (3.8.x as opposed to 3.7.x).

      To expand a bit on the steps Anonymous Monk gave you, the steps would be:

      1. Start the CPAN shell:
        cpan
      2. Download and unpack the DBD::SQLite distribution:
        look DBD::SQLite
      3. Recompile DBD::SQLite with the options that Anonymous Monk proposed:
        perl Makefile.PL DEFINE=-DSQLITE_ENABLE_FTS4_UNICODE61
      4. Watch the output for errors.
      5. Run the tests
        make test
      6. Verify that Unicode now works in the way you expect it to.
      7. Install the module if all tests pass and things work for you
        make install
        Thanks, Corion. I saw that you are one of the maintainers. Any plans to enable the unicode61 tokenizer in future releases?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1069642]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (3)
As of 2014-11-23 03:53 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (128 votes), past polls