Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister

Re^5: Using the unicode61 tokenizer in DBD::SQLite

by Anonymous Monk
on Jan 07, 2014 at 10:20 UTC ( #1069642=note: print w/replies, xml ) Need Help??

in reply to Re^4: Using the unicode61 tokenizer in DBD::SQLite
in thread Using the unicode61 tokenizer in DBD::SQLite

I seem to have missed that bit. Are you saying that the unicode61 tokenizer works out of the box in 1.41_03?

I don't know; If the stuff I copy/pasted means unicode61 is working, then I guess it is.

  • Comment on Re^5: Using the unicode61 tokenizer in DBD::SQLite

Replies are listed 'Best First'.
Re^6: Using the unicode61 tokenizer in DBD::SQLite
by elef (Friar) on Jan 07, 2014 at 17:27 UTC
    I got 1.41_03 installed using the cpan shell with install ISHIGAKI/DBD-SQLite-1.41_03.tar.gz.
    Still no dice:
    use strict; use warnings; use DBI; my $dbh = DBI->connect( "dbi:SQLite:dbname=test.db", "", "", { # RaiseError => 1, # we'll hand +le the errors ourselves sqlite_unicode => 1, } ) or die "Can't connect to database: $DBI::err +str"; my $sqliteversion = $dbh->{sqlite_version}; print "\nSQLite version: $ +sqliteversion\n"; $dbh->do("DROP TABLE IF EXISTS data") or print "drop table failed"; $dbh->do("CREATE VIRTUAL TABLE data USING fts4 (col1 TEXT, col2 TEXT, +tokenize=unicode61)") or print "failed to create fts4 table";

    SQLite version: DBD::SQLite::db do failed: unknown tokenizer: unicode61 at e:\uni612.p +l line 22. failed to create fts4 table

      To expand a bit on the steps Anonymous Monk gave you, the steps would be:

      1. Start the CPAN shell:
      2. Download and unpack the DBD::SQLite distribution:
        look DBD::SQLite
      3. Recompile DBD::SQLite with the options that Anonymous Monk proposed:
      4. Watch the output for errors.
      5. Run the tests
        make test
      6. Verify that Unicode now works in the way you expect it to.
      7. Install the module if all tests pass and things work for you
        make install
        Thanks, Corion. I saw that you are one of the maintainers. Any plans to enable the unicode61 tokenizer in future releases?

      I got 1.41_03 installed using

      Previously I said I think recompile with perl Makefile.PL DEFINE=-DSQLITE_ENABLE_FTS4_UNICODE61 and give it a shot , so using the way you did doesn't magically give Makefile.PL thos extra arguments, so I'm not surprised nothing changed for you

        Well, if you read back your post you'll see that it wasn't clear at all, which is why I asked if it works out of the box in 1.41_03. By 'out of the box' I meant 'out of the box'. It wouldn't have been that surprising as 1.41_03 has a newer version of sqlite than 1.40 (3.8.x as opposed to 3.7.x).

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1069642]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (2)
As of 2017-07-22 20:24 GMT
Find Nodes?
    Voting Booth?
    I came, I saw, I ...

    Results (341 votes). Check out past polls.