Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re: DBD::Sqlite queries slow - and gives wrong results

by sundialsvc4 (Abbot)
on Dec 11, 2017 at 17:57 UTC ( #1205311=note: print w/replies, xml ) Need Help??


in reply to DBD::Sqlite queries slow - and gives wrong results

(Setting-aside (for the moment ...) the apparently-immediate issue of “wrong results” ...), the immediate potential-issue that occurs to me here is SQLite’s handling of transactions.

Basically, unless your request (whether read or write ...) is wrapped in a transaction, SQLite will not avail itself of any niceties such a “laziness.”   It will physically repeat every read, and(!) it will physically verify every write!

(We might reasonably assume that SQLite Studio Editor “would, of course, be fully aware of such things,” whereas (likewise, maybe, “of course”) your Perl code – just like mine, back in the day – might not yet know to do that.)

Short-Answer:   Do This Now:   “Bracket all of your SQLite calls, whether(!) read or write, in transactions.”

Even though the SQLite engine “earnestly attempts to maintain” the abstract niceties of other SQL engines, it ultimately must physically-confront the physical-reality that there is no(!) “SQL server.”   Its actual physical situation is therefore no more “evolutionarily advanced,” in any-and-all “shared-file” situation, than that of dBase, Paradox, or Clipper (say).   The implementation has obviously therefore decided to use “a transaction,” regardless of (apparent, but irrelevant) kind, as “a file-lock,” and thus the ability to use “lazy™” in-memory caching.

“Personally been there.   Personally whacked my head against that.”

HTH™ ...

  • Comment on Re: DBD::Sqlite queries slow - and gives wrong results

Replies are listed 'Best First'.
Re^2: DBD::Sqlite queries slow - and gives wrong results
by Your Mother (Bishop) on Dec 12, 2017 at 00:31 UTC

    You side-stepped the OP's question, predictably. NetWallah managed to make an interesting and helpful technical suggestion even without having an answer to the problem.

    Could you please cite the SQLite documentation which suggests the wisdom of using transactions for SELECT statements? Barring that, please provide benchmark code supporting your position that it is a best practice. COGTFO®

Re^2: DBD::Sqlite queries slow - and gives wrong results
by herveus (Parson) on Dec 12, 2017 at 13:45 UTC
    Howdy!

    First, "citation needed", because you are making wild claims with no supporting evidence.

    A select requires a SHARED lock, and that is done via a transaction. If you have multiple select statements that you want to ensure all read the same set of data, wrap them in a transaction. Also to reduce locking overhead.

    What is the actual relevance of the lack of a "SQL server"? What do you mean by that? The SQLite engine does its thing just as any other RDBMS engine does, but not as a persistent external process. That has nothing to do with all of the italicized and scare-quoted phrases your imprecation is larded with.

    And, on top of that, what relevance does your prescription have to the actual question at hand? You rail about something without explaining how it matters.

    So, your closing remark is, on its face, incorrect, as it appears that no actual help was provided. Please stop.

    yours,
    Michael
Re^2: DBD::Sqlite queries slow - and gives wrong results - HEADWHACK
by erix (Parson) on Dec 13, 2017 at 06:20 UTC

    Short-Answer: Do This Now: “Bracket all of your SQLite calls, whether(!) read or write, in transactions.”

    He's already using transactions; see the connection parameter 'AutoCommit=>0' or TIAS:

    $ perl -MDBI -e 'my$d=DBI->connect("dbi:SQLite:dbname=x.db","","",{Aut +oCommit=>0}); $d->begin_work or die "ah, of course! *HEADWHACK*\n"' DBD::SQLite::db begin_work failed: Already in a transaction at -e line + 1. ah, of course! *HEADWACK*

Re^2: DBD::Sqlite queries slow - and gives wrong results
by marto (Archbishop) on Dec 12, 2017 at 05:42 UTC

    “Personally been there. Personally whacked my head against that.”

    I wonder if this could be contributing to the nonsense continually pollute this site with?

Re^2: DBD::Sqlite queries slow - and gives wrong results
by eyepopslikeamosquito (Chancellor) on Dec 13, 2017 at 17:09 UTC

    Short-Answer:   Do This Now:  

      Pick up all “SQL manuals” on your “desk” ... whether small or big(!)
      Personally whack your head against them

    HTH™ ...

Re^2: DBD::Sqlite queries slow - and gives wrong results
by Anonymous Monk on Dec 11, 2017 at 18:17 UTC
    it will physically verify every write

    the same BS youve been repeating for years even after you were proven wrong. troll

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1205311]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (5)
As of 2019-07-18 07:16 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?