Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked

Comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

erix, I can't thank you enough for do that.

  1. Ouch! I swear I had it in there at one point, but looking back to where this code originated, it was missing or in the wrong place back then.

    Position and frequency of commits is one of the many variables I've been playing with trying to get a handle on how they affect the overall performance. For example, if you uncomment the line:

    # $n % 10 or $dbh->commit

    The insertion takes 4 times as long.

  2. $N is a command line parameter.

    My typical test runs consist of the command: for /L %t in (1,1,4) do @1045292 -N=1e6 -T=%t -Q=100 which is probably self explanatory.

From around N$ = 10_000 and upwards, the threaded version *is* (much) faster.

Weird thing is, varying the number of threads (-T=n) makes no difference to the time required to process all the records:

C:\test>1049236 -N=1e6 -T=1 -Qsize=1000 Took 86.039947 seconds (for Populate DB with 1e6 records) Took 2.232394 seconds (for Create primary index) Took 16.897088 seconds (for Retrieve the whole lot) 1000000 Took 0.268509 seconds (for Read all 1e6 records using 1 threads) C:\test>1049236 -N=1e6 -T=4 -Qsize=1000 Took 85.957917 seconds (for Populate DB with 1e6 records) Took 3.164024 seconds (for Create primary index) Took 16.881054 seconds (for Retrieve the whole lot) 1000000 Took 0.251258 seconds (for Read all 1e6 records using 4 threads)

Leastwise, not when reading the records for the second time. Comment out the "Retrieve the whole lot" block and then things get muddy again:

C:\test>1049236 -N=1e6 -T=1 -Qsize=1000 Took 86.307525 seconds (for Populate DB with 1e6 records) Took 2.363253 seconds (for Create primary index) 1000000 Took 0.978504 seconds (for Read all 1e6 records using 1 threads) C:\test>1049236 -N=1e6 -T=4 -Qsize=1000 Took 86.182529 seconds (for Populate DB with 1e6 records) Took 3.017540 seconds (for Create primary index) 1000000 Took 1.925757 seconds (for Read all 1e6 records using 4 threads)

Once the single-threaded, first pass through the data is removed, it takes twice as long for 4 threads to read all the records as it does for 1 thread.

But then, why does it take the single-threaded, read-all 16 seconds to do so; when it only takes 1 second to start an entirely new thread, make a completely new connection, and then read-all the records?

Many questions and little consistency from which to derive answers at this point. If you can see any other obvious cock-ups I'd be grateful to hear of them.

(Ps. Any feel for how Pg would compare in similar a application?)

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

In reply to Re^5: Threads and DBD::SQLite? by BrowserUk
in thread Threads and DBD::SQLite? by BerntB

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    [Tux]: And I just tested, neither do other Tk apps :) (95% of my Tk Apps is either just monitoring (no input) or input of just numbers). Something to investigate then
    [choroba]: Maybe something like this?
    [choroba]: I vaguely remember we had problems with dead keys in Tk at $work - 2, but I forgot all the details
    [choroba]: but it was worse on MSWin than on Linux
    [choroba]: (and OSX didn't exist yet)
    [choroba]: Also, I'd try starting the App with the US layout and switch to the dead keys layout later

    How do I use this? | Other CB clients
    Other Users?
    Others contemplating the Monastery: (5)
    As of 2017-09-25 20:49 GMT
    Find Nodes?
      Voting Booth?
      During the recent solar eclipse, I:

      Results (290 votes). Check out past polls.