Beefy Boxes and Bandwidth Generously Provided by pair Networks Joe
Syntactic Confectionery Delight
 
PerlMonks  

Re^7: threads::shared seems to kill performance (Workaround).

by BrowserUk (Pope)
on Jul 19, 2013 at 18:50 UTC ( #1045392=note: print w/ replies, xml ) Need Help??


in reply to Re^6: threads::shared seems to kill performance (Workaround).
in thread threads::shared seems to kill performance

Great call!

C:\test>dir file 19/07/2013 19:49 0 file C:\test>streams file Streams v1.56 - Enumerate alternate NTFS data streams Copyright (C) 1999-2007 Mark Russinovich Sysinternals - www.sysinternals.com C:\test\file: :memdb2?mode=memory&cache=shared:$DATA 347136

Doesn't explain why its not being deleted as (I think) it should.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.


Comment on Re^7: threads::shared seems to kill performance (Workaround).
Download Code
Re^8: threads::shared seems to kill performance (Workaround).
by MidLifeXis (Prior) on Jul 22, 2013 at 12:55 UTC

    I wonder (have not tested) if the ADS is being created, forcing creation of the "normal" file, and at the end, the ADS is being attempted to be deleted, but failing, leaving the data for the next run. ISTR needing to delete the main file (or perhaps some special tool besides unlink) to remove the ADS.

    --MidLifeXis

      An ADS cannot exist without a main file; and unlinking the main file will always delete the ADS; so I'm not sure what you are getting at here?

      I asked the question on the sqlite mailing list; but besides getting the obvious reply -- "if the data persists it isn't an in-memory db"; Well d'uh! -- total silence. Probably there is noone with any windows knowledge on the list.

      My suspicion is:

      • When creating an in-memory db; they use directly mapped virtual address space (memmapped files) to allow the file to grow beyond available memory and be paged on and off disk.

        To do that the need to create a file to 'back' the VAS.

      • On *nix; they probably create the backing file and then immediately unlink it (whilst still holding an open handle to it) so that it disappears when the process ends.

        This mechanism of self=managing temporary files isn't available of Windows, so they've opted to use an alternate datastream. (Perhaps as an attempt to prevent any spooled data being assessible; though that would be forlorn; more.exe can find it.)

        And they forgot to delete the file afterwards.

      Another possibility is that using threads and multiple handles to an in-memory db in conjunction with DBI's broken threading semantics is preventing the clean-up code being invoked.

      I'm not sure where else to go looking for knowledgeable sqlite+windows devs?


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

        Aaah, ok. Misunderstanding on my part. I read you to say that unless you delete the main file, the data persists.

        --MidLifeXis

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1045392]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (5)
As of 2014-04-18 01:24 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (460 votes), past polls