Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Shared array memory leak in Perl5.8

by sureshr (Beadle)
on May 20, 2003 at 09:24 UTC ( [id://259411]=perlquestion: print w/replies, xml ) Need Help??

sureshr has asked for the wisdom of the Perl Monks concerning the following question:

I am using Perl5.8 with threading. I am seeing that use of shared array (or hashes) or queues, results in memory leak. A google newsgroup search refers this to be known problem. Is there some fix or patch for this or what is the workaround used?
As a workaround, I am thinking of using limited size memory pools & use the local arrays fill-&-flush technique to share the data between the threads. But, I am not happy about doing this, as this would also require a constant size memory always in use by my process :( Any other ideas or suggestions, welcome...
Thank you,
-sureshr

Replies are listed 'Best First'.
Re: Shared array memory leak in Perl5.8
by Joost (Canon) on May 20, 2003 at 14:00 UTC
Re: Shared array memory leak in Perl5.8
by fokat (Deacon) on May 20, 2003 at 13:06 UTC

    I'm not sure I'm following your question correctly. Your process' virtual memory allocation won't ever reduce. You might find the explanation at this node useful, if my interpretation is correct.

    Best regards

    -lem, but some call me fokat

Re: Shared array memory leak in Perl5.8
by petesmiley (Friar) on May 20, 2003 at 15:50 UTC
    I haven't checked the links provided here yet for lack of time. I have run into this in the past. The error I was having was with pop and shift not releasing the memory on shared arrays.

    For a quick fix I changed the dequeue method in Thread::Queue to do a: delete $q->[-1] (which returns the value deleted) instead of a shift. I also changed the enqueue to unshift instead of push to return it the correct behavior of a queue. This seems to work.

    Not sure if this will help, or what the internal differences between a delete and shift would be to a shared array. Good Luck.

    smiles

    Edited: Ah there certainly is a patch, I could of used that few months ago. :(

      Pete & Joost,

      The patch did'nt really help. I have updated the bug in the bug link http://bugs6.perl.org/rt2/Ticket/Display.html>, bug id 15893, with a sample code that would reproduce the problem. I tried even using delete/unshift technique, which did'nt help either :(
      Currently, I am using a fixed size circular-Q, which would recycle a __MAX__ size array by overwriting the data that is read. Sometimes we would loose old-data due to overwriting, in which case I should choose a better Q size. Sounds not so good, but better than a memory leak :). I shall post more details if I find any update on this.

      Thank you,
      Suresh R

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://259411]
Approved by Tanalis
Front-paged by Tanalis
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (4)
As of 2024-04-22 18:59 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found