Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

I would argue that this behavior is not a leak ... that it is exactly the behavior you would logically expect, and it is necessary and correct.   (But IMHO it is not the right way to do what you apparently are wanting to do.)

If you fork a process that owns a hash with two million keys in it, then copy-on-write semantics allow the forking to occur quickly.   But as each value is modified by the child process, the operating system makes a duplicate copy of the entire virtual-memory page.   In the example that you describe, this occurs for every entry because you touched it.   (Even if you assigned a variable or bucket the same value that it already had.)   16 megabytes of data are duplicated because they had to be.

And you don’t even have to touch it.   For instance:   my $foo = $shared_hash->{'bar'};   Like it or not, that's a new reference to that hash-element, and so its reference count just went up.   (And when $foo goes out of scope, it will go down.)   Since it just went up, a write and therefore a copy-on-write just occurred.   (I presume that this is intended to be a read-only data structure ...)   Even if you merely intend to reference it, you’re fairly doomed to copy-on-write it, if you try to use ordinary Perl constructs.   You need to take a fundamentally different approach, and (of course) CPAN has a number of ready-made alternatives to choose from.

I submit that, if you need to manipulate a large shared data structure, you need to resort to something like, say, IPC::Shareable to store the data in a properly-protected shared memory segment.   Or you might use interprocess communication along the same lines as the memcached daemon in Unix/Linux.   The large-data-structure should not be part of what is affected by fork.


In reply to Re: PL_strtab/SHAREKEYS and copy-on-write leak by sundialsvc4
in thread PL_strtab/SHAREKEYS and copy-on-write leak by sezal

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chanting in the Monastery: (4)
As of 2024-04-19 01:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found