Re: Sharing data structures among http processes?

[tomhukins]

I've already partly answered your question in Re: (ar0n: pnotes) Re: Sharing data structures among http processes?, to clarify another response.

To directly answer your question, I should explain that you are encountering a feature of Unix called copy on write memory.

Apache 1.x on Unix uses a pre-forking model, whereby a parent httpd process forks off a number of child httpds each of which handle one request at a time. When a Unix process forks, another identical process is created. To save memory, this identical process shares all of its data with the parent process. However, as soon as either process changes a certain area of memory, separate copies of that memory are created for each process. Hence the name copy on write. This is what you are observing as data become unshared.

Aside from memory requirements, the technique you are using suffers from another problem. Imagine you have 3 httpd processes (called A, B and C) serving requests, each with a name and phone number data structure. Initially, each process holds the following information:

NAME          PHONE
----          -----
Fingermouse   528
Bergen        392
Lobster       771

If process A receives a request to change Bergen's number to 398, the data structures in processes B and C will not be affected. So, if a request to retrieve Bergen's number reaches process C, it will report that Bergen's number is still 392. Thus, it is important to share data sets between processes if the data may be written to.

[sutch]

Thanks tomhukins, this looks to be what is needed. And your explanation of copy on write memory sheds light on why I was experiencing the unsharing of memory.

I do have another related question: are there any methods for sharing a process (or a Perl object) among processes? For example, I want one shared object to update the data structure, to ensure integrity and to write the changes back to the persistent storage. Or is there a better method for handling this than one shared object attempting to service many requests?

[tilly]

There are many ways to share data between processes. You can use a local dbm file. You can use IPC::Shareable. And so on. But all of the efficient ones have the rather significant problem that all of the requests in your series have to come back to the same physical machine. This does not play well with load balancing.

However one crazy way of doing it is like this. One machine gets the request and forks off a local server. Other CGI requests are passed the necessary information to be able to access this temporary server, which is run in a persistent process, and then when this server decides the time is right, it de-instantiates itself. This would be a lot of work though.

Personally I would just see if you can keep the temporary state in the database, and just have each individual request deal with the bit of the state that they need to handle. But I cannot, of course, offer any guesses on whether this would work without knowing more details than you have given us.

[pope]

Having trying myself to implement a semaphore-based locking mechanism for IPC SysV shared memory, I'd recommend IPC::ShareLite for general purposes. It comes with a powerful locking mechanism, which is incredibly similar to flock() !
Apache::SharedMem also depends on this module.