Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re^6: How to share complex data structure in threads ?

by exilepanda (Monk)
on Dec 31, 2012 at 02:26 UTC ( #1010975=note: print w/ replies, xml ) Need Help??


in reply to Re^5: How to share complex data structure in threads ?
in thread How to share complex data structure in threads ?

Ah Yes! Thank you very much again, for stopping me here!

This reminds me some yrs ago, I used threads shared hash for a simple chat server for our 20 something ppl dept, the hash queues unread messages... and what periodically happening was a Out of memory error, and the mesgs on the server and the server self gone. I finally have to backup the hash per every minute, and start another script to keep "ping" if the server alive or restart with restore data.

A simple hash could go in this way, I can imagine what is going to happen for such complex structure. Seems I can really ONLY rely on the db/fs now. It's not about accuracy or efficiency, but the basic stability and extendability.


Comment on Re^6: How to share complex data structure in threads ?
Download Code
Re^7: How to share complex data structure in threads ?
by BrowserUk (Pope) on Dec 31, 2012 at 03:35 UTC
    This reminds me some yrs ago, I used threads shared hash for a simple chat server for our 20 something ppl dept, the hash queues unread messages... and what periodically happening was a Out of memory error, and the mesgs on the server and the server self gone.

    Out of memory for a 20 person chat server? Then you wrote it wrong!

    I have threaded servers (some posted here) that quite happily handle 100 (or even 1000, simple persistent) concurrent connections, shipping high volumes of data bidirectionally that will run forever without leaks or the memory problems.

    What kills the idea of threading for your MMURPG is mostly down to your proposed architecture. RPG users expect to pick up where they left off the last time they logged on, which means you need to persist the state of their game at the point they left and reinstate it when they come back. Whether that's in 2 minutes or 2 years time. A shared state DB is inherently the wrong architecture for such an application.

    The second is the possible future scale of the application. A 20-man dept. is unlikely to suddenly increase by 2 or 3 orders of magnitude over night; but a role playing game, if sufficiently compelling, easily could.

    Were I writing your MMURPG, I would split the comms servers and the DB/FS servers into different processes as I outlined above; but I would still favour threads over an event driven architecture for both parts. This because the simple linear flows and local state of threaded code are easier to reason about, write, maintain and scale than the open-access, global state and having to break logically linear code into the iddy-biddy chunks that event-driven architectures require in order to ensure responsiveness.

    But I wouldn't use one thread per connection. It is simple too wasteful of resources. A single foreground thread would receive inbound action requests and immediately queue them to a pool of threads that would talk to the background DB (via sockets for inter/intra-machine flexibility), to perform those actions on the DB. Replies to clients go directly from the pool threads to the clients. I'd uses a similar thread-pool architecture for the DB processes.

    The benefit of this de-coupled, threaded architecture is that when the number of clients grow you have the choice of how to expand:

    • You can buy/lease bigger boxes with more cores; start more copies of either or both types of server as required, and know that they will fully utilise however many cores are available without the need to re-architect.

      (Something that cannot be said of event-driven architectures that will only ever use one processor at a time!).

    • Or you can throw a cluster of smaller boxes at the problem and everything still works.

      All you need is a load-distributor on the main socket and everything else scales accordingly without change.

      Again, this does not work with event-driven architectures because you have to re-tune those iddy-biddy chunks of code in order to match the change in processor and comms performance.

    One scales easily and automatically; the other requires extensive re-development and testing as the underlying hardware changes. I prefer the former.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1010975]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (4)
As of 2014-09-20 06:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (155 votes), past polls