Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re^4: How to share complex data structure in threads ?

by exilepanda (Monk)
on Dec 30, 2012 at 10:16 UTC ( #1010905=note: print w/ replies, xml ) Need Help??


in reply to Re^3: How to share complex data structure in threads ?
in thread How to share complex data structure in threads ?

THANK YOU VERY VERY VERY MUCH Pope!! A very great lesson learned!

push @s, &share( { $cmd => time } }; ## The & before share() is import +ant here; and one of the few times you should ever use it. ## or push @s, &share_clone( { $cmd => time } }; ## or my %hash : shared = ( $cmd => time() ); push @s, \%hash;
my @array : shared; $array[ 0 ] = 'fred'; ## ok $array[ 1 ] = 12345; ## ok $array[ 2 ] = [ 1,2,3 ] ## NOT OK $array[ 2 ] = &share( [ 1,2,3 ] ); ## OK $array[ 3 ] = { 'a'..'z' }; ## NOT OK my %hash : shared = ( 'a' .. 'z' ); $array[ 3 ] = \%hash; ## OK
These examples explained everything that I was missing.

My real project is hard to explain, as the project scope is still defining, but from what I can understand, the structure is sound like a "online ARPG game". So, there's a World Map, and there's monsters, and players.

$World = { Players => [ "player1" => { # each key is a player id ID=>"player1", HP=>9999, MP=>2000, coX=>0, coY=>0, Equip => [ qw/xxxSword yyyShield/ ], Magic => [ qw/Earthquake/ ], Invent => { Cure => 1, Poison => 3 } }, "playerx" => { ... } ], Monsters=>{ MonsterA => [ "coX1-coY1", "coX2-coY2", ... ] MonsterB => [ "coX1-coY1", "coX2-coY2", ... ] } };
So, whenever a player is connected, a new thread created, while everyone will access the same world map. When players walk, they may able to see monster or other players. and their coords changed, which should able to reflect to other players. And by certain time, killed monsters will reborn.

This is almost the case I am likely to face. Indeed, this is more safe in the "queue" approach, but on the other side, I believe I will create more bugs while coding as the full picture is harder to retrieve.

However one situation is true here which is fuzzy condition acceptable. Glitch from time race is acceptable in a certain amount of range.

Indeed, I still can't decide which approach is better for me


Comment on Re^4: How to share complex data structure in threads ?
Select or Download Code
Re^5: How to share complex data structure in threads ?
by BrowserUk (Pope) on Dec 30, 2012 at 17:53 UTC
    So, whenever a player is connected, a new thread created, while everyone will access the same world map.

    Hm. If you have any ambition that your game will support more than a few tens of concurrent players, then I agree with anonymonk, 1 thread per player is the wrong way to go. Imagine your game became moderately popular and you have 1000 players on line at a time. 1000 threads all sitting there consuming memory doing essentially nothing waiting for their user to type something. You'd need a machine with huge memory; but the processing could be done by a 386 class cpu and it would still be mostly idle.Now think about how you would scale that to 10,000 players. You'd need 10 times as much memory but still not much more cpu.

    Equally, using an in-memory database would be very dangerous. One errant piece of code or a faulty power supply and bang! your game -- ie. your database -- is gone.

    If you are serious about developing this, you need to seriously reconsider your architecture. Your game will need to reside in the database, and the database will need to be resilient and scale. You will need (to allow for) multiple comms servers talking to your player clients; and redundant, resilient, actively distributed database server back ends (or a cluster of machines accessing a distributed filesystem like hadoop).

    Even if you want to start small on a single machine, you should separate the comms components and database components of the game and run them as separate processes on that machine and have them communicate through the database/distributed filesystem ONLY. Thus, when and if you need more capacity, you can separate the comms onto a different machine from the DB/game handling. And then if things grow larger, you can increase the number of comms servers and DB servers independently of each other to match your growing requirements.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      Ah Yes! Thank you very much again, for stopping me here!

      This reminds me some yrs ago, I used threads shared hash for a simple chat server for our 20 something ppl dept, the hash queues unread messages... and what periodically happening was a Out of memory error, and the mesgs on the server and the server self gone. I finally have to backup the hash per every minute, and start another script to keep "ping" if the server alive or restart with restore data.

      A simple hash could go in this way, I can imagine what is going to happen for such complex structure. Seems I can really ONLY rely on the db/fs now. It's not about accuracy or efficiency, but the basic stability and extendability.

        This reminds me some yrs ago, I used threads shared hash for a simple chat server for our 20 something ppl dept, the hash queues unread messages... and what periodically happening was a Out of memory error, and the mesgs on the server and the server self gone.

        Out of memory for a 20 person chat server? Then you wrote it wrong!

        I have threaded servers (some posted here) that quite happily handle 100 (or even 1000, simple persistent) concurrent connections, shipping high volumes of data bidirectionally that will run forever without leaks or the memory problems.

        What kills the idea of threading for your MMURPG is mostly down to your proposed architecture. RPG users expect to pick up where they left off the last time they logged on, which means you need to persist the state of their game at the point they left and reinstate it when they come back. Whether that's in 2 minutes or 2 years time. A shared state DB is inherently the wrong architecture for such an application.

        The second is the possible future scale of the application. A 20-man dept. is unlikely to suddenly increase by 2 or 3 orders of magnitude over night; but a role playing game, if sufficiently compelling, easily could.

        Were I writing your MMURPG, I would split the comms servers and the DB/FS servers into different processes as I outlined above; but I would still favour threads over an event driven architecture for both parts. This because the simple linear flows and local state of threaded code are easier to reason about, write, maintain and scale than the open-access, global state and having to break logically linear code into the iddy-biddy chunks that event-driven architectures require in order to ensure responsiveness.

        But I wouldn't use one thread per connection. It is simple too wasteful of resources. A single foreground thread would receive inbound action requests and immediately queue them to a pool of threads that would talk to the background DB (via sockets for inter/intra-machine flexibility), to perform those actions on the DB. Replies to clients go directly from the pool threads to the clients. I'd uses a similar thread-pool architecture for the DB processes.

        The benefit of this de-coupled, threaded architecture is that when the number of clients grow you have the choice of how to expand:

        • You can buy/lease bigger boxes with more cores; start more copies of either or both types of server as required, and know that they will fully utilise however many cores are available without the need to re-architect.

          (Something that cannot be said of event-driven architectures that will only ever use one processor at a time!).

        • Or you can throw a cluster of smaller boxes at the problem and everything still works.

          All you need is a load-distributor on the main socket and everything else scales accordingly without change.

          Again, this does not work with event-driven architectures because you have to re-tune those iddy-biddy chunks of code in order to match the change in processor and comms performance.

        One scales easily and automatically; the other requires extensive re-development and testing as the underlying hardware changes. I prefer the former.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1010905]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (8)
As of 2014-07-23 09:10 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (138 votes), past polls