Jump to content

Multimangos


Recommended Posts

Okay, lots of people seem to be misunderstanding this stuff.

Whether you use threads or processes doesn't matter in terms of multi-core CPUs. A process is just a 'container' that holds one ore more 'threads'. A CPU does NOT execute processes, it executes the code in the THREADS. There is no work required in either case to "make one thread run on each CPU", the thread scheduler performs all this automatically and is generally much better at setting processor affinities than humans are, nevertheless if you really wanted to you COULD override it.

Multi-threading will give a performance increase on multi-core systems, but introduces a LOT of problems if not implemented correctly. These will include race conditions, bottlenecks, etc. Also, one thread crashing brings down the entire process. The advantage is that multi-threaded code is a lot easier to write than multi-process.

Multi-processes have the advantage of being "distinct" from one another. Usually you would have one "parent" process and a bunch of "children", all in the same Job group (or whatever your OS's equivalent is). One on side you have the advantages of increased stability in the form of one process not bringing down the bunch, on the other side the difference is possibly negligible in the case of software like this unless it is heavily rewritten. (ie if a core component dies your entire job is fucked anyway). Another disadvantage is the big difference between IPC over different operating systems, I personally use Boost for all my multi-threading needs in C++, I'm not sure if there is anything for multi-process that is as easy to use.

People are talking about "clustering", I dont THINK that was the intention of the developers in terms of multiple-processors because that would requrie a lot more work and would require network communications and syncronizations along with the base IPC.

There is no real difference in terms of multi-processes and multi-threading in terms of how many clients a single computer could handle. The differences on a single server would be stability (assuming its implemented very well) and ease of use.

On multiple servers, if "clustering" were introduced, that would requrie multi-processes although that would be a very large scale change and would require massive amounts of work to be able to be used stablely and not just in a perpetual state of "unstable/beta".

Anyway, my vote personally is for multi-processing because I would like to see better distinction between core components and if it was implemented correctly (thats the huge key) then it would be a lot more stable and fault tollerant.

Link to comment
Share on other sites

  • Replies 159
  • Created
  • Last Reply

Top Posters In This Topic

ACE has inplementation of CORBA thats called TAO, which runs on all platforms on which ACE runs.

http://www.dre.vanderbilt.edu/~schmidt/DOC_ROOT/TAO/TAO-INSTALL.html

Ah nice.

At any rate though. Using COBRA or not, IPC is more work to implement in terms of architecture. It has to be structured right otherwise its pointless to do. ie If one child dying kills the entire job then whats the advantage over plain multithreading? I wasn't saying that IPC per-se is hard to implement compared to multi-threading, but rather its harder to design correctly (imo).

Thanks for the link btw. ;)

Link to comment
Share on other sites

hmm multithreading sounds more like for the average users use, not for the learning process mangos tries to achieve. thus id vote for multiprocessing, since that will hold much more stability and is actually what a server with a huge ammount of clients needs.

Link to comment
Share on other sites

Well at least the 32bit users will be happy with Multi-Processing. Cause the current problem is that even if you have a System which support more then 4GB as 32bit System then a Application cant use more then 3GB RAM.

But if you have for example 2 processes then you can use 6GB totally, and so on...

Link to comment
Share on other sites

There is no work required in either case to "make one thread run on each CPU",

While this is true, you do not get as good of a performance improvement out of it as you would if the devs put work into optimizing the threading. And, of course, various components (e.g. maps) run on a single thread that could be split out into multiple threads.

Cause the current problem is that even if you have a System which support more then 4GB as 32bit System then a Application cant use more then 3GB RAM.

That is what AWE is for (and applications don't address RAM, they address virtual address space), and on some platforms they can use all 4GB VAS.

Link to comment
Share on other sites

While this is true, you do not get as good of a performance improvement out of it as you would if the devs put work into optimizing the threading. And, of course, various components (e.g. maps) run on a single thread that could be split out into multiple threads.

Errr. Either you don't understand how threads work or you don't understand what I was trying to say. Either way I will address your concern.

My point was not to say that multi-threading is not needed, on the contrary in fact, my point was that people are saying "if you have more than one thread you can tell x thread to run on CPU core y" or w/e. The point I was trying to make is the thread scheduler already does this and handles context switches and affinity masks a lot better than a human would so its best left alone. ie I'm saying that yes threads are good, but you shouldn't fiddle with thread scheduling settings because you'll most likely decrease performance (compared to a non-modified multi-threaded setup). Naturally you can set thread priorities but most of the other stuff should be left alone, that was the point I was trying to make.

That is what AWE is for (and applications don't address RAM, they address virtual address space), and on some platforms they can use all 4GB VAS.

Don't you mean PAE? (For the 4GB on x32) This introduces problems of its own though and is not really an elegant solution. The only true solution if you want more than 3GB of addressable RAM in a reliable fashion on Windows is x64. The kernel has to go somewhere, all that PAE does is allow an individual process to address more RAM, it doesn't actually increase the amount available to the user-space as a whole. There is a setting to enable LARGEADDRESSSSUPPORT or w/e, but the kernel is generally given 2GB for a good reason so unless you're running a very heavy SQL server or something that needs access to GBs of data its probably best left alone.

Link to comment
Share on other sites

My point was not to say that multi-threading is not needed, on the contrary in fact, my point was that people are saying "if you have more than one thread you can tell x thread to run on CPU core y" or w/e. The point I was trying to make is the thread scheduler already does this and handles context switches and affinity masks a lot better than a human would so its best left alone. ie I'm saying that yes threads are good, but you shouldn't fiddle with thread scheduling settings because you'll most likely decrease performance (compared to a non-modified multi-threaded setup). Naturally you can set thread priorities but most of the other stuff should be left alone, that was the point I was trying to make.

No one here suggested that you tell the system which thread to run on which core/CPU. My point is that multithreading things like maps is FAR more valuable than providing cluster support. While I do have a ton of DL360 G2s available, I don't /really/ need a cluster. I'd prefer to have better multithreading support.

Don't you mean PAE? (For the 4GB on x32)

No, AWE. AWE allows PPros or higher to address up to 64GB PAS when using the NT kernel.

This introduces problems of its own though and is not really an elegant solution.

Not for Mangos, which currently has the largeaddressaware flag enabled. You have a point for Term Servs, but again, not for Mangos which is what this thread is about.

Also, remember that OSes, like OS X/Darwin, provide a 4/4 split. It causes TLB trashing more-so than a system with a 2/2 or 3/2 split, but it does exist.

Link to comment
Share on other sites

No one here suggested that you tell the system which thread to run on which core/CPU. My point is that multithreading things like maps is FAR more valuable than providing cluster support. While I do have a ton of DL360 G2s available, I don't /really/ need a cluster. I'd prefer to have better multithreading support.

Just because you gain clustering doesn't mean you loose concurrency.. What do you see as the advantage of multithreading over multiple-processes? No-one is forcing you to use multiple servers, my understanding is that using a single server will be fine if you choose to.

No, AWE. AWE allows PPros or higher to address up to 64GB PAS when using the NT kernel.

My mistake.

Not for Mangos, which currently has the largeaddressaware flag enabled. You have a point for Term Servs, but again, not for Mangos which is what this thread is about.

Also, remember that OSes, like OS X/Darwin, provide a 4/4 split. It causes TLB trashing more-so than a system with a 2/2 or 3/2 split, but it does exist.

You also have to keep in mind, not everyone is running a dedicated server just for mangos. Mangos (afaik) is designed to be an 'educational' project, so you need to think about to rest of your userbase. Some people (such as myself) are just enthusiasts who use mangos to "sandbox" WoW and test things or aid in reverse engineering.

EDIT:

And just to point out, even though Mangos has the flag enabled the flag still has to be enabled in the bootloader or it will do nothing.

Link to comment
Share on other sites

You also have to keep in mind, not everyone is running a dedicated server just for mangos.

Which is why focusing on multithreading would be more advantegous to the user base than focusing on clustering support. Focus on the many. I think Derex pointed out the disadvantages fairly well, although I still don't understand why a multithreaded + multiprocess application could not be created, and why it requires multiple processes to make it so (leverage existing OS clustering support, perhaps?).

And just to point out, even though Mangos has the flag enabled the flag still has to be enabled in the bootloader or it will do nothing.

Yes, you do have to enable /3GB, but I've been running it that way for some time now (I submitted the original LAA flag patch).

Link to comment
Share on other sites

Well, Multiproc is the way i would go, Well it would be the next real step for a MMORPG server framework, Cross-realm bgs and Chat, Holding Great amount of clients ... SEXY!

But i dont understand quite well, if we did managed to have Multiproc and Clustering support, why still we cant get the full advantage of the Machine, Means use the 2/4/6 Cores avaliable?

Link to comment
Share on other sites

Which is why focusing on multithreading would be more advantegous to the user base than focusing on clustering support. Focus on the many. I think Derex pointed out the disadvantages fairly well, although I still don't understand why a multithreaded + multiprocess application could not be created, and why it requires multiple processes to make it so (leverage existing OS clustering support, perhaps?).

Yes, you do have to enable /3GB, but I've been running it that way for some time now (I submitted the original LAA flag patch).

Yep, enabling /3GB /PAE helped me in the past Mangos can use up to 3GB RAM then it crash cause it cant use more. 32bit Support 8GB for example fine, if you start 2x Mangos it use up to 6GB RAM Physical on Win2k AS.

Anyway, I probably switch to Linux 64bit to use the whole 8GB for one application cause currently it crash very often cause of a RAM overload :/

Link to comment
Share on other sites

There is no way an app to address more than 4GB on 32bit platform.

A pointer is 32 bits, 2^32 = 4GB

With PAE 36bit (64GB max), it work fine. When you have 2 processes then you could use 6GB / 8GB totally. With the /3GB switch 3 GB / process are maximum possible.

I can use the whole 8GB of my phsyical memory with my 32bit System fine. So Multi-Processing would make 32bit users very happy, cause currently Mangos is not really useable on 32bit Systems with more then 200~ players, it simply crash then cause the RAM need more then 3GB when a lot of instances and other stuff is running. (If you have GridUnload DISABLED -> You get a lag spree when you enable it with 200+)

You see that clearly in the CrashLogs...

Number Of Processors: 4

Physical Memory: 4194303 KB (Available: 3562520 KB)

Commit Charge Limit: 4194303 KB

CRITICAL ERROR.

Couldn't initialize the symbol handler for process. - Error [Für diesen Befehl ist nicht genügend Speicher verfügbar.].

As you see it show the other part of the 4GB and says it crash cause of a full memory (But it are still 3,5GB free but Mangos cant access this memory). Using 2+ Processes would fix it 100% ;-)

(5GB~ used on a 32bit System, totally 8GB available with PAE)

64516433ys8.jpg

Mangos crash when it reach 3.1GB~ :/ (It cant use more as single process)

Link to comment
Share on other sites

Yes, so need several porcesses in order to alow 32bit OS to handle more than 200 online. And I realy dont want to think how much new memory will be needed when mmaps are developed ...

I have seen a lot of crashes of by trying to address more than 3GB.

Most linuxes are x64 now, so its not that big problem on the linux front.

Link to comment
Share on other sites

That would be at least for the RAM related issues a good solution. With GridUnload it need about 30-40% less RAM but it has about the same up time cause it crash of the GridUnload issues and it lag a lot more.
Yes this is why I'm currently opting for multithreading actually. There is no way in hell the memory usage should be this high even with hundreds of players online if you have gridunloading on. There must be some memory leaks somewhere or I don't know. The fact that gridunload makes crashes and lags just means the core is not well optimized. If we just throw more hardware at them then these problems will be neglected and we will go more and more towards and end where the core is a horribly slow / crashy memory hog that you can only cope with by having a cluster of 10 computers working at it. So don't think that clustering is the only solution to the problem .. in fact it's one of the worst solutions .. the lazy man's solution. We should be a learning project that delivers a quality product instead of a corporate hog where you don't care if you write horrible code as long as you write it quickly and you just have the hardware department give you enough muscle to chug that pig along.
Link to comment
Share on other sites

Multiprocessing, because on a larger scale one machine just won't be able to cope with the load.

It'd be very interesting how much overhead there is when you run all seperated processes communicating via CORBA on one machine compared to doing it multithreaded. Splitted processes will naturally lead to good design, whereas multithreading is imo more likely to be implemented in a sloppy way that will lead to unsolvable race conditions.

But let's put things clear: multithreading has limits, multiprocessing does not. The question is where those limits are. How many clients are atm supported by mangos? How many will it be with line of sight checking?

I just don't know whether one machine is enough. I don't see it like wyk3d. I think making mangos multithreaded is the pragmatic "we want some performance in some way"-way wheras multiprocessing is the theoreticaly perfect and scalable solution if done right and IPC doesn't generate an overhead as high as the power improvements of (affordable) clustering gives us. This is of course mostly dependent on the speed and mem usage CORBA has when the processes run on a single machine.

Link to comment
Share on other sites

But let's put things clear: multithreading has limits, multiprocessing does not. The question is where those limits are. How many clients are atm supported by mangos?
I don't know where the limits are but I do know that we are far from reaching it since I can list a lot of things that can be done better. So for me the question is not IF we need multiprocessing but WHEN. The answer I can give at this moment is .. NOT YET.
Link to comment
Share on other sites

but if multithreading gets implemented, please look out, that the battleground-system can get its own process (cause somewhere derex written that it won't be possible to add multiprocessing to a multithreaded-code) - only if this will be possible

i voted btw for multithreading, cause i don't have any clue what will be better and want that this poll has a draw at the end, so that people with knowledge about this have to decide it by thereself :)

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. Privacy Policy Terms of Use