Multimangos

[Auntie Mangos]

Cypherjb yea seems you are smart, but cluster comes transparently with multiprocess version simply because we can use CORBA for IPC. CORBA is not the thing it was some years before, current CORBA implementations got realy advanced.

[Auntie Mangos]

Cypherjb yea seems you are smart, but cluster comes transparently with multiprocess version simply because we can use CORBA for IPC. CORBA is not the thing it was some years before, current CORBA implementations got realy advanced.

Oh really? Havn't used COBRA in a looong time.

Whats its cross-platform support like?

[Auntie Mangos]

Yes, so need several porcesses in order to alow 32bit OS to handle more than 200 online. And I realy dont want to think how much new memory will be needed when mmaps are developed ...

Exactly, and I speak here of a normal Server without VMaps or anything else. It hold only about 30-90 minutes till it reach 3GB RAM which force you to make a 2nd Realm for now.

I have seen a lot of crashes of by trying to address more than 3GB.

It simply exit itself cause it cant use any RAM, a lot of developers tried to use at least 4GB on 32bit System with one process but I cant remember much programs who had success except Windows itself.

Most linuxes are x64 now, so its not that big problem on the linux front.

Thats the reason why I want to switch to Linux, I got it up and running with the SAME core which I use for Windows but its compared to Windows very unstable cause of the "assert" stuff. Since there exist no GDB tutorial anymore I cant check the exact reason ( http://getmangos.eu/community/viewtopic.php?id=4287 ). I hope that the Linux related tutorials get updated soon so i can switch to Linux so fast as possible

Anyway, since Multi-Processing affect all Mangos users its probably the best way for the future of the project and defiantly better then it is currently (No real Multithreading / Multiprocessing).

No, you need to fix the problems with grid unloading and then you won't need that much memory.

That would be at least for the RAM related issues a good solution. With GridUnload it need about 30-40% less RAM but it has about the same up time cause it crash of the GridUnload issues and it lag a lot more.

[Auntie Mangos]

As you know, I tried to support mtmaps part of Derex's ACE patch and made a fork of mangos located here

http://github.com/scamp/mangos, mtmaps branch.

Some period of time it worked fine, both on linux and win32, see http://getmangos.eu/community/viewtopic.php?id=4461.

But now the old mtmaps code does not work since revision 6824 of mangos.

The changes done in the way the maps work have broken multithreading.

I am not so skillful coder and I do not understand the mangos code structure enough to rewrite the patch.

So I' lloking for anybody who can help me with this.

I think that a server application must be multithreaded for adequate use if all CPU's present, and to prevent blocking of all process by one part of it, which gives a bit of delay. Apache, MySQL, etc do work in this way.

Without this functionality mangos cannot be used in SMP environments and high loads.

When main thread is consuming ~99% of single logical CPU, other CPU's are not used, so there is a limit of performance for mangos. For example, a system with Q6600 quad-core Intel CPU can handle about 1000 of users with multithreading, and not more than 250 without multi, and it lags like hell.

I was very unhappy when the ACE netcode patch was merged to upstream, but only the network sockets' part.

Maybe you can change your decision and will support this part of patch until the new multiprocessing model will be ready?

[Auntie Mangos]

As you know, I tried to support mtmaps part of Derex's ACE patch and made a fork of mangos located here

http://github.com/scamp/mangos, mtmaps branch.

Some period of time it worked fine, both on linux and win32, see http://getmangos.eu/community/viewtopic.php?id=4461.

But now the old mtmaps code does not work since revision 6824 of mangos.

The changes done in the way the maps work have broken multithreading.

I am not so skillful coder and I do not understand the mangos code structure enough to rewrite the patch.

So I' lloking for anybody who can help me with this.

I think that a server application must be multithreaded for adequate use if all CPU's present, and to prevent blocking of all process by one part of it, which gives a bit of delay. Apache, MySQL, etc do work in this way.

Without this functionality mangos cannot be used in SMP environments and high loads.

When main thread is consuming ~99% of single logical CPU, other CPU's are not used, so there is a limit of performance for mangos. For example, a system with Q6600 quad-core Intel CPU can handle about 1000 of users with multithreading, and not more than 250 without multi, and it lags like hell.

I was very unhappy when the ACE netcode patch was merged to upstream, but only the network sockets' part.

Maybe you can change your decision and will support this part of patch until the new multiprocessing model will be ready?

i would be happy for that too

[Auntie Mangos]

As you know, I tried to support mtmaps part of Derex's ACE patch and made a fork of mangos located here

http://github.com/scamp/mangos, mtmaps branch.

Some period of time it worked fine, both on linux and win32, see http://getmangos.eu/community/viewtopic.php?id=4461.

But now the old mtmaps code does not work since revision 6824 of mangos.

The changes done in the way the maps work have broken multithreading.

I am not so skillful coder and I do not understand the mangos code structure enough to rewrite the patch.

So I' lloking for anybody who can help me with this.

I think that a server application must be multithreaded for adequate use if all CPU's present, and to prevent blocking of all process by one part of it, which gives a bit of delay. Apache, MySQL, etc do work in this way.

Without this functionality mangos cannot be used in SMP environments and high loads.

When main thread is consuming ~99% of single logical CPU, other CPU's are not used, so there is a limit of performance for mangos. For example, a system with Q6600 quad-core Intel CPU can handle about 1000 of users with multithreading, and not more than 250 without multi, and it lags like hell.

I was very unhappy when the ACE netcode patch was merged to upstream, but only the network sockets' part.

Maybe you can change your decision and will support this part of patch until the new multiprocessing model will be ready?

I agree here Multithreading worked perfectly on a Quadcore with this branch and the current branches are not usable on MultiCPU Systems or simply useless.

[Auntie Mangos]

The problem is that atm I can code only in the weekends. If I have free time this weekend I may try to port it ... but I deffinitely won't have time to stress-test it.

As you know, I tried to support mtmaps part of Derex's ACE patch and made a fork of mangos located here

http://github.com/scamp/mangos, mtmaps branch.

Some period of time it worked fine, both on linux and win32, see http://getmangos.eu/community/viewtopic.php?id=4461.

But now the old mtmaps code does not work since revision 6824 of mangos.

The changes done in the way the maps work have broken multithreading.

I am not so skillful coder and I do not understand the mangos code structure enough to rewrite the patch.

So I' lloking for anybody who can help me with this.

I think that a server application must be multithreaded for adequate use if all CPU's present, and to prevent blocking of all process by one part of it, which gives a bit of delay. Apache, MySQL, etc do work in this way.

Without this functionality mangos cannot be used in SMP environments and high loads.

When main thread is consuming ~99% of single logical CPU, other CPU's are not used, so there is a limit of performance for mangos. For example, a system with Q6600 quad-core Intel CPU can handle about 1000 of users with multithreading, and not more than 250 without multi, and it lags like hell.

I was very unhappy when the ACE netcode patch was merged to upstream, but only the network sockets' part.

Maybe you can change your decision and will support this part of patch until the new multiprocessing model will be ready?

[Auntie Mangos]

Cypherjb yea seems you are smart, but cluster comes transparently with multiprocess version simply because we can use CORBA for IPC. CORBA is not the thing it was 2-3 years before, current CORBA implementations got realy advanced.

damned, used corba about 2 or 3 years ago.

It was a bit of a pain in the as, but when used correctly it gave nice results.

I do not really see your point why multiprocessing excludes multithreading, except nobody would like to implement both at the same time into a project like mangos.

thinking about multiple process running maps on each node, and each map on each node could be multithreaded too... but this is not point of this discussion.

(of cause do clustering on map base ist creepy and imo not how a cluster should work like, but well....)

about the switch from multiprocess to multithread, as you said.. it's possible in theory, but no one would like to do it.

If you want to read some things about frameworks that could make it possible (if everything in code respekts that whish)

I can suggest the book (german) "automotive embedded systems"

I personally love clusters, so I would perfer this solution first.

( looking forward for a character database merge tool to combine testing communities )

I only want say: make pull for selection development way is nonsense

this is correct, so I won't vote, it's up to the devs to decide such things,

but really nice to see a discussion about this!

[Derex]

Hi,

This thread is the beginning/planning of project called multimangos. The goal of this project will be adding multithreading/multiprocessing in mangos, having much more clients connected to the server with less latency.

First thing that needs to be decided is wheater to go for multithreading or multiprocessing.

Pros/Cons with simple words:

Multithreading ( this means mtmaps for Players/WorldSessions also )

Advantages:

* Easy to use for noob admins ( I think if admin is noob better not make the server at all ).

* Won't require adding aditional IPC libraries/frameworks to mangos source.

Disadvantages:

* It wont be possible to make multiprocessing version out from the multithreaded version. ( who thinks its possible - go to hell ).

* Will require hudge rewrite of mangos, starting from vmaps, database support, half of the files in src/game directory and much more. A lot of bottlenecks/overhead may appear from the mutlithreading.

* It will be extremely easy for developer who dont know how to do multithreading to add threading-bugs to the core.

Multiprocessing ( this means develop cluster )

Advantages:

* If its ready it will be much more stable ( less crashes ). It will include fault tolerance and fault detection services, this will alow when one node crashes to not shut down the whole server.

* Can be used like the multithreaded version with the same performance ( maybe a bit bigger memory usage ).

* Implementing things like cross-realm battlegrounds and voicechat will come somehow nautral and easy.

Disadvantages:

* A bit harder to use for admins, will require starting 2-3 aditional services in the simplest case.

* Requires some knowlege for IPC technologies for implementation. Currently the only technology I see for C++ is CORBA, and CORBA is not that simple.

Both will require the same amount of time to develop. I think the second will be easyer to debug and test.

[Rhyll]

Didn't you do some clustering work already for mangos long time ago?

[Derex]

Didn't you do some clustering work already for mangos long time ago?

Yea, it was my personal research + Sinnerg

[Wyk3d]

Bottom line: if you don't plan on getting more machines .. clustering = BAD. With clustering on the same hardware you will be able to host much less clients and if you had lag until now will have even more lag with clustering unless you get additional machines.

We are in the age of the multithreading .. cpus with more and more cores are coming out and software tools are making advances in supporting them .. A lot of the core needs to be cleaned up anyway, so having to rewrite stuff is actually a good thing. With proper multithreading on new hardware noone would be complaining about lags.

[Wyk3d]

Also note that in general scalability is not our biggest problem currently. There are a lot of things that if left uncorrected will bog down clustered servers too. So I think the time has not come yet to choose how we want to scale .. the time has come to start fixing those fundamental problems, and after the core is so fast that nobody will even need scaling .. then think about scaling .. oh wait.

[Derex]

As far as I remember you were the one saying there is no need to rewrite the old code, and that it was running fine and so on.

About performance of cluster when running on single machine: It will be the same when using multiprocessing and when using multithreading. Even the multithreading can be much slower if its not syncronized correctly.

[imblizzspy]

I'd go with multithreading , cpus these days are very powerful, and we can always host sql, realmd and mangos on different machines.

[Wyk3d]

As far as I remember you were the one saying there is no need to rewrite the old code, and that it was running fine and so on.

Whenever I say that I don't want something to be rewritten I mean that yes it could use a lot of refactoring, but it does not need to be rewritten from scratch. If you don't change what the code does, only how it does that, you shouldn't rewrite but refactor. I did say that some things are doing fine but also that many other things aren't .. so you shouldn't generalize.

About performance of cluster when running on single machine: It will be the same when using multiprocessing and when using multithreading. Even the multithreading can be much slower if its not syncronized correctly.

The point is that with clustering you need a lot of message passing in places that can be much better optimized with multithreading.

[Derex]

About why clustering will be the same when running on single machine:

CORBA SHMIOP alows using shared memory for IPC for processes on the same machine.

CORBA Colocation alows calling servants directly, if they are in the same process.

I dont go for multiprocessing, but I put it as strong alternative.

Implementing full multithreading will result in the same server layout as with implementing multiprocessing:

* Some threads/processes that handle player connections.

* Some threads/processes that handle each map ( Its possible one thread to handle several maps ... this is not inportant for now ).

* ONE thread/process that handles inter-map things ... chat, guilds, friends etc.

[Nitrogen]

I would go for Multiprocessing .

[maverick891]

I would like to see multiprocessing .. seems to offer almost both with only a slight disadvantage to those running it on a single box but if they have a single box which can take it all maybe it can take a bit more.

[rilex]

* It wont be possible to make multiprocessing version out from the multithreaded version. ( who thinks its possible - go to hell ).

Seems like an odd limitation with Mangos.

I'd vote multithreaded. I would think there are more users with 2+ cores vs 2+ machines.

[caeruleaus]

well with multi-processing it can support much larger servers but most people don't have 3 computers lying around to run each process on. multi-processing (or clustering as most people know it) allows you to reduce the load on the server by running it off multiple pcs at the same time. So even if one of those computers loses connections the other ones can pick up where that one left off (i think this is what the aim is anyways unless im completely mistaken). Most companies use multi-processing but for general use people like mose of the mangos community i think multi-threading would be the best choice. I already voted multi-processing but i think i'd like to change my vote.

[vdesai]

If you understood Derex you can see that he said even with Multiprocessing you can start multiple threads on one machine to make it multi-thread on one machine.

[Dythzer]

delete this

[VladimirMangos]

I only want say: make pull for selection development way is nonsense

[Aligatro]

I vote for multi-processing , because it's going be better for users. One mangos process crashes, but others will continue to work, so only one part of the world will disconnect. And on dual core servers you can simply put one process on one core and second process on another core. What if we create one process per player? Will it slow down a server/increase memory usage on overall ?