Start MaNGOS Four from where?

[Loghman]

Hi

Me and some of my friends are interested to work on a core for MoP. I see that MaNGOS Four is on development level. We compiled MaNGOS Two and test it but it seems that MaNGOS Four doesn't work.

We want to start work on MaNGOS Four for 5.4 to make it playable. we didn't work on any cores before. we want some information about the whole project and some guides to understand it. Our problem is that we don't know about the structure and architecture of MaNGOS. We should start our journey from where?

[TheLuda]

Sorry for the late reply, but the team is more or less drawn away from mangos due to work and university workload.

First, you should probably give more details what in mangos four did not work for you. Where did you build and install? Windows, Linux, etc.

If you really want to help there, it will be a lot of work probably requiring knowledge on reverse engineering the game client.

Since you never have used mangos, I would instead recommend to start with understanding how the database works, which informations are stored in the clients DBC files, etc. The basics need to be learned first.

Also the question is, how well you know C++. If you're just starting, updating the server for a new client version might be a task too large.

[antz]

Hi

Me and some of my friends are interested to work on a core for MoP. I see that MaNGOS Four is on development level. We compiled MaNGOS Two and test it but it seems that MaNGOS Four doesn't work.

We want to start work on MaNGOS Four for 5.4 to make it playable. we didn't work on any cores before. we want some information about the whole project and some guides to understand it. Our problem is that we don't know about the structure and architecture of MaNGOS. We should start our journey from where?

Some of the key areas of research needed atm are:-

* Details of the revised dbc layouts

* Latest Opcodes offsets.

and so on

Once we have that major step complete, then we can move onto actual game mechanic changes and content

[Loghman]

...

Also the question is, how well you know C++. If you're just starting, updating the server for a new client version might be a task too large.

I am a Delphi programmer since 9 years ago so my main language is Delphi.

But before I start Delphi I knew C++ (for the University all of our assignments where on C++ so I had to learn it. btw I have a bachelor degree in software engineering )

I am familiar with C++, C#, Java, PHP and as I have my own business around automation systems and accounting I wrote some software in all of these languages. for instance for windows APP Delphi and C# and for the web services which work with those App I am using PHP and for the mobile I use Java.

The latest project that I am working on it now is a software which analyze syslog messages which are generated from Cisco firewalls and it is kind of security software that works on the network. so I know about network and it's protocols and socket programming and also familiar with SQL Server and MySQL.

I didn't ever do the reverse engineering but it would be nice to do it in this project.

The first version that we compiled is WoTLK (MaNGOS Two). We run it and it was so amazing that I can login to the game(to be honest I became astonished. and I even did a quest)

We have WotlK Client 3.3.5 and MoP Client 5.3.0.

I thought that dbc files are something like this link. http://msdn.microsoft.com/en-us/library/st4a0s68(v=vs.80).aspx

But after I myself read the dbc files with a piece of code that I wrote in Delphi I understood that it is something else and I found this link http://www.pxr.dk/wowdev/wiki/index.php?title=DBC

We tried to compile the MaNGOS four at any cost ;D but we found out that we should understand the whole project to achieve what we want. Since the current ManGOS four needs lots of editing and fixes mainly about the dbc extractor and working with the extracted dbc files. some dbc files are removed in mop at it needs some major fixes for talents and spells and some parts of current MaNGOS four is still cata and not updated yet so each part we fix we find another related part to fix so to be honest we decided to do a clean work for 5.4 instead with a better knowledge of the whole project architecture.

I and my friends were players of Blizzard from TBC to beginning of Cataclysm and we quit it about three years and we knew the game so we are not just programmers but also are players

This is me and my friends are some like me. None of them like me didn’t ever do reverse engineering.

We are not native in English language so I apologize for my bad English

[antz]

We have since moved up to 5.4.2 - and the 5,4,7 branch has just been created.

- Hopefully we can start to play catch up now

[antz]

The current extractor will extract the 5.3.0 dbc's correctly, I am currently trying to get the vmap extractor to work as well.

Extracting the dbc's is just one side of the challenge, actually mapping what each of the columns mean since cata is the main part of the work.

- I did a fair amount of work in identifying which dbcs had not changed, but there are still quite a few that have.

We appreciate any help you can provide to help get things updated and moving along.

Please also check out our wiki at github.com/mangoswiki/wiki

[Loghman]

...

Please also check out our wiki at github.com/mangoswiki/wiki

Thanks for the link.

I see your wiki but I want something more clearly (and the wiki need to be complete, there is a lot of TODO there. and we don't want just compile the project). Something likes a map. For instance I draw something that may clear my purpose. This scratch that I scanned is http://loghman.ir/MyImages/Pic1.jpg

this drawing is not the exact and complete project that I am working on but it can show how it works. We want to understand how does this huge project that U built work! There is many .cpp and .h files. If we want to understand the project it will take a lot of time to understand the relation between all parts, but if there was a graph, map or whatever U call it to make it clear how these parts work together it would be easier to jump to the work. btw I don't mean that we need a complete document with UML. If there is an ugly scratch like what I just draw, it would be helpful.

[TheLuda]

There is no drawing, no diagrams whatsoever.

We've just started building proper docs and all of the missing things in an effort for mangos-zero. Have a look at https://bitbucket.org/danielsreichenbach/mangos-zero-database/wiki/

It's not for mangos-four, but there are similarities and it should cover a few things in terms of knowledge and information which are still valid for mangos-four.

But: beyond that, any documentation needs to be written. This project never had many people interested in crafting docs, except for me probably.

[srajaee]

The current extractor will extract the 5.3.0 dbc's correctly, I am currently trying to get the vmap extractor to work as well.

well when I wanted to extract the DBCs and maps the extractor was detecting a different version than my client. I have 5.3.0.128 but the extractor detect 5.3.0.116 , I had a look at the source code and see the MPQ file it is reading the version from does not have version updated by blizz. so I manually updated the version in the MPQ and it fixed the extracted files version mismatch problem. however after that I am getting a new error saying 40 of DBC files are not matched in fields or types as I checked the DBCfmt.h , I tried to edit some of the strings in DBCfmt.h or some of related structures in the code just to ignore some errors for now and see if I can get the server running to the login screen but then in some points it needs major editing of code as well mostly for talents and spells since most of the project is still Cata and not updated to MoP

[antz]

That is basically where we are at, please feel free to issue a pull request containing any updates that help us along

[Loghman]

...

Have a look at https://bitbucket.org/danielsreichenbach/mangos-zero-database/wiki/

...

I see that. It is more about the database. I think I need something around the code. Of course there are comments on the code, but it is not easy task to understand the logic. After I tried to read the code from somewhere I just started with the game part in the solution. I found link related to

Morgan McGuire and after I understood that he is a project manager of G3D Innovation Engine. so I understood that there is an engine and the whole work have not be done just by the folks who built this MaNGOS. Yet I have no idea how this project works and I should try to read the codes as much as I can. To be honest it is not easy for me. I know little C++ but I see that I have to learn more about it and my knowledge at this point seems not be enough, but I think I can learn while reading the code. (trying to read of course! )

It seems that there is two important parts that I should consider those as the heart of this project. mangosd and realmd.

I think first of all I should tell U what I am thinking and if there was no engine what should I do to write such project. If my thinking way is not true tell me to correct my way.

We have a client. It has all things in itself. It has all items, all functions that can show us anything. for instance it has function for walking and just by sending few parameters it show us moving the character. when I press W to move forward it must two function calls. first in my own client and second send the data to the server to tell him that I am moving forward and my position is (x,y,z) and server save my character's position somewhere. maybe in memory or maybe in database. I don't saving the position at any moment in database is a good idea. I think it is better for each character a thread started on server and when server is going to stop it or the player himself leave the game or his connection is lost save the thread information in the database. So for reverse engineering the server we need to extract some data from the client to get all manipulatable things. When my character is standing on floor of Orgrimmar I don't need to get data of buildings from server. I have the data. I just need the other characters' position and npcs(live or dead). So we sniff the data with something like wireshark. we understand what is need to be received and send by the server to the client. but there is a question for me. why a 3d game engine needed here? I accept that server should know about the buildings and obstacles but it doesn't need to render anything. does it? May be the developers of MaNGOS just used G3D Engine for calculating collisions. I don't know. and my goal is to understand how does this machine work.

There is another thing that tickles me. Maybe there is no need of change or manipulate the code of the Core to Run MoP server. Maybe if just we extract the dbcs and vmaps from client and giving those to the core it works fine. but I am kinda sure about this matter that it wouldn't like such thing. because what should we do with dbcs data? Those who wrote the Core before didn't know what will be pop up by Blizzard in future in the client application. system of game is changed in some parts. like talents build which completely changed from its descendant.

If it is possible in a few lines just say how does this machine work?

You may not answer. But this will leads me to dive in to the code. I may find out how this machine works or I may give up.

[TheLuda]

1. G3D is just used for collision detection purposes, as we extract data from the game client (namely the DBC files to jump start game definitions, and all map files) and calculate our own set of server side maps from these.

2. src/framework/ provides a few cross-platform abstractions, src/shared/ provides a library connecting the server to MySQL/PostgreSQL, OpenSSL and Zlib and also providing basic classes for reuse in the rest of the server.

3. src/game/ is just a library which abstracts the server side of communicating with the client, and reacts to any commands the client sends, with the proper replies, and does all the magic behind. E.g. show a friend list or available auctions.

4. realmd is the service which allow logging in, mangosd provides the functions from src/game/ thus it is usually named world server as it "runs" the game world.

5. Extracting data from the client will not by far make anything work. You have the data, but that still does not answer in which format the client and the server communicate. This is what is found out by watching the original server and client communicate and log their TCP packets. And then, you go ahead and try to make sense of them. Also many things from the client help us understand how this all works. But much is guess-work.

Does this help?

[antz]

There are several third party libraries included (in the dep folder), I think the library you have stumbled across is the one used for calculating movement paths for creatures/mobs in 3d space.

The server itself is split into two sections:

RealmD - controls user authorisation and connection

MangosD - it the world server, this reads data from the dbc files as well as from a world database.

In addition, it also uses .map, .vmap and .mmap files to help calculate the objects are and where they can move.

map files contain the actual terrain information including liquid data

vmap files contain the information of the vertical areas and accessable areas.

mmap files contain the pathing information it prevent mobs flying through the air in a straight line towards the player.

dbc files contain vital information that is not held in the database since it is regarding as read only information. i.e. race information, spell ids, names and effects etc.

Changing the talent build will imvolve changing parts of the server code to handle talents as well as changing the talent information in the database and also working out where the data is in the talent.dbc

I hope this helps

[Loghman]

Thanks a lot. :-*

Knowing these information is better than being ignorant.

Me and my friends will start working on the project soon.

Wait for our coming questions. ;D

[TheLuda]

Wait for our coming questions. ;D

Keep them coming, and if you happen to have some spare time, we'd welcome if you would document your learning process too. This would be great as we would improve our project documentation based on what you found missing.

[Loghman]

Keep them coming, and if you happen to have some spare time, we'd welcome if you would document your learning process too. This would be great as we would improve our project documentation based on what you found missing.

True! Documentation is an important part in my opinion. We try to not omit that part.

[antz]

I managed to get some more updates done this evening https://github.com/mangosfour/server/commits/master

I have tested this using 5.3.0.17128 and was able to extract dbcs, maps, vmaps and mmaps

[srajaee]

I have tested this using 5.3.0.17128 and was able to extract dbcs, maps, vmaps and mmaps

good job mate I checked the updates and could extract the vmap and mmap files however I still had to edit the version to 5.3.0.17128 in the MPQ file first since the extractor is still detecting 5.3.0.17116 for my client build while it is 5.3.0.17128 to make the extracted files have proper version info

anyways thanks for the updates. keep the good job mate

[antz]

which extractor was complaining about 17116 ?

[srajaee]

which extractor was complaining about 17116 ?

Map & DBC Extractor

[antz]

i will looking into adjusting the min required level down to 17116

[srajaee]

i will looking into adjusting the min required level down to 17116

it is already working with 17116 and is extracting the dbc and map files. the problem is it writes the version 17116 in the extracted files instead of 17128 and when mangosd try to read and use them it will look for version 17128 but find 17116 instead (while they are actually 17128 not 17116) and give error.

[antz]

Ahh i see now, thank you for that info

[srajaee]

no answer from a mangos dev?