Jump to content

How did the dev team do it?


kurJay

Recommended Posts

How do we do it?

The short answer is actually more boring than you think... hours of studying hex code gathered from the client and network traffic with the server.

The long answer I'm about to offer would have seen me burned as a heretic for spilling precious secrets, if this were still the old days, but MaNGOS today is about being free in every way. The three pillars of the MaNGOS Open philosophy are Open Source, Open Community, and Open Learning. So we all do our best to make sure the code is free, our door is open to everyone, and knowledge must be shared.
 
A more complete answer must be prefaced with a brief trip to the past. Bear with me, if you please.

Game server emulation has been around far longer than WoW itself, beginning in the late 90s with multiplayer games like Quake. Back then, these games often came with an official server you could install from the game disk so players could host their own multiplayer games. People smarter than me got curious and reverse-engineered these servers to create the foundation of knowledge and procedures that are still used to this day.

MMOs use a centralized server, accessed over the Internet. One of the earliest examples is Phantasy Star Online, released for the Sega Dreamcast in 1998 and Windows a year later. So how do you create a server for a game when you don't actually have the server itself? The two basic tools used for creating a server emulator from scratch are the debugger and packet sniffer. I'll say more on that a little later.

What I do know of the early genesis of WoW private servers is second-hand knowledge gained from the devs who'd been around nearly from the the beginning. It all began when a simple WoW sandbox evolved into a full server called Stormcraft. Early WoW p-servers were primitive by today's standard and closed source. Much of the game's features did not work correctly, if at all, and there was little anyone could do because the various devs kept their program code and knowledge secret. WoWDaemon (usually abbreviated to WoWD) was the beginning of the modern projects you see now, programmed in C++ and using an SQL database.

So why the history lesson? Because every WoW server emulator in existence today owes a debt to those who came before. Whether it's MaNGOS or Ascent, or one of their many forks like Trinity, Cmangos, or Arcemu... we've built on over a decade of work by hundreds of developers. MaNGOS was started from a leak of the old WoWD server source. Standing on the shoulders of giants, as it were.

The way this was all done involved using those two tools I mentioned earlier. First, a debugger is employed to dig into the heart of the WoW game client, reverse-engineering the client's inner workings to find opcodes and data handlers. Then a packet sniffer was used to save all the packets sent between the client and server during an actual game play session, also called "sniffs". A hex editor was frequently used to analyze file formats and the captured packets, allowing bright fellows to develop specialized tools to more efficiently sift through the client and network traffic for valuable data. Once it was understood how things worked, it then became possible to create compatible program code to make use of the methods and data that makes WoW playable.

Since that time, specialized tools have been created to more efficiently get at the precious data needed to improve existing p-servers and keep up with new expansions so they can also be emulated correctly... eventually. It has taken all this time to get this far. I'm certain it will take longer still before anyone can truthfully declare any WoW p-server is 100% complete.

That's a broad overview for you. As for the finer details, feel free to ask more questions. Either myself or someone will be happy to provide you the best information we have.

I hope I've given you the answer you were seeking. Remember, MaNGOS belongs to you!

  • Like 6
Link to comment
Share on other sites

Great write up Unkle :) Sorry we had to drag you out or retirement yet again! Me & @antz keep doing this hehe :)

@kurJay
I hope Unkle's reply has helped you understand a little more where MaNGOS comes from, as Unkle said, if you have anymore questions feel free to ask. Hopefully I wont have to pull Unkle out of retirement again ;)

  • Like 2
Link to comment
Share on other sites

Even if that were true, the point of kuJay's question was how a WoW server emulator is created from scratch without access to the actual retail server code.

You are correct that much of the effort for older clients now is directed toward implementing and fixing features to replicate the game play as it existed at the time a particular client was current on official retail. That doesn't mean no further work is needed on the other areas. A lot of assumptions about the reliability of old research led to the perpetuation of bad information and data. There are still many opcodes whose functions are unknown. Certain file types were incorrectly analyzed in the past. Maps are only just now being correctly understood in their proper context, which has finally allowed transports to function properly after many years.

There is still much data mining and reverse-engineering that needs done, especially for more recent expansions.

  • Like 2
Link to comment
Share on other sites

  • 4 weeks later...

What I said still applies, even going all the way back to vanilla WoW 1.12. We still don't know everything as fully as we need. Otherwise, MaNGOS would have had 100% retail support for older WoW clients years ago.

The development history of WoW private servers goes back a long way, with much of what is known based on the work of mostly hobby coders. They had no formal training and relied on their intuition as much as skill. On top of that, the constant push to get features working by any means, even if it was only "good enough", gave no incentive to get things totally correct. This eventually lead to guesswork somehow becoming gospel. Those who chose to question the status quo had to face the wrath of the developers in charge. Politics cemented this further with certain individuals seeking to protect their status within the community by keeping their knowledge for themselves.

It's only grows more complicated by the fact that each new expansion made changes to file formats and data structures that required starting the research all over again.

  • Like 1
Link to comment
Share on other sites

I think that our core is not 100% retail because a refactoring or the code base is necessary to do so. But the functionnalities are all available in the core though.

Other private servers demonstrated that it was possible to be vanilla-like as much as possible. The things that we don't know are some calculations and formulas but not "technical handling" such as unknown opcodes or so.

  • Like 1
Link to comment
Share on other sites

You are indeed right! The code base does need refactoring. Antz has been reworking parts as he goes along, but overhauling a decade of work is a mammoth task.

There are opcodes that still have the status of 'UNK' in the cores. Some likely have no bearing on a private server, such as Battlenet client interoperability.  There are a few that may reveal themselves to be necessary pieces to the puzzle. My hope is the formulas can be fixed correctly by researching archives of sites like Elitist Jerks and even making use of old versions of utilities like Rawr and Simcraft. I'll have to leave that one up to smarter people. Math and I have an uneasy truce, at best. :$

Link to comment
Share on other sites

  • 4 years later...

Let me just create an account here to write THANK YOU to all the devs who contributed to the WoW unofficial servers.)

It was about 15 years ago that it allowed me and my friends to explore the World of Warcraft since paying for official servers wasn't really an option.

It still amazes me (more so being a programmer myself) how groups of enthusiasts can reengineer and build whole game server from scratch. Back then I thought it might have been something like applying "crack" to the official server distribution (something like warez games), but when I started to look into how those servers were actually made 😁, it just blew me away how wrong my assumptions were.

Thanks again!

Link to comment
Share on other sites

On 11/12/2021 at 7:05 PM, lukasradek said:

Let me just create an account here to write THANK YOU to all the devs who contributed to the WoW unofficial servers.)

It was about 15 years ago that it allowed me and my friends to explore the World of Warcraft since paying for official servers wasn't really an option.

It still amazes me (more so being a programmer myself) how groups of enthusiasts can reengineer and build whole game server from scratch. Back then I thought it might have been something like applying "crack" to the official server distribution (something like warez games), but when I started to look into how those servers were actually made 😁, it just blew me away how wrong my assumptions were.

Thanks again!

Thank you @lukasradek it is a big team work.

You can also bring your own brick to the building :) 

Cheers !

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. Privacy Policy Terms of Use