Content Development Tools

[shlainn]

As always this is more of an braindump/motivational/thinking out loud post. I would like to go in a MaNGOS University direction with this, but right now my knowledge is too limited.

I have been involved in some TBC-DB development (Hi X-Savior!) and most prominently made the quest trackers for UDB and TBC-DB (mangos-one testserver) (udbtrac.dyndns.org / tracker.mangosone.org), so I can say I have been staring at the DB (in phpMyAdmin) for a while. And it is a reasonable pain.

Example 1: To find out who/what starts a quest, you need to check at least 3 different tables.

Example 2: To find out all possible ways to get an item (loot, vendors, quest rewards) you need to check a good dozen tables.

So, if we go to pipedream county for a short moment, and say, we want to create a MMO using MaNGOS as the engine, [insert your favorite SQL tool] is probably not the most efficient way to create world content (I am only talking about world content. NOT art assets or such. Or at least not one you would dump onto your creative crew of hipsters who run off screaming if they see something that even remotely resembles code *shudder*

Someone once said that developing content creation tools takes about as much time as the game engine itself - and I believe this is an area where MaNGOS is lacking. Sorely. (Although, I am not sure if this is intentional to keep the entry level artificially high and make people learn SQL instead of clicking around some fancy GUI)

So, in the wake of starting into Cata/MoP development (and the spirit of fresh start that theluda and Salja are spreading) I would like to propose a side project to develop a user-friendly (browser-based and thus platform-independent??) tools suite for database/db-script/(EventAI??) development for the community to use. I know that there are tools (in particular for EventAI development) - these would be included/improved into this (with proper credits, of course)

Two final question: Who would join such an effort? (UnkleNuke, what about this??)

And maybe from the experience of other DB devs and people who failed to become a db dev due to the complexity of the underlying system, what specific tools should be included in such a suite?

[michalpolko]

This is a good idea.

For GUI I can recommend ExtJS library, it is a very friendly gui lib which can help in creating such browser-based tools. Since I'm working with ExtJS for over a year and I really like this lib, I could do this.

What we need for this is a list of things that require such tools.

[lillecarl]

This could be developed until forever, i guess something like the old ascent database editor is a good start (creatures, items, quests, queststarter etc) And the add EventAI, idk....

[Svarr_]

Are you talking about an SDK or about a tool like quice?

[shlainn]

Are you talking about an SDK or about a tool like quice?

A tool like quice would be part of a SDK, I guess. So, this would be going more to the SDK direction.

[bugeyedcreepy]

ooh, Awesome! This might actually be something I could have meaningful contribution to!

[Unkle Nuke]

If, by "content development tools", you mean a GUI utility, where users simply plug in values and click a button to generate database content, that has been done to death.

WoW-V is a web-based example, where one can create items, gear, books, NPCs, vendors, portals, and even quests.

However, nobody has ever managed to combine all those features into a single application and almost none work well enough to avoid the necessity of hand editing for full compatibility. All the existing apps are outdated, as well.

I'd love to see a super-app, that allows for nearly unlimited database content creation, even custom stuff by using dead IDs in the client. How you would simplify things like flags, types, and IDs so they are more friendly to unskilled users would be the real challenge.

[shlainn]

I took a brief look at WoW-V. Looks like a VERY thin front end to the mysql database to me

I was thinking more about a very intergrated content editing tool, like Bethesdas TES Construction Kit (except for the terrain editing) with access to the relevant tables of the database.

For IDs it's rather simple: As a (trivial) example if you create a quest from NPC "Shlainn" to NPC "UnkleNuke" you don't go looking up the creature ids of those guys, but you type in the names and the tool finds the relevant entries for you. The objective is to... dunno, kill 10 Evil Forum Trolls, so you don't go looking up the creature ID for Evil Forum Trolls, etc.

Types, Factions and Flags are a bit more tricky, I guess. At the simplest level, it is more convenient to set flags by checking boxes than by hand-calculating them. For more complex things one could create a "wizard" where the user answers some questions and gets the proper flags set (Like "Is this creature a guard?"...) I will have to learn more about those flags before I can give you a good answer here.

(And it is a great opportunity to learn more about the databases and their interactions for those who want that.)

Shlainn

[Unkle Nuke]

I forgot bitmasks in that list. Have to account for those, too. I understand them in principle, but my eyes cross, brain gets squishy, and I begin drooling on myself as my neurons go into vapor-lock when I try to actually pick one apart.

I've read the UDB wiki (yes, really!) and recommend it highly to anyone who wants to understand the database schema for MaNGOS. Most of it is really just getting the correct IDs and values into the proper tables and columns. I even took a stab at making my own custom items and gear by hand editing, back before patch 3.2.2 made it difficult with that stupid hard-linking of IDs.

[shlainn]

The UDB wiki was a tremendous resource to building the UDB quest tracker (http://udbtrac.dyndns.org). Which I built. This is the next logical step.

[Unkle Nuke]

Perhaps you might also include some sort of WoWHead style tab to allow users to compare online data sources with the pserver database and client DBC files, to help pinpoint DB errors?

[shlainn]

Huh? Whaddayamean by WoWHead style tab? Check out the tracker @ http://tracker.mangosone.org - it even provides wowhead links for... everything!!

[Unkle Nuke]

That's what I meant. I'm hoping that feature will be in this new tool you're proposing to develop.

I'm getting very excited with anticipation at the possibilities your work will open up for hopeful content creators!

One extra thought I had is more on the selfish side...

Although using a web interface would help with cross-platform compatibility and using online resources, like WoWHead or Thottbot, will make data retrieval more lightweight... how much more difficult would it be to create this application as an offline version? I imagine an offline database like WoWHead would have to be especially difficult due to sheer size alone.

I ask due to my very limited web access. Using tools and resources online is not very easy with this stupid phone and its tiny screen. However, I'm not so important as to demand offline capabilities just to accomodate my needs! :lol:

[michalpolko]

Working offline you would simply get errors while loading wowhead content, thats all.

I mean, working with such tool in web browser implies installing apache or some other web server.

[shlainn]

Exactly, you can just install the tools on your computer and use a local server for this. On windows you can use something like USB WebServer from www.usbwebserver.net/ - on linux just install apache2

[antz]

count me in, the database has long been a source of frustration to me and seem overly complex in places.

I did some work a long time ago in working out what went where in the various tables and attempting to create friendly views to display the information in helpful ways.

From memory...

there were some areas where lookup tables were missing from the Db and only contained in the server codebase. Which meant that any tool using the DB would need to be compiled with the lookup tables from the Mangos Codebase.

Personally, I'd like to move some of those lookup tables out of the code and into the DB thus meaning that the DB can be referenced and queried as a separate entity from the Mangos server code.

From my vague memory (may be different now)

RaceId/RaceName are stored in the codebase, not in the DB - but the ID's are used elsewhere in the DB.

If thats no longer true, then great... scrub this post - But in still in

[shlainn]

Hy Antz,

I will be setting up a branch of the tracker to build on shortly - any help is welcome.

Actually RaceID/RaceName is in Races.dbc, though

[antz]

Hy Antz,

I will be setting up a branch of the tracker to build on shortly - any help is welcome.

Actually RaceID/RaceName is in Races.dbc, though

Ahhh, might be dbc rather than code then

- might look at trying to squirt the contents into a table from the dbc file as it would make DB only lookups so much easier - possibly a simple tool that drops and recreates the tables in one nice step

[antz]

because with the actual game spellid's in a lookup table, we can instantly check for invalid spells elsewhere in the db... just a thought

[shlainn]

actually the tracker does that as far as it is needed. check out the code at https://github.com/shlainn/tracker

[Mark_]

This probably is extremely foolish question, but has anyone looked at using LibreOffice Base to do what is being suggested?

[shlainn]

Uhm... no. And quite frankly I do not see how you would apply libreoffice base as a specific content development tool...

[Unkle Nuke]

I presume it is because you can read, edit, and create database information with LibreOffice Base. Most office productivity suites have similar apps.

The advantage is using it as an IDE. Being open source, there is the possibility of modifying the Libre code into a custom dev tool for the MaNGOS database, including the plug-in scripts feature of Libre that would allow linking in other tools or scripting more features.

[shlainn]

I presume it is because you can read, edit, and create database information with LibreOffice Base. Most office productivity suites have similar apps.

The advantage is using it as an IDE. Being open source, there is the possibility of modifying the Libre code into a custom dev tool for the MaNGOS database, including the plug-in scripts feature of Libre that would allow linking in other tools or scripting more features.

I never viewed an IDE as an advantage, but that may be me...

AFAIK Libre Office is not using SQL as database backend, making it much harder to base any efforts on.

[Unkle Nuke]

The backend should be trivial. A plug-in can be scripted to allow any database to be used, but it helps to think of using Libre as just an editor. SQL data can be edited or created, then applied to MySQL in the usual way.

You did open this can of worms.

It was just meant to show one possible way Libre could be used, like converting csv data into SQL. The tools you're creating will be less cumbersome than an office app and have better features.