[Discussion] Ingame bug report command

[Auntie Mangos]

Following up the ideas suggested in Playerbot thread

I thought of a more generic system to enable ingame bug reports.

For userside I think some "all-you-can-drink" command(s) like

.report <type> [hidden] <status> [guid or target or Shiftlink] <comment>

.reportTo <Whom> <type> <status> <comment>

with possible versions like:

<type> quest: <status> = 0 undefined (cannot be used)
                                    1 not-completable
                                    2 bugged completable
                                    3 looking good
                                    4 blizzlike            // Possible more, similar to project tracker
             Shiftlink = Quest Shiftlink
        npc:  <status> = 0 missing spawn
                                    1 wrong position
                                    2 double spawn
                                    3 bad stats
                                    4 bad candy (ie equip)
             guid or target (for status > 0) the npc of whom we are talking
        go:   <status> = 0 missing spawn
                                    1 wrong position
                                    2 double spawn
                                    3 bad stats
             guid or target (for status > 0) the go of which we are talking

[hidden] Mark report as hidden

The reportTo command should behave the same, only that this report is for automatic report to the project-sites (ie something like the udb quest tracker)

Now about the internally mechanism:

The reports should be stored in a database table, with this format:

(id, whom, process-state, type, status, entryOrGuid, comment, reporter, pos(map, x, y, z), version(core, db, scripting))

Where

whom = To Whom it should be reported (0 for .report command)

process-state: 0 = normal, 1= hidden, 2=to-sent, 3=validated, 4=sent // Maybe some additonal states like fixed, etc..

type, status, entryOrGuid, comment from above

reporter = user that reported (I think char-guid is fine

pos = position where the user was

For the reportTo command, I some additional special configs are required (for each used WHOM):

- WHOM_NAME (ie "udb" for the udb quest tracker)

- WHOM_USER (some random string identifying the user on udb-tracker - iE a hash value of AdminsUsername+RandomSeed )

- WHOM_BEHAVIOUR (validate before submitting / direct submit)

So what is to discuss:

* types - status combinations

* ReportTo - must agree on some sort of way to send datas (format)

(Actually I have no idea how this could be handled when down to coding )

* Possible GM/Admin addon to work with these things?

* Role based command rights might come in handy for reportTo command - so that only knowing users are able to use it

* Interaction to Bugtracker/ Website - I don't know how to handle such things

* Feedback from our site hosters, if there would be possibilities to create wrappers to ie forward to github issues or other trackers

[Auntie Mangos]

Did anything ever happen with bugreportaddon: index.php/topic,16068.0.html/?

[Auntie Mangos]

It's better just write LUA addon using API mentioned in this topic. Or you can even copy Blizzard's bug reporting addon from PTR client and use it

[Unkle Nuke]

Wouldn't it be simpler to just find some way to forward GM Tickets reporting bugs to the tracker?

[DaemonCantor]

Personally I think both ways are good....a better way of sending reports to the local GM for quick fixes and a way to forward to the proper group for the harder fixes like a bad DB problem the GM can forward to UDB's error reporting not just to mangos's Lighthouse...

Good catch Schmoozerd!

[TheLuda]

I'd rather see bug reports hard wired to go the projects instead of a GM. There are tons of servers out there, but only a tiny fracture of those thousands (probably five to ten) actually report issues back or even send in their fixes to the projects.

[shlainn]

I see one huge problem with reporting all and any bugs directly to the project base, which is custom mods/fixes. I see no way in enabling this kind of reporting only under the circumstance that an unmodified DB/Core/SD2 is used. Therefore I would definitely vote for the local GM thing.

[RANT]

But seriously, gentlemen, what is our main problem here: Getting Bug reports on new Issues, or really finding people who are willing to sift through the copious amount of Bug reports in this forum as well as UDB/SD2 and invest some real work in fixing them? I think that a systematic sweep through the various forums collecting, sorting, classifying and verifying bug reports (possibly in an automated way) would be much more helpful...

[/RANT]

[shlainn]

Now that I have actually read your post and not just skimmed the surface I have some technical thoughts:

* types - status combinations

If you want to combine it with a web tracker, obviously the status combinations should be identical...

* ReportTo - must agree on some sort of way to send datas (format)

* Interaction to Bugtracker/ Website - I don't know how to handle such things

If the tracker is off-site (like the UDB quest tracker) mangos could send data via a HTTP connection. This is not difficult (except for the fact that MaNGOS would need some more libs to handle it). It is also possible to put the reports in a database table and use a separate reporting tool which could run every few hours/days to send the database content to the tracker. A unified data format (JSON, XML, ...) would be a prerequisite for that.

If the tracker has access to the same database as mangos, the database table solution fully applies.

* Feedback from our site hosters, if there would be possibilities to create wrappers to ie forward to github issues or other trackers

I see no problem to create a script for the UDB quest tracker that could receive and incorporate reports from anywhere, as long as we stick to a parseable format.

[tyrael]

well I still believe that reports are not inoff... for gameplay I mean...

It is very useful that the report is linked to tracker but what about players?

we need an additional db where quests that do not work to be added by the gms... also this db server for a custom future to your idea.. I am suggesting using that db to give auto complete to the reported q, IF that q is approved by a gm. In the moment when a certain player will report the q, by it's name the server will check if that q is in db and if it's approved as auto complete by a gm... in that moment server will give that player complete to q... I repeat only after he reports it!

Also to avoid overflow of reports about a quest, we can make that every player that takes that q to receive auto complete, only if the gm approve it. The gm must see the clear report and decide if in that report there are in off information about that q bug.

Respect

[Schmoozerd]

@ Unkle Nuke

I think not:

- Tickets can be written by anybody, hence no security controll (like .reportTo only for some usergroup "tester")

- Tickets would have to been parsed, which can get very complicated I think

@shlainn

"If you want to combine it with a web tracker, obviously the status combinations should be identical..."

The problem is that ie quests have very different states than npcs, so I cannot see how we can combine this reasonable

About Sending:

As you just said in IRC - maybe a "simple" script (python, whatever) is superior and easier to use than to add unneeded stuff to mangos

About custom content:

Therefore I thought of different states (to-sent, validate, sent) which would enable an admin to give some rough check before forwarding (marking for forward) a report - ie an admin would know that ie DK starting area is totally custom, so he could skip all reports related to this area

OT:

this Topic is located in "Core Modification" which is a part of "Developer's Corner

Forum"

I don't care about players.

[Patman128]

I think the best would be to have a table that is filled with user data reported with the command. Then a python script or something like that can take the table contents and send it to mangos (via some URL as POST data or something like that) that admins can choose to run.

[shlainn]

@Schmoo: How hard would it be to create a kind of chatbot to whom errors can be reported and who will ask (configurable) followup questions?

Example conversation

Player: /t ReportGuy quest <Plagued Lands>

ReportGuy: Hi Player, is the Quest "Plagued Lands" completable?

Player: No

ReportGuy: Your objective is to Use/Cast Creature "Captured Rabid Thistle Bear" (Rabid Thistle Bear Captured). Is that possible?

Player: No

ReportGuy: Why?

Player: When I set the Trap nothing happens

ReportGuy: Thank you for your report

Player: /t ReportGuy npc <Captain Placeholder>

ReportGuy: Hi Player, what is wrong with the NPC Captain Placeholder? (location, values, faction, spawn rate)

Player: location

ReportGuy: The NPC Captain Placeholder is currently spawned in 1 Place: 1x Menethil Harbor

ReportGuy: Should he be spawned more often?

Player: No

ReportGuy: Should he be spawned less often?

Player: Yes

ReportGuy: Should he be spawned in other places?

Player: No

ReportGuy: Do you have any more comments I should add to this report?

Player: No

ReportGuy: Thank you for your report

[Unkle Nuke]

The greatest concern is the degree to which trust becomes a factor when reporting bugs. In other words, how reliable is the information? This depends on each layer involved in the network reporting the bug. Primarily, you must have some way to validate the integrity of each report. There musty be some way to know the exact makeup of not only the server, but the client involved. Things like custom patches, additional mods, alterations of the database, and even the addons used by the client can lead to bugs not originally present in a "stock" configuration that uses only a clean core, script library, and database and a clean client that uses no addons. Even the OS and hardware used can lead to problems. How do you sort such issues apart from bugs caused only by the clean MaNGOS core, database, or scripting library?

There is also the ability of the end-user to make accurate reports. shlainn's idea of a scripted Q&A would guide the user into providing more useful reporting, but that would begin to break down without cooperation from the server admin and/or developers in accurately judging which bugs are caused by any modifications to the server, client or just simple user error.

On paper, this automated bug reporting sounds great, but, in the end, it really is up to each server admin to work with the MaNGOS community. We can't force such behavior and assume any level of trust in the data provided without knowing the reliability of the client, server, end-user, and administrator. Therefore we must rely upon the goodwill of these private servers to help make MaNGOs, the database, and scripting elements better. Leechers are as much a part of this community's troubles as the bugs being dealt with every day. Much as we may hate that, it's part of being open-source. MaNGOS is not the only such project to have its share of those who take without giving back.

[Schmoozerd]

@shlainn

Implementing such a dialogue reporting system would be much more work - starting with the need to whisper to a virtual entity.

Likely the cleanest way to do such a thing would be an independend project with one of the pseudo clients (login and chat commands)

However:

True this would improve the quality of reports by players.

But honestly - most players are stupid anyways so the bug-reports will still be ill written

This was actually the main reason why I suggest two commands (one for normal players) and one for a selected group of users - which also can be used in a checking way

Also it is slower, especially when it comes to the part of submitting the "working, good or blizzlike" states of quests (which are also usefull information).

Because actually we know that 70% of quests are good, so a dialogue is much more work than a copy/paste of ".reportTo udb quest <shiftLink> 3"

@UnkleNuke

UNfortunately yes, we cannot force anybody to work with/ for us.

But we can try to make it easier - and this is where these automated functions come in

Also I think that a couple of reasonable bug-reports are woth a few ill reports.

I observe that whenever I start playing wow I notice a couple of bugs just while walking around. - And many other informed players will notice such, too.

And often such things require very simple fixes (like ie a quest-item drop chance) - but if no one cares to report, it takes ages until they are caught.

This is the thing it is about the many small bugs that simply exist and no one knows of

[Unkle Nuke]

I'm all for any tools that will speed up fixes and improve workflow efficiency. I do think your idea is a good one, schmooz, and the concerns I've voiced are purely academic, not from any real objections to your proposal.

Perhaps, to increase reliability, this modification can have three levels. One would be for general reporting by the common end-user. The top level would be for the administrator/developer. I propose a middle one, for those designated as testers or at least knowledgeable enough to offer good reporting. Each level could have a tag, with those from the top level having a higher level of priority/accuracy and being tagged as such.

Just be prepared for a flood of useless reports if this core mod finds its way into repacks!

[Schmoozerd]

hmm, I see no reason to not use a lua addon.

Infact this might even work as basic guide for a report.

However somebody (not me) would have to write this addon.

This would have the advantage that this whole discussion topic would boil down to

* An Addon which gives a well formatted string

* A tool that parses a table and will be able to work (ie forward to some trackers) with this formatted string

* Maybe some tiny additions to the characters.bugreport table to support some additonal noting of states