Increasing Mangos Stability

[Auntie Mangos]

In response to this thread I was wondering not so much how long you can keep your mangos up, but what you did to improve stability?

I am hoping to create a list of methods that will help everyone get a more stable Mangos experience, so please, if you have any tips (that still work!) post them here! I'll start with our changes:

Core / Code modifications

• * MTMaps stability patch 0.13 / 0.14 & MTMaps stability patch 0.12 (Only for use in combination with mtmaps! Revision 8754)
  * Prevent stack move during enchant (0.12 branch)
  * Object::GetUInt64Value crash fix.
  * Increase stack size (windows) - Update: fixed in [9486]
  * Map stability patch (0.12 branch)
  

Database modifications

• * Removed Lightning Shield from Razormane Mystic (0.12 branch)
  * Cleanup instance lists (topic)
  * Update Onyxia's Firebreath ability (topic) (0.12 branch only?)
  

Configuration Settings

• * MaxCoreStuckTime = 120 - Setting this value too low caused crashes during heavy lag (for example during the daily database backup).
  * DetectPosCollision = 0 - Prevents certain freezes
  

Please, note that all suggested changes improve stability by features disabling and hacks adding.

[Auntie Mangos]

This is really a interesting thread FragFrog because Stability is in my opinion a target of a MMORPG server.

Threads like http://getmangos.eu/community/viewtopic.php?id=10216 , http://getmangos.eu/community/viewtopic.php?id=10642 or http://getmangos.eu/community/viewtopic.php?id=8896 show that a lot of users got "custom patches" or "stability patches" to increase the uptime but most of them aren't sharing the patches here.

I personally can only suggest to use a clean Linux system and a dedicated MySQL server (Try to cache most parts of the database). That together with a 1year~ old Mangos (Mtmaps) revision gives you high stability and less lags but it also needs a lot of resources and got tons of exploits. (You can fix a lot manually but that is quite annoying and doesn't supports the project at all)

Here seems to be also a hacky fix http://getmangos.eu/community/viewtopic.php?id=10661 (Removing another assert )

I look forward to collect some useful informations here

[Auntie Mangos]

Hi a have 1 problem with mtmaps using rev 8573. I currently using derek's mtmaps (ACE), because i'm unable to use this http://getmangos.eu/community/showpost.php?p=90117&postcount=49, if I use that openmp mtmaps, all threads starts in 7X% of cpu ussage and raising...and infinity's (raczman, etc.). I currently have gcc 4.4.0. Any ideas to do with this crash or somebody with my openmp problems?

[Core was generated by `./mangos-worldd'.
Program terminated with signal 11, Segmentation fault.
[New process 1355]
[New process 1357]
[New process 1365]
[New process 1361]
[New process 1362]
[New process 1342]
[New process 1367]
[New process 1352]
[New process 1350]
[New process 1364]
[New process 1366]
[New process 1349]
[New process 1358]
[New process 1359]
[New process 1354]
[New process 1356]
#0  0x00000000006a3630 in Spell::DoAllEffectOnTarget (this=0x7f59e6dacdb0, 
   target=0x10) at ../../../src/game/Spell.cpp:922
922        if(target->processed)                                  // Check target
#0  0x00000000006a3630 in Spell::DoAllEffectOnTarget (this=0x7f59e6dacdb0, 
   target=0x10) at ../../../src/game/Spell.cpp:922
   mask = <value optimized out>
   unit = <value optimized out>
   caster = <value optimized out>
   missInfo = <value optimized out>
   procVictim = <value optimized out>
   procAttacker = 16384
   procEx = <value optimized out>
#1  0x00000000006b1b44 in Spell::handle_immediate (this=0x7f59e6dacdb0)
   at ../../../src/game/Spell.cpp:2667
No locals.
#2  0x00000000006b0fb8 in Spell::cast (this=0x7f59e6dacdb0, skipCheck=true)
   at ../../../src/game/Spell.cpp:2641
   castResult = <value optimized out>
#3  0x00000000006f576a in Unit::CastCustomSpell (this=<value optimized out>, 
   Victim=<value optimized out>, spellInfo=<value optimized out>, 
   bp0=0x430e4a38, bp1=0x0, bp2=0x0, triggered=<value optimized out>, 
   castItem=0x0, triggeredByAura=0x7f5a3d410760, originalCaster=144773)
   at ../../../src/game/Unit.cpp:977
   targets = {m_srcX = 0, m_srcY = 0, m_srcZ = 0, m_destX = 1024.65173, 
 m_destY = 860.800659, m_destZ = -62.5263367, m_strTarget = {
   static npos = 18446744073709551615, 
   _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0xaebbb8 ""}}, m_targetMask = 2, 
 m_unitTarget = 0x7f5a2cafba40, m_GOTarget = 0x0, m_itemTarget = 0x0, 
 m_unitTargetGUID = 140061, m_GOTargetGUID = 0, m_CorpseTargetGUID = 0, 
 m_itemTargetGUID = 0, m_itemTargetEntry = 0}
#4  0x00000000007092af in Unit::HandleDummyAuraProc (this=0x7f5a5acbf420, 
   pVictim=<value optimized out>, damage=568, triggeredByAura=0x7f5a3d410760, 
   procSpell=<value optimized out>, procFlag=<value optimized out>, procEx=1, 
   cooldown=0) at ../../../src/game/Unit.cpp:6395
   dummySpell = (const SpellEntry *) 0x7f5a554f2fe0
   castItem = (class Item *) 0x0
   triggered_spell_id = <value optimized out>
   effIndex = 1
   triggerAmount = <value optimized out>
   target = (Unit *) 0x7f5a2cafba40
   basepoints0 = 227
#5  0x000000000070c69e in Unit::ProcDamageAndSpellFor (this=0x7f5a5acbf420, 
   isVictim=<value optimized out>, pTarget=<value optimized out>, 
   procFlag=<value optimized out>, procExtra=<value optimized out>, 
   attType=<value optimized out>, procSpell=0x7f5a53c6cd40, damage=568)
   at ../../../src/game/Unit.cpp:11369
   triggeredByAura = (class Aura *) 0x7f5a3d410760
   spellProcEvent = <value optimized out>
   spellInfo = <value optimized out>
   cooldown = 0
   removedSpells = {<std::_List_base<unsigned int, std::allocator<unsigned int> >> = {
   _M_impl = {<std::allocator<std::_List_node<unsigned int> >> = {<__gnu_cxx::new_allocator<std::_List_node<unsigned int> >> = {<No data fields>}, <No data fields>}, _M_node = {_M_next = 0x430e4b40, 
       _M_prev = 0x430e4b40}}}, <No data fields>}
   procTriggered = {<std::_List_base<ProcTriggeredData, std::allocator<ProcTriggeredData> >> = {
   _M_impl = {<std::allocator<std::_List_node<ProcTriggeredData> >> = {<__gnu_cxx::new_allocator<std::_List_node<ProcTriggeredData> >> = {<No data fields>}, <No data fields>}, _M_node = {_M_next = 0x7f59e76870e0, 
       _M_prev = 0x7f59cc710bd0}}}, <No data fields>}
#6  0x000000000070cdc4 in Unit::ProcDamageAndSpell (this=0x7f5a5acbf420, 
   pVictim=0x7f5a2cafba40, procAttacker=<value optimized out>, 
   procVictim=1572864, procExtra=1, amount=568, attType=BASE_ATTACK, 
   procSpell=0x7f5a53c6cd40) at ../../../src/game/Unit.cpp:4411
No locals.
#7  0x0000000000698711 in Aura::PeriodicTick (this=0x7f5a33b3b7f0)
   at ../../../src/game/SpellAuras.cpp:6452
   resist = 0
   amount = <value optimized out>
   pdamage = 0
   isCrit = false
   procVictim = 0
   pCaster = <value optimized out>
   absorb = 0
   cleanDamage = {damage = 0, attackType = BASE_ATTACK, 
 hitOutCome = MELEE_HIT_NORMAL}
   pInfo = {aura = 0x7f5a33b3b7f0, damage = 568, overDamage = 0, 
 absorb = 0, resist = 0, multiplier = 0, critical = false}
#8  0x0000000000699ace in Aura::Update (this=0x7f5a33b3b7f0, diff=152)
   at ../../../src/game/SpellAuras.cpp:650
No locals.
#9  0x00000000006f8cef in Unit::_UpdateSpells (this=0x7f5a2cafba40, time=152)
   at ../../../src/game/SpellAuras.h:311
   i_aura = (class Aura *) 0x7f5a33b3b7f0
#10 0x00000000006fbe46 in Unit::Update (this=0x7f5a2cafba40, p_time=152)
   at ../../../src/game/Unit.cpp:195
No locals.
#11 0x0000000000671311 in Player::Update (this=0x7f5a2cafba40, p_time=152)
   at ../../../src/game/Player.cpp:1060
   now = <value optimized out>
   pet = <value optimized out>
#12 0x00000000005b82d0 in Map::Update (this=0x7f59fd409410, 
   t_diff=@0x7f5a1f9b9a60) at ../../../src/game/Map.cpp:591
   plr = (class Player *) 0x7f59e6dacdb0
   updater = {i_timeDiff = 152}
   __PRETTY_FUNCTION__ = "virtual void Map::Update(const uint32&)"
#13 0x000000000077cba9 in MapInstanced::Update (this=0x7f5a27b73860, 
   t=@0x7f5a1f9b9a60) at ../../../src/game/MapInstanced.cpp:63
   i = {<std::tr1::__detail::_Hashtable_iterator_base<std::Pair<const unsigned int, Map*>, false>> = {_M_cur_node = 0x7f5a1ca33470, 
   _M_cur_bucket = 0x7f5a27b72708}, <No data fields>}
#14 0x0000000000730b82 in MapUpdateRequest::call (this=0x7f5a1f9b9a40)
   at ../../../src/game/MapUpdater.cpp:65
No locals.
#15 0x00000000007bbfe1 in DelayExecutor::svc (this=0x2b9ee88)
   at ../../../src/shared/DelayExecutor.cpp:61
   rq = (class ACE_Method_Request *) 0x7f5a1f9b9a40
#16 0x00007f5a5f233067 in ACE_Task_Base::svc_run (args=<value optimized out>)
   at ../../../../dep/ACE_wrappers/ace/Task.cpp:275
   t = (ACE_Task_Base *) 0x2b9ee88
   svc_status = <value optimized out>
#17 0x00007f5a5f234421 in ACE_Thread_Adapter::invoke (this=0x7f5a58c01e60)
   at ../../../../dep/ACE_wrappers/ace/Thread_Adapter.cpp:98
   exit_hook_instance = <value optimized out>
   exit_hook_maybe = {instance_ = 0x0}
   exit_hook_ptr = <value optimized out>
#18 0x00007f5a5d3503f7 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#19 0x00007f5a5cc2d1dd in clone () from /lib64/libc.so.6
No symbol table info available.

[Auntie Mangos]

Hi a have 1 problem with mtmaps using rev 8573. I currently using derek's mtmaps (ACE), because i'm unable to use this http://getmangos.eu/community/showpost.php?p=90117&postcount=49, if I use that openmp mtmaps, all threads starts in 7X% of cpu ussage and raising...and infinity's (raczman, etc.). I currently have gcc 4.4.0. Any ideas to do with this crash or somebody with my openmp problems?

It is correct way of how openmp works when forking threads that do their job very quickly. By default, threads will not sleep when waiting for new jobs to appear.

It can be controlled by OMP_WAIT_POLICY environment variable http://gcc.gnu.org/onlinedocs/libgomp/OMP_005fWAIT_005fPOLICY.html . However, default (ACTIVE) leads to better performance.

[xeross155]

Hmm, useful, but more useful would be actual fixes for the spells.

[darkstalker]

i had a fix for reflective shield somewhere, need to find it and make git patch

[FragFrog]

Hmm, useful, but more useful would be actual fixes for the spells.

The problem is not just spells. There are a great many possible causes for crashes, some related to the core, others to the DB, and some to simple configuration settings. That said, I much rather know which spells potentially crash the server so I can disable them and have a stable server than something buggy working and uptimes of half an hour max.

I only found out about Reflective Shield a few weeks ago, I think there are quite a few simple changes that could make servers stay up for weeks instead of hours - if only people share them. So please stay ontopic, I'll keep the startpost up to date if there are more suggestions

[FragFrog]

Thanks crashuncle, the assert "fix" mentioned there was already in the list but I've added the database cleanup suggestion. I got the same feeling, there are fixes and patches out there, and while they may break functionality (like removing a spell) I think that's worth it if you can get a more stable server through it.

[Veretos]

Great topic, really needed for people who just started to use mangos.

I was wondering if you have any solution for the following freeze?

[VladimirMangos]

I can pin thread, if author plan have it up-to-date (remove oudated/unneeded options).

[darkstalker]

patch for reflective shield (extractred from my old 3.1.3 svn and converted to git 8734, didnt test)

diff --git a/src/game/SpellAuras.cpp b/src/game/SpellAuras.cpp
index 582946a..f8da694 100644
--- a/src/game/SpellAuras.cpp
+++ b/src/game/SpellAuras.cpp
@@ -6378,7 +6378,7 @@ void Aura::PeriodicTick()
            if (isCrit)
                cleanDamage.hitOutCome = MELEE_HIT_CRIT;

-            pCaster->CalcAbsorbResist(m_target, GetSpellSchoolMask(GetSpellProto()), DOT, pdamage, &absorb, &resist);
+            pCaster->CalcAbsorbResist(m_target, GetSpellSchoolMask(GetSpellProto()), DOT, pdamage, &absorb, &resist, !(GetSpellProto()->AttributesEx2 & SPELL_ATTR_EX2_CANT_REFLECTED));

            sLog.outDetail("PeriodicTick: %u (TypeId: %u) attacked %u (TypeId: %u) for %u dmg inflicted by %u abs is %u",
                GUID_LOPART(GetCasterGUID()), GuidHigh2TypeId(GUID_HIPART(GetCasterGUID())), m_target->GetGUIDLow(), m_target->GetTypeId(), pdamage, GetId(),absorb);
@@ -6433,7 +6433,7 @@ void Aura::PeriodicTick()

            pdamage = pCaster->SpellDamageBonus(m_target, GetSpellProto(), pdamage, DOT, GetStackAmount());

-            pCaster->CalcAbsorbResist(m_target, GetSpellSchoolMask(GetSpellProto()), DOT, pdamage, &absorb, &resist);
+            pCaster->CalcAbsorbResist(m_target, GetSpellSchoolMask(GetSpellProto()), DOT, pdamage, &absorb, &resist, !(GetSpellProto()->AttributesEx2 & SPELL_ATTR_EX2_CANT_REFLECTED));

            if(m_target->GetHealth() < pdamage)
                pdamage = uint32(m_target->GetHealth());
diff --git a/src/game/Unit.cpp b/src/game/Unit.cpp
index a282c96..4badad8 100644
--- a/src/game/Unit.cpp
+++ b/src/game/Unit.cpp
@@ -1116,7 +1116,7 @@ void Unit::CalculateSpellDamage(SpellNonMeleeDamage *damageInfo, int32 damage, S
        }

        uint32 absorb_affected_damage = CalcNotIgnoreAbsorbDamage(damage,damageSchoolMask,spellInfo);
-        CalcAbsorbResist(pVictim, damageSchoolMask, SPELL_DIRECT_DAMAGE, absorb_affected_damage, &damageInfo->absorb, &damageInfo->resist);
+        CalcAbsorbResist(pVictim, damageSchoolMask, SPELL_DIRECT_DAMAGE, absorb_affected_damage, &damageInfo->absorb, &damageInfo->resist, !(spellInfo->AttributesEx2 & SPELL_ATTR_EX2_CANT_REFLECTED));
        damage-= damageInfo->absorb + damageInfo->resist;
    }
    else
@@ -1407,7 +1407,7 @@ void Unit::CalculateMeleeDamage(Unit *pVictim, uint32 damage, CalcDamageInfo *da

        // Calculate absorb & resists
        uint32 absorb_affected_damage = CalcNotIgnoreAbsorbDamage(damageInfo->damage,damageInfo->damageSchoolMask);
-        CalcAbsorbResist(damageInfo->target, damageInfo->damageSchoolMask, DIRECT_DAMAGE, absorb_affected_damage, &damageInfo->absorb, &damageInfo->resist);
+        CalcAbsorbResist(damageInfo->target, damageInfo->damageSchoolMask, DIRECT_DAMAGE, absorb_affected_damage, &damageInfo->absorb, &damageInfo->resist, true);
        damageInfo->damage-=damageInfo->absorb + damageInfo->resist;
        if (damageInfo->absorb)
        {
@@ -1627,7 +1627,7 @@ uint32 Unit::CalcArmorReducedDamage(Unit* pVictim, const uint32 damage)
    return (newdamage > 1) ? newdamage : 1;
}

-void Unit::CalcAbsorbResist(Unit *pVictim,SpellSchoolMask schoolMask, DamageEffectType damagetype, const uint32 damage, uint32 *absorb, uint32 *resist)
+void Unit::CalcAbsorbResist(Unit *pVictim,SpellSchoolMask schoolMask, DamageEffectType damagetype, const uint32 damage, uint32 *absorb, uint32 *resist, bool canReflect)
{
    if(!pVictim || !pVictim->isAlive() || !damage)
        return;
@@ -1911,7 +1911,7 @@ void Unit::CalcAbsorbResist(Unit *pVictim,SpellSchoolMask schoolMask, DamageEffe
    }

    // Cast back reflect damage spell
-    if (reflectSpell)
+    if (canReflect && reflectSpell)
        pVictim->CastCustomSpell(this,  reflectSpell, &reflectDamage, NULL, NULL, true, NULL, reflectTriggeredBy);

    // absorb by mana cost
diff --git a/src/game/Unit.h b/src/game/Unit.h
index 95a0626..b98789b 100644
--- a/src/game/Unit.h
+++ b/src/game/Unit.h
@@ -1461,7 +1461,7 @@ class MANGOS_DLL_SPEC Unit : public WorldObject
                                                            // redefined in Creature

        uint32 CalcArmorReducedDamage(Unit* pVictim, const uint32 damage);
-        void CalcAbsorbResist(Unit *pVictim, SpellSchoolMask schoolMask, DamageEffectType damagetype, const uint32 damage, uint32 *absorb, uint32 *resist);
+        void CalcAbsorbResist(Unit *pVictim, SpellSchoolMask schoolMask, DamageEffectType damagetype, const uint32 damage, uint32 *absorb, uint32 *resist, bool canReflect = false);

        void  UpdateSpeed(UnitMoveType mtype, bool forced);
        float GetSpeed( UnitMoveType mtype ) const;

[FragFrog]

Ooh, very nice Darkstalker, looks like a good fix!

Vladimir: I intend to keep this thread up-to-date as much as possible (but can only test 0.12 fixes), if you want to pin it that would be nice

[Morphau]

Hello, first thanks and gg for this post whose it a good idea

Personally I use 3..2.2 and I confirm what you say FragFrog it's ok

Sorry darkstalker but this spell (http://fr.wowhead.com/?spell=33619) it's ok since your patch

Sorry for my engish but I am french.

[VladimirMangos]

patch for reflective shield (extractred from my old 3.1.3 svn and converted to git 8734, didnt test)

In [8748] and 0.12. Thank you.

To all posters: Please not post patches to this thread! Only links if patches posted some where in another thread. Except maybe short hack patches maybe.

I pin topic and add clarify about hack way fixes.

[FragFrog]

Thanks Vladimir! Updated startpost with fixes in revisions 8753 and 8748.

[Ambal]

Want to share a small patch, which should improve stability of those crazy servers, which use mtmaps: http://filebeam.com/d1505fa74f7190eed4f5e44f3b228dcb . It makes BgQueueEvent***::Execute() methods thread-safe. Patch is for rev 8754.

WARNING: this patch will not be commited to main repo!!!

[cyrex]

Just a quick question, how much effect does 8753 commit have on stability issues.

[FragFrog]

Included it Ambal, thanks!

Cyrex: as far as I know this is still unclear, I've added it since it's better to have a fix than an ugly hack, you might want to try both methods though and let us know whether it works or not.

[leak]

Want to share a small patch, which should improve stability of those crazy servers, which use mtmaps: http://filebeam.com/d1505fa74f7190eed4f5e44f3b228dcb . It makes BgQueueEvent***::Execute() methods thread-safe. Patch is for rev 8754.

WARNING: this patch will not be commited to main repo!!!

http://pastebin.com/f60956892

Ambal's patch backported for 0.12 branch.

[FragFrog]

Added mtmaps patch and hack for rogue poision crash to startpost - thanks everyone

[FragFrog]

Added suggested fix here for Onyxia crashing TBC servers.

[FragFrog]

Added this fix to prevent aura related crashes.

[VladimirMangos]

I think

Remove assert from TypeContainer (ugly hack, but works?) topic (0.14 / 0.15 branch) - Update: possible fix in [8753]

Already long time not reproducable and fixed

[FragFrog]

I have not seen bugreports about it for a long time now, so probably fixed well Updated.

[przemratajczak]

MTMaps stability patch 0.13 / 0.14 & MTMaps stability patch 0.12 (Only for use in combination with mtmaps! Revision 8754)

is it still needed after http://github.com/mangos/mangos/commit/72f7a7ee56c63672faa7786b1d162a7a3b9ab406 ?