SQL connection errors

Alright, without changing anything, EXACTLY fifty minutes after starting the server and not a soul playing, the SQL errors started being logged.
[code]
2015-10-18 23:27:34 SERVER STARTUP TIME: 0 minutes 6 seconds
2015-10-18 23:27:34
2015-10-18 23:27:34
2015-10-18 23:28:02 GameEvent 61 "Stormwind City - Stockades Jail Break" started.
2015-10-18 23:37:08 ERROR:Creature (GUIDLow: 15251) X: -2242.801025 Y: -2199.574707 (original) in grid[27,27]cell[12,13] instead grid[27,27]cell[12,14]
2015-10-18 23:38:01 GameEvent 61 "Stormwind City - Stockades Jail Break" removed.
2015-10-18 23:58:01 GameEvent 61 "Stormwind City - Stockades Jail Break" started.
2015-10-19 00:00:01 GameEvent 16 "Gurubashi Arena Booty Run" started.
2015-10-19 00:00:01 GameEvent 29 "Edge of Madness, Gri'lek" removed.
2015-10-19 00:07:34
2015-10-19 00:07:34 >> Flushed all ranking points
2015-10-19 00:07:34
2015-10-19 00:07:34 >> Loaded 0 Horde and 0 Ally honor standing definitions
2015-10-19 00:08:01 GameEvent 61 "Stormwind City - Stockades Jail Break" removed.
2015-10-19 00:17:20 ERROR:SQL: can not execute 'UPDATE account SET active_realm_id = ? WHERE id = ?'
2015-10-19 00:17:20 ERROR:SQL ERROR: Lost connection to MySQL server during query
2015-10-19 00:24:46 ERROR:SQL: can not execute 'UPDATE account SET last_ip = ? WHERE username = ?'
2015-10-19 00:24:46 ERROR:SQL ERROR: Lost connection to MySQL server during query
2015-10-19 00:24:46 ERROR:SQL: can not execute 'UPDATE realmlist SET population = ? WHERE id = ?'
2015-10-19 00:24:46 ERROR:SQL ERROR: Lost connection to MySQL server during query
2015-10-19 00:25:00 ERROR:SQL: can not execute 'UPDATE account SET active_realm_id = ? WHERE id = ?'
2015-10-19 00:25:00 ERROR:SQL ERROR: Lost connection to MySQL server during query
2015-10-19 00:25:50 ERROR:SQL: can not execute 'UPDATE account SET active_realm_id = ? WHERE id = ?'
2015-10-19 00:25:50 ERROR:SQL ERROR: Lost connection to MySQL server during query
[/code]
I logged in and out a few times after midnight and that is what happened. Something is happening that within roughly one hour, causes the SQL connections to go bonkers. I will continue working on this with this knowledge. I am simply posting it here as a record for myself as well as for others.

After rebooting the server process (mangos) the errors were gone again for about an hour, but they returned. This is entirely on the side of MaNGOS as far as my DB server is concerned. Below is a dump of the server status.
[code]
Variable_name Value
Aborted_clients 206
Aborted_connects 0
Access_denied_errors 0
Aria_pagecache_blocks_not_flushed 0
Aria_pagecache_blocks_unused 15737
Aria_pagecache_blocks_used 2
Aria_pagecache_read_requests 48
Aria_pagecache_reads 4
Aria_pagecache_write_requests 8
Aria_pagecache_writes 0
Aria_transaction_log_syncs 0
Binlog_commits 8260
Binlog_group_commits 8260
Binlog_snapshot_file mariadb-bin.000049
Binlog_snapshot_position 863379
Binlog_bytes_written 1815063
Binlog_cache_disk_use 0
Binlog_cache_use 8254
Binlog_stmt_cache_disk_use 0
Binlog_stmt_cache_use 6
Busy_time 0.000000
Bytes_received 522746
Bytes_sent 43244493
Com_admin_commands 3
Com_alter_db 0
Com_alter_db_upgrade 0
Com_alter_event 0
Com_alter_function 0
Com_alter_procedure 0
Com_alter_server 0
Com_alter_table 0
Com_alter_tablespace 0
Com_analyze 0
Com_assign_to_keycache 0
Com_begin 113
Com_binlog 0
Com_call_procedure 0
Com_change_db 0
Com_change_master 0
Com_check 0
Com_checksum 0
Com_commit 113
Com_create_db 0
Com_create_event 0
Com_create_function 0
Com_create_index 0
Com_create_procedure 0
Com_create_server 0
Com_create_table 0
Com_create_trigger 0
Com_create_udf 0
Com_create_user 0
Com_create_view 0
Com_dealloc_sql 0
Com_delete 8193
Com_delete_multi 6
Com_do 0
Com_drop_db 0
Com_drop_event 0
Com_drop_function 0
Com_drop_index 0
Com_drop_procedure 0
Com_drop_server 0
Com_drop_table 0
Com_drop_trigger 0
Com_drop_user 0
Com_drop_view 0
Com_empty_query 0
Com_execute_sql 0
Com_flush 1
Com_grant 0
Com_ha_close 0
Com_ha_open 0
Com_ha_read 0
Com_help 0
Com_insert 223
Com_insert_select 0
Com_install_plugin 0
Com_kill 0
Com_load 0
Com_lock_tables 0
Com_optimize 0
Com_preload_keys 0
Com_prepare_sql 0
Com_purge 0
Com_purge_before_date 0
Com_release_savepoint 0
Com_rename_table 0
Com_rename_user 0
Com_repair 0
Com_replace 0
Com_replace_select 0
Com_reset 0
Com_resignal 0
Com_revoke 0
Com_revoke_all 0
Com_rollback 0
Com_rollback_to_savepoint 0
Com_savepoint 0
Com_select 1007
Com_set_option 225
Com_show_authors 0
Com_show_binlog_events 0
Com_show_binlogs 0
Com_show_charsets 0
Com_show_client_statistics 0
Com_show_collations 0
Com_show_contributors 0
Com_show_create_db 0
Com_show_create_event 0
Com_show_create_func 0
Com_show_create_proc 0
Com_show_create_table 0
Com_show_create_trigger 0
Com_show_databases 0
Com_show_engine_logs 0
Com_show_engine_mutex 0
Com_show_engine_status 0
Com_show_errors 0
Com_show_events 0
Com_show_fields 0
Com_show_function_status 0
Com_show_grants 0
Com_show_index_statistics 0
Com_show_keys 0
Com_show_master_status 0
Com_show_open_tables 0
Com_show_plugins 0
Com_show_privileges 0
Com_show_procedure_status 0
Com_show_processlist 0
Com_show_profile 0
Com_show_profiles 0
Com_show_relaylog_events 0
Com_show_slave_hosts 0
Com_show_slave_status 0
Com_show_status 2
Com_show_storage_engines 0
Com_show_table_statistics 0
Com_show_table_status 0
Com_show_tables 0
Com_show_triggers 0
Com_show_user_statistics 0
Com_show_variables 1
Com_show_warnings 0
Com_signal 0
Com_slave_start 0
Com_slave_stop 0
Com_stmt_close 0
Com_stmt_execute 442
Com_stmt_fetch 0
Com_stmt_prepare 28
Com_stmt_reprepare 0
Com_stmt_reset 0
Com_stmt_send_long_data 0
Com_truncate 1
Com_uninstall_plugin 0
Com_unlock_tables 0
Com_update 174
Com_update_multi 0
Com_xa_commit 0
Com_xa_end 0
Com_xa_prepare 0
Com_xa_recover 0
Com_xa_rollback 0
Com_xa_start 0
Compression OFF
Connections 372
Cpu_time 0.000000
Created_tmp_disk_tables 14
Created_tmp_files 6
Created_tmp_tables 85
Delayed_errors 0
Delayed_insert_threads 0
Delayed_writes 0
Empty_queries 236
Executed_events 0
Executed_triggers 0
Feature_dynamic_columns 0
Feature_fulltext 0
Feature_gis 0
Feature_locale 0
Feature_subquery 4
Feature_timezone 0
Feature_trigger 0
Feature_xml 0
Flush_commands 2
Handler_commit 17512
Handler_delete 8173
Handler_discover 0
Handler_icp_attempts 0
Handler_icp_match 0
Handler_mrr_init 0
Handler_mrr_key_refills 0
Handler_mrr_rowid_refills 0
Handler_prepare 17136
Handler_read_first 36
Handler_read_key 356544
Handler_read_last 34
Handler_read_next 353047
Handler_read_prev 0
Handler_read_rnd 3289
Handler_read_rnd_deleted 3
Handler_read_rnd_next 739999
Handler_rollback 0
Handler_savepoint 0
Handler_savepoint_rollback 0
Handler_tmp_update 0
Handler_tmp_write 281594
Handler_update 181
Handler_write 223
Innodb_adaptive_hash_cells 553229
Innodb_adaptive_hash_hash_searches 35483
Innodb_adaptive_hash_heap_buffers 57
Innodb_adaptive_hash_non_hash_searches 15931
Innodb_background_log_sync 51369
Innodb_buffer_pool_bytes_data 15892480
Innodb_buffer_pool_bytes_dirty 65536
Innodb_buffer_pool_pages_data 970
Innodb_buffer_pool_pages_dirty 4
Innodb_buffer_pool_pages_flushed 26080
Innodb_buffer_pool_pages_free 15355
Innodb_buffer_pool_pages_LRU_flushed 0
Innodb_buffer_pool_pages_made_not_young 0
Innodb_buffer_pool_pages_made_young 0
Innodb_buffer_pool_pages_misc 58
Innodb_buffer_pool_pages_old 376
Innodb_buffer_pool_pages_total 16383
Innodb_buffer_pool_read_ahead 64
Innodb_buffer_pool_read_ahead_evicted 0
Innodb_buffer_pool_read_ahead_rnd 0
Innodb_buffer_pool_read_requests 3078016
Innodb_buffer_pool_reads 896
Innodb_buffer_pool_wait_free 0
Innodb_buffer_pool_write_requests 78688
Innodb_checkpoint_age 112
Innodb_checkpoint_max_age 7782360
Innodb_checkpoint_target_age 7539162
Innodb_current_row_locks 0
Innodb_data_fsyncs 22222
Innodb_data_pending_fsyncs 0
Innodb_data_pending_reads 0
Innodb_data_pending_writes 0
Innodb_data_read 17911808
Innodb_data_reads 1014
Innodb_data_writes 45450
Innodb_data_written 867927040
Innodb_dblwr_pages_written 26080
Innodb_dblwr_writes 954
Innodb_deadlocks 0
Innodb_descriptors_memory 8000
Innodb_dict_tables 52
Innodb_have_atomic_builtins ON
Innodb_history_list_length 3535
Innodb_ibuf_discarded_delete_marks 0
Innodb_ibuf_discarded_deletes 0
Innodb_ibuf_discarded_inserts 0
Innodb_ibuf_free_list 0
Innodb_ibuf_merged_delete_marks 0
Innodb_ibuf_merged_deletes 0
Innodb_ibuf_merged_inserts 0
Innodb_ibuf_merges 0
Innodb_ibuf_segment_size 2
Innodb_ibuf_size 1
Innodb_log_waits 0
Innodb_log_write_requests 7807
Innodb_log_writes 17442
Innodb_lsn_current 22162895
Innodb_lsn_flushed 22162895
Innodb_lsn_last_checkpoint 22162783
Innodb_master_thread_1_second_loops 10445
Innodb_master_thread_10_second_loops 950
Innodb_master_thread_background_loops 947
Innodb_master_thread_main_flush_loops 947
Innodb_master_thread_sleeps 10445
Innodb_max_trx_id 50793
Innodb_mem_adaptive_hash 5380320
Innodb_mem_dictionary 1398114
Innodb_mem_total 275513344
Innodb_mutex_os_waits 20
Innodb_mutex_spin_rounds 634
Innodb_mutex_spin_waits 28
Innodb_oldest_view_low_limit_trx_id 50793
Innodb_os_log_fsyncs 18394
Innodb_os_log_pending_fsyncs 0
Innodb_os_log_pending_writes 0
Innodb_os_log_written 12849664
Innodb_page_size 16384
Innodb_pages_created 14
Innodb_pages_read 959
Innodb_pages_written 26080
Innodb_purge_trx_id 50793
Innodb_purge_undo_no 0
Innodb_read_views_memory 192
Innodb_row_lock_current_waits 0
Innodb_row_lock_time 0
Innodb_row_lock_time_avg 0
Innodb_row_lock_time_max 0
Innodb_row_lock_waits 0
Innodb_rows_deleted 8173
Innodb_rows_inserted 223
Innodb_rows_read 93950
Innodb_rows_updated 176
Innodb_s_lock_os_waits 954
Innodb_s_lock_spin_rounds 28620
Innodb_s_lock_spin_waits 954
Innodb_truncated_status_writes 0
Innodb_x_lock_os_waits 0
Innodb_x_lock_spin_rounds 0
Innodb_x_lock_spin_waits 0
Key_blocks_not_flushed 0
Key_blocks_unused 100991
Key_blocks_used 6180
Key_blocks_warm 15
Key_read_requests 748786
Key_reads 6180
Key_write_requests 0
Key_writes 0
Last_query_cost 0.000000
Max_used_connections 9
Not_flushed_delayed_rows 0
Open_files 318
Open_streams 0
Open_table_definitions 257
Open_tables 220
Opened_files 673
Opened_table_definitions 225
Opened_tables 195
Opened_views 0
Performance_schema_cond_classes_lost 0
Performance_schema_cond_instances_lost 0
Performance_schema_file_classes_lost 0
Performance_schema_file_handles_lost 0
Performance_schema_file_instances_lost 0
Performance_schema_locker_lost 0
Performance_schema_mutex_classes_lost 0
Performance_schema_mutex_instances_lost 0
Performance_schema_rwlock_classes_lost 0
Performance_schema_rwlock_instances_lost 0
Performance_schema_table_handles_lost 0
Performance_schema_table_instances_lost 0
Performance_schema_thread_classes_lost 0
Performance_schema_thread_instances_lost 0
Prepared_stmt_count 25
Qcache_free_blocks 16
Qcache_free_memory 64772840
Qcache_hits 114
Qcache_inserts 265
Qcache_lowmem_prunes 0
Qcache_not_cached 652
Qcache_queries_in_cache 218
Qcache_total_blocks 606
Queries 10250
Questions 10221
Rows_read 1141645
Rows_sent 730868
Rows_tmp_read 16359
Rpl_status AUTH_MASTER
Select_full_join 4
Select_full_range_join 0
Select_range 2
Select_range_check 0
Select_scan 187
Slave_heartbeat_period 0.000
Slave_open_temp_tables 0
Slave_received_heartbeats 0
Slave_retried_transactions 0
Slave_running OFF
Slow_launch_threads 0
Slow_queries 0
Sort_merge_passes 0
Sort_range 3
Sort_rows 5873
Sort_scan 17
Sphinx_error
Sphinx_time
Sphinx_total
Sphinx_total_found
Sphinx_word_count
Sphinx_words
Ssl_accept_renegotiates 0
Ssl_accepts 0
Ssl_callback_cache_hits 0
Ssl_cipher
Ssl_cipher_list
Ssl_client_connects 0
Ssl_connect_renegotiates 0
Ssl_ctx_verify_depth 0
Ssl_ctx_verify_mode 0
Ssl_default_timeout 0
Ssl_finished_accepts 0
Ssl_finished_connects 0
Ssl_session_cache_hits 0
Ssl_session_cache_misses 0
Ssl_session_cache_mode NONE
Ssl_session_cache_overflows 0
Ssl_session_cache_size 0
Ssl_session_cache_timeouts 0
Ssl_sessions_reused 0
Ssl_used_session_cache_entries 0
Ssl_verify_depth 0
Ssl_verify_mode 0
Ssl_version
Subquery_cache_hit 0
Subquery_cache_miss 0
Syncs 11
Table_locks_immediate 9152
Table_locks_waited 0
Tc_log_max_pages_used 0
Tc_log_page_size 0
Tc_log_page_waits 0
Threadpool_idle_threads 0
Threadpool_threads 0
Threads_cached 6
Threads_connected 3
Threads_created 9
Threads_running 1
Uptime 51743
Uptime_since_flush_status 51743
[/code]
Take note of the following values.
[code]
Aborted_connects 0
Max_used_connections 9
Threads_connected 3
[/code]
The aborted connections are connections which are lost, interrupted, etc. In other words, the server has not dropped any connections, yet MaNGOS is stating that the connection was dropped. It also states that the maximum simultaneous connections was only 9, but if I set the max connections to 100, the AHBot stops working. Something is very fishy here. Oh, and currently only three connected clients. One is the MySQL client I am using. That leaves one for MaNGOS and one for the realm daemon. So this just seems to get stranger and stranger the deeper I dig.

Can anybody tell me if the AHBot and everything else uses a single connection from the server process to do all queries, or are there separate connections for various things?

*UPDATE*

I just realized that there are over two-hundred aborted_client connections. Per the MySQL manual, these are connections which the client closed prematurely or incorrectly.
[quote]
Aborted_clients

The number of connections that were aborted because the client died without closing the connection properly. See Section B.5.2.11, “Communication Errors and Aborted Connections”.
[/quote]
So the server is aborting connections, but why is the question. Below is what I am using to troubleshoot this now.

[url]https://dev.mysql.com/doc/refman/5.0/en/communication-errors.html[/url]

*UPDATE*

I am focusing on these three items for the time being, based on the fact that there is no packet-loss between the VMs.
[quote]
The client program did not call mysql_close() before exiting.

The client had been sleeping more than wait_timeout or interactive_timeout seconds without issuing any requests to the server. See Section 5.1.4, “Server System Variables”.

The client program ended abruptly in the middle of a data transfer.
[/quote]
I do not believe "mysql_close()" is the issue due to the server not reading or writing data. I also do not feel that the last point is the issue, as the server remains up. If the server creates a connection and never closes it, this could EASILY be our issue. The a few hours in, a player logs in and the server cannot speak to the DB due to the connection exceeding the timeout. Now I need to figure out if the server creates a connection each time it needs data, or if we open it and only close it on shutdown. If we do keep it open, we need to do something like a "keepalive" packet or something to prevent the server from closing it while nobody is on.

After getting home I checked the database server logs. It shows the aborted connections but offers no more clues.
[code]
Oct 19 06:40:32 db01 mysqld: 151019 6:40:32 [Warning] Aborted connection 264 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 06:40:32 db01 mysqld: 151019 6:40:32 [Warning] Aborted connection 265 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 06:41:11 db01 mysqld: 151019 6:41:11 [Warning] Aborted connection 267 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 06:41:11 db01 mysqld: 151019 6:41:11 [Warning] Aborted connection 266 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 06:41:11 db01 mysqld: 151019 6:41:11 [Warning] Aborted connection 268 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 06:41:15 db01 mysqld: 151019 6:41:15 [Warning] Aborted connection 269 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 06:57:34 db01 mysqld: 151019 6:57:34 [Warning] Aborted connection 261 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 07:10:48 db01 mysqld: 151019 7:10:48 [Warning] Aborted connection 271 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 07:10:48 db01 mysqld: 151019 7:10:48 [Warning] Aborted connection 272 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 07:11:27 db01 mysqld: 151019 7:11:27 [Warning] Aborted connection 274 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 07:11:27 db01 mysqld: 151019 7:11:27 [Warning] Aborted connection 273 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 07:11:27 db01 mysqld: 151019 7:11:27 [Warning] Aborted connection 275 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 07:11:31 db01 mysqld: 151019 7:11:31 [Warning] Aborted connection 276 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 07:17:34 db01 mysqld: 151019 7:17:34 [Warning] Aborted connection 270 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 07:41:04 db01 mysqld: 151019 7:41:04 [Warning] Aborted connection 278 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 07:41:04 db01 mysqld: 151019 7:41:04 [Warning] Aborted connection 279 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 07:41:43 db01 mysqld: 151019 7:41:43 [Warning] Aborted connection 281 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 07:41:43 db01 mysqld: 151019 7:41:43 [Warning] Aborted connection 280 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 07:41:43 db01 mysqld: 151019 7:41:43 [Warning] Aborted connection 282 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 07:41:47 db01 mysqld: 151019 7:41:47 [Warning] Aborted connection 283 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 07:47:34 db01 mysqld: 151019 7:47:34 [Warning] Aborted connection 277 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:11:19 db01 mysqld: 151019 8:11:19 [Warning] Aborted connection 285 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:11:19 db01 mysqld: 151019 8:11:19 [Warning] Aborted connection 286 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:11:58 db01 mysqld: 151019 8:11:58 [Warning] Aborted connection 287 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:11:59 db01 mysqld: 151019 8:11:59 [Warning] Aborted connection 288 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:11:59 db01 mysqld: 151019 8:11:59 [Warning] Aborted connection 289 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:12:03 db01 mysqld: 151019 8:12:03 [Warning] Aborted connection 290 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:17:34 db01 mysqld: 151019 8:17:34 [Warning] Aborted connection 284 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:27:34 db01 mysqld: 151019 8:27:34 [Warning] Aborted connection 291 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:41:35 db01 mysqld: 151019 8:41:35 [Warning] Aborted connection 293 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:41:35 db01 mysqld: 151019 8:41:35 [Warning] Aborted connection 294 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:42:14 db01 mysqld: 151019 8:42:14 [Warning] Aborted connection 295 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:42:15 db01 mysqld: 151019 8:42:15 [Warning] Aborted connection 296 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:42:15 db01 mysqld: 151019 8:42:15 [Warning] Aborted connection 297 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:42:20 db01 mysqld: 151019 8:42:20 [Warning] Aborted connection 298 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:47:34 db01 mysqld: 151019 8:47:34 [Warning] Aborted connection 292 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 08:57:34 db01 mysqld: 151019 8:57:34 [Warning] Aborted connection 299 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 09:11:51 db01 mysqld: 151019 9:11:51 [Warning] Aborted connection 301 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 09:11:51 db01 mysqld: 151019 9:11:51 [Warning] Aborted connection 302 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 09:12:30 db01 mysqld: 151019 9:12:30 [Warning] Aborted connection 303 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 09:12:31 db01 mysqld: 151019 9:12:31 [Warning] Aborted connection 304 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 09:12:31 db01 mysqld: 151019 9:12:31 [Warning] Aborted connection 305 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 09:12:36 db01 mysqld: 151019 9:12:36 [Warning] Aborted connection 306 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 09:27:34 db01 mysqld: 151019 9:27:34 [Warning] Aborted connection 300 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 09:42:07 db01 mysqld: 151019 9:42:07 [Warning] Aborted connection 308 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 09:42:07 db01 mysqld: 151019 9:42:07 [Warning] Aborted connection 309 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 09:42:46 db01 mysqld: 151019 9:42:46 [Warning] Aborted connection 310 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 09:42:46 db01 mysqld: 151019 9:42:46 [Warning] Aborted connection 312 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 09:42:46 db01 mysqld: 151019 9:42:46 [Warning] Aborted connection 311 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 09:42:52 db01 mysqld: 151019 9:42:52 [Warning] Aborted connection 313 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 09:47:34 db01 mysqld: 151019 9:47:34 [Warning] Aborted connection 307 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 10:12:23 db01 mysqld: 151019 10:12:23 [Warning] Aborted connection 315 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 10:12:23 db01 mysqld: 151019 10:12:23 [Warning] Aborted connection 316 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 10:13:02 db01 mysqld: 151019 10:13:02 [Warning] Aborted connection 317 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 10:13:02 db01 mysqld: 151019 10:13:02 [Warning] Aborted connection 319 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 10:13:02 db01 mysqld: 151019 10:13:02 [Warning] Aborted connection 318 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 10:13:08 db01 mysqld: 151019 10:13:08 [Warning] Aborted connection 320 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 10:27:34 db01 mysqld: 151019 10:27:34 [Warning] Aborted connection 314 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 10:42:39 db01 mysqld: 151019 10:42:39 [Warning] Aborted connection 322 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 10:42:39 db01 mysqld: 151019 10:42:39 [Warning] Aborted connection 323 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 10:43:18 db01 mysqld: 151019 10:43:18 [Warning] Aborted connection 324 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 10:43:18 db01 mysqld: 151019 10:43:18 [Warning] Aborted connection 326 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 10:43:18 db01 mysqld: 151019 10:43:18 [Warning] Aborted connection 325 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 10:43:24 db01 mysqld: 151019 10:43:24 [Warning] Aborted connection 327 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 10:47:34 db01 mysqld: 151019 10:47:34 [Warning] Aborted connection 321 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 11:12:55 db01 mysqld: 151019 11:12:55 [Warning] Aborted connection 329 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 11:12:55 db01 mysqld: 151019 11:12:55 [Warning] Aborted connection 330 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 11:13:34 db01 mysqld: 151019 11:13:34 [Warning] Aborted connection 331 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 11:13:34 db01 mysqld: 151019 11:13:34 [Warning] Aborted connection 334 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 11:13:34 db01 mysqld: 151019 11:13:34 [Warning] Aborted connection 333 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 11:13:40 db01 mysqld: 151019 11:13:40 [Warning] Aborted connection 332 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 11:27:34 db01 mysqld: 151019 11:27:34 [Warning] Aborted connection 328 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 11:43:10 db01 mysqld: 151019 11:43:10 [Warning] Aborted connection 336 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 11:43:10 db01 mysqld: 151019 11:43:10 [Warning] Aborted connection 337 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 11:43:50 db01 mysqld: 151019 11:43:50 [Warning] Aborted connection 338 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 11:43:50 db01 mysqld: 151019 11:43:50 [Warning] Aborted connection 340 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 11:43:50 db01 mysqld: 151019 11:43:50 [Warning] Aborted connection 339 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 11:43:57 db01 mysqld: 151019 11:43:57 [Warning] Aborted connection 341 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 11:47:34 db01 mysqld: 151019 11:47:34 [Warning] Aborted connection 335 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 12:13:26 db01 mysqld: 151019 12:13:26 [Warning] Aborted connection 343 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 12:13:26 db01 mysqld: 151019 12:13:26 [Warning] Aborted connection 344 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 12:14:06 db01 mysqld: 151019 12:14:06 [Warning] Aborted connection 345 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 12:14:06 db01 mysqld: 151019 12:14:06 [Warning] Aborted connection 346 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 12:14:06 db01 mysqld: 151019 12:14:06 [Warning] Aborted connection 347 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 12:14:13 db01 mysqld: 151019 12:14:13 [Warning] Aborted connection 348 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 12:17:34 db01 mysqld: 151019 12:17:34 [Warning] Aborted connection 342 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 12:43:42 db01 mysqld: 151019 12:43:42 [Warning] Aborted connection 350 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 12:43:42 db01 mysqld: 151019 12:43:42 [Warning] Aborted connection 351 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 12:44:22 db01 mysqld: 151019 12:44:22 [Warning] Aborted connection 352 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 12:44:22 db01 mysqld: 151019 12:44:22 [Warning] Aborted connection 354 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 12:44:22 db01 mysqld: 151019 12:44:22 [Warning] Aborted connection 353 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 12:44:29 db01 mysqld: 151019 12:44:29 [Warning] Aborted connection 355 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 12:47:34 db01 mysqld: 151019 12:47:34 [Warning] Aborted connection 349 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 13:13:58 db01 mysqld: 151019 13:13:58 [Warning] Aborted connection 357 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 13:13:58 db01 mysqld: 151019 13:13:58 [Warning] Aborted connection 358 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 13:14:38 db01 mysqld: 151019 13:14:38 [Warning] Aborted connection 359 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 13:14:38 db01 mysqld: 151019 13:14:38 [Warning] Aborted connection 361 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 13:14:38 db01 mysqld: 151019 13:14:38 [Warning] Aborted connection 360 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 13:14:45 db01 mysqld: 151019 13:14:45 [Warning] Aborted connection 362 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 13:17:34 db01 mysqld: 151019 13:17:34 [Warning] Aborted connection 356 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 13:44:14 db01 mysqld: 151019 13:44:14 [Warning] Aborted connection 364 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 13:44:14 db01 mysqld: 151019 13:44:14 [Warning] Aborted connection 365 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 13:44:54 db01 mysqld: 151019 13:44:54 [Warning] Aborted connection 366 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 13:44:54 db01 mysqld: 151019 13:44:54 [Warning] Aborted connection 367 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 13:44:54 db01 mysqld: 151019 13:44:54 [Warning] Aborted connection 368 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 13:45:01 db01 mysqld: 151019 13:45:01 [Warning] Aborted connection 369 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 13:47:34 db01 mysqld: 151019 13:47:34 [Warning] Aborted connection 363 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:14:30 db01 mysqld: 151019 14:14:30 [Warning] Aborted connection 374 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:14:30 db01 mysqld: 151019 14:14:30 [Warning] Aborted connection 375 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:15:10 db01 mysqld: 151019 14:15:10 [Warning] Aborted connection 376 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:15:10 db01 mysqld: 151019 14:15:10 [Warning] Aborted connection 377 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:15:10 db01 mysqld: 151019 14:15:10 [Warning] Aborted connection 378 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:15:18 db01 mysqld: 151019 14:15:18 [Warning] Aborted connection 379 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:17:34 db01 mysqld: 151019 14:17:34 [Warning] Aborted connection 372 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:27:34 db01 mysqld: 151019 14:27:34 [Warning] Aborted connection 380 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:44:46 db01 mysqld: 151019 14:44:46 [Warning] Aborted connection 382 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:44:46 db01 mysqld: 151019 14:44:46 [Warning] Aborted connection 383 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:45:26 db01 mysqld: 151019 14:45:26 [Warning] Aborted connection 384 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:45:26 db01 mysqld: 151019 14:45:26 [Warning] Aborted connection 385 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:45:26 db01 mysqld: 151019 14:45:26 [Warning] Aborted connection 386 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:45:34 db01 mysqld: 151019 14:45:34 [Warning] Aborted connection 387 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:47:34 db01 mysqld: 151019 14:47:34 [Warning] Aborted connection 381 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 14:57:34 db01 mysqld: 151019 14:57:34 [Warning] Aborted connection 388 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:15:01 db01 mysqld: 151019 15:15:01 [Warning] Aborted connection 389 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:15:01 db01 mysqld: 151019 15:15:01 [Warning] Aborted connection 390 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:15:42 db01 mysqld: 151019 15:15:42 [Warning] Aborted connection 392 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:15:42 db01 mysqld: 151019 15:15:42 [Warning] Aborted connection 394 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:15:42 db01 mysqld: 151019 15:15:42 [Warning] Aborted connection 393 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:15:50 db01 mysqld: 151019 15:15:50 [Warning] Aborted connection 395 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:27:34 db01 mysqld: 151019 15:27:34 [Warning] Aborted connection 391 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:45:17 db01 mysqld: 151019 15:45:17 [Warning] Aborted connection 397 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:45:17 db01 mysqld: 151019 15:45:17 [Warning] Aborted connection 398 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:45:58 db01 mysqld: 151019 15:45:58 [Warning] Aborted connection 399 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:45:58 db01 mysqld: 151019 15:45:58 [Warning] Aborted connection 400 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:45:58 db01 mysqld: 151019 15:45:58 [Warning] Aborted connection 401 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:46:06 db01 mysqld: 151019 15:46:06 [Warning] Aborted connection 402 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:47:34 db01 mysqld: 151019 15:47:34 [Warning] Aborted connection 396 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 15:57:34 db01 mysqld: 151019 15:57:34 [Warning] Aborted connection 403 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 16:15:33 db01 mysqld: 151019 16:15:33 [Warning] Aborted connection 405 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 16:15:33 db01 mysqld: 151019 16:15:33 [Warning] Aborted connection 406 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 16:16:14 db01 mysqld: 151019 16:16:14 [Warning] Aborted connection 407 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 16:16:14 db01 mysqld: 151019 16:16:14 [Warning] Aborted connection 409 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 16:16:14 db01 mysqld: 151019 16:16:14 [Warning] Aborted connection 408 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 16:16:22 db01 mysqld: 151019 16:16:22 [Warning] Aborted connection 410 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 16:27:34 db01 mysqld: 151019 16:27:34 [Warning] Aborted connection 404 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 16:45:49 db01 mysqld: 151019 16:45:49 [Warning] Aborted connection 411 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 16:45:49 db01 mysqld: 151019 16:45:49 [Warning] Aborted connection 412 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 16:46:30 db01 mysqld: 151019 16:46:30 [Warning] Aborted connection 414 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 16:46:30 db01 mysqld: 151019 16:46:30 [Warning] Aborted connection 416 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 16:46:30 db01 mysqld: 151019 16:46:30 [Warning] Aborted connection 415 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 16:46:39 db01 mysqld: 151019 16:46:39 [Warning] Aborted connection 417 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 16:57:34 db01 mysqld: 151019 16:57:34 [Warning] Aborted connection 413 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 17:16:05 db01 mysqld: 151019 17:16:05 [Warning] Aborted connection 419 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 17:16:05 db01 mysqld: 151019 17:16:05 [Warning] Aborted connection 420 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 17:16:46 db01 mysqld: 151019 17:16:46 [Warning] Aborted connection 421 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 17:16:46 db01 mysqld: 151019 17:16:46 [Warning] Aborted connection 423 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 17:16:46 db01 mysqld: 151019 17:16:46 [Warning] Aborted connection 422 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 17:16:55 db01 mysqld: 151019 17:16:55 [Warning] Aborted connection 424 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 17:27:34 db01 mysqld: 151019 17:27:34 [Warning] Aborted connection 418 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 17:46:21 db01 mysqld: 151019 17:46:21 [Warning] Aborted connection 426 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 17:46:21 db01 mysqld: 151019 17:46:21 [Warning] Aborted connection 427 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 17:47:02 db01 mysqld: 151019 17:47:02 [Warning] Aborted connection 428 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 17:47:02 db01 mysqld: 151019 17:47:02 [Warning] Aborted connection 430 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 17:47:02 db01 mysqld: 151019 17:47:02 [Warning] Aborted connection 429 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 17:47:11 db01 mysqld: 151019 17:47:11 [Warning] Aborted connection 431 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:16:37 db01 mysqld: 151019 18:16:37 [Warning] Aborted connection 432 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:16:37 db01 mysqld: 151019 18:16:37 [Warning] Aborted connection 433 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:17:18 db01 mysqld: 151019 18:17:18 [Warning] Aborted connection 434 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:17:18 db01 mysqld: 151019 18:17:18 [Warning] Aborted connection 435 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:17:18 db01 mysqld: 151019 18:17:18 [Warning] Aborted connection 436 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:17:27 db01 mysqld: 151019 18:17:27 [Warning] Aborted connection 437 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:17:34 db01 mysqld: 151019 18:17:34 [Warning] Aborted connection 425 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:27:34 db01 mysqld: 151019 18:27:34 [Warning] Aborted connection 438 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:46:53 db01 mysqld: 151019 18:46:53 [Warning] Aborted connection 440 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:46:53 db01 mysqld: 151019 18:46:53 [Warning] Aborted connection 441 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:47:34 db01 mysqld: 151019 18:47:34 [Warning] Aborted connection 442 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:47:34 db01 mysqld: 151019 18:47:34 [Warning] Aborted connection 444 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:47:34 db01 mysqld: 151019 18:47:34 [Warning] Aborted connection 443 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:47:34 db01 mysqld: 151019 18:47:34 [Warning] Aborted connection 439 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 18:47:44 db01 mysqld: 151019 18:47:44 [Warning] Aborted connection 445 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 19:07:34 db01 mysqld: 151019 19:07:34 [Warning] Aborted connection 446 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 19:17:09 db01 mysqld: 151019 19:17:09 [Warning] Aborted connection 452 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 19:17:09 db01 mysqld: 151019 19:17:09 [Warning] Aborted connection 453 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 19:17:50 db01 mysqld: 151019 19:17:50 [Warning] Aborted connection 455 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 19:17:50 db01 mysqld: 151019 19:17:50 [Warning] Aborted connection 457 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 19:17:50 db01 mysqld: 151019 19:17:50 [Warning] Aborted connection 456 to db: 'zero_world' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 19:18:00 db01 mysqld: 151019 19:18:00 [Warning] Aborted connection 458 to db: 'zero_toons' user: '' host: 'x.y.z.211' (Unknown error)
Oct 19 19:27:34 db01 mysqld: 151019 19:27:34 [Warning] Aborted connection 454 to db: 'realms' user: '' host: 'x.y.z.211' (Unknown error)
[/code]
I still have no lost packets, no packet collisions, or anything else, so I am assuming that what the server sends out, the DB engine gets. I have one last idea up my sleeve before I will have to call in some MySQL guys to help with this. I can't stand when a project as large as MySQL cannot tell me what's going on and logs an unknown error, but all major products do this. Either way, I need more than "unknown" on both ends to figure it out.

I changed the max connections to 128 (default was 100) from the 1024 I had set. I then changed 'wait_timeout' on the DB server to 1200 (default 600). After that I rebooted both VMs and restarted Zero. The results are that MaNGOS has not recorded a single error all night, SQL or otherwise. Normally I get errors even though I am not playing during the night and I am obviously sleeping. This morning, no errors at all.

However, the database server has logged over one-hundred aborted connections. This greatly narrows down the cause of the errors. The default timeout of 600 is ten minutes. That means any noninteractive connections that have had no activity are closed after ten minutes by default. Doubling this to twenty minutes removed the errors being logged on MaNGOS, but we still get aborted connections, possibly due to lack of activity. This tells me that the issue has something to do with MaNGOS creating a connection to the server and then not using it and closing it. Then, fifteen minutes later it tries using the connection but it has been closed and data is not stored or read.

Now before I go doing to much digging and changing, why are we keeping connections open for over ten minutes without using them? I can probably go through the code and force a shorter timeout, but will it affect the server? Will it cause the server just to reopen an inactive connection if it closes after say, three minutes?

I appear to have found the solution. I changed my timeout to 15min (wait_timeout = 900) and while I still get some aborted clients, I have no SQL errors in my log and the AHBot works. In fact, multiple issues that I had have disappeared since upping my timeout from 10min to 15min.

Now, why has this been such a pain since I loaded a clean box and did a fresh install of everything? In version 5.5 of MySQL and MariaDB the default timeout was lowered from [b]EIGHT HOURS[/b] to ten minutes. That's right, the old default was way out of bounds (28,800 if I recall) and having a lot of idle, noninteractive connections chewed through resources. It is also best practice for a noninteractive connection to connect, do its work, and close. So this new limit of ten minutes, which works fine with SQL clients which adhere to the standards, is causing issues with older software or software that simply lays around with a connection open but doesn't send or receive any data before the timeout expires. Then the program (client) attempts to do something, but the server is no longer listening. I have now been up for almost 24hrs without a single SQL error on the server.

So where do we go from here? It is my personal opinion and suggestion that we figure out what code is spawning connections it isn't using and fix that. That or modify the SQL connection code to connect as needed except for critical things which may be used constantly. Even then, a connection on a 100Mbps network takes a few micro-seconds, so it isn't a big deal using them on the fly. What do the rest of you think?

All the world process uses a DB connection. I can also tell you that SQL queries are sometimes queued and flushed when the server has time.

It's a common practice to keep a pool of connections open, this is because establishing a connection is CPU-consuming. Doing what you suggest is a bad idea as-is. On the other side, identifying the component that doesn't re-use the connection it has open to see whether it doesn't lack a proper pooling mechanism makes more sense.

By the way, if you run the world, connections will be made but if nobody is playing on it, the problem does not occur nether because you never reach the time-out.

Another idea is to handle properly a DB-side timeout.

Actually, creating connections on the fly uses very little CPU. I have done multiple sites using PHP5 and they all create connections as needed. CPU usage is relatively small even during peak hours on these sites, which can see tens of thousands of unique hits a day, excluding the bots (Google, Yahoo, etc). It is also taught in most SQL/DB classes that connections should not remain open. However, this is an MMO and I can easily see the reason for leaving a main connection open for loading world data as players roam between zones and go into instances. Heck, if it was a live, public server with a thousand or more players it would be crazy to keep opening connections.

As for the not seeing the errors because nobody is on, that is not accurate. The way it was working was that after roughly one hour of uptime, I would start getting SQL errors even though nobody was on the server. The server only has two accounts on it (my fiancée and my self) and both of us were asleep, yet SQL errors popped up during the night. After upping the timeout from 10 to 15 minutes, this no longer happens. I am guessing that the other SQL errors during the night were events starting and stopping. I never dug into what was accessing the DB, just that the errors were popping up.

As it stands, the server has been up for over 24hrs and here are the only errors in the log.
[code]
~$ cat ./zero/logs/zero-world_2015-10-19_19-50-08.log | grep ERROR
2015-10-20 09:49:46 ERROR:Creature (GUIDLow: 26705) X: -1542.377808 Y: -367.140808 (original) in grid[29,31]cell[1,5] instead grid[29,31]cell[1,4]
2015-10-20 09:50:00 ERROR:Creature (GUIDLow: 26076) X: -1527.861572 Y: -467.248657 (original) in grid[29,31]cell[2,2] instead grid[29,31]cell[2,1]
2015-10-20 10:14:51 ERROR CreatureEventAI: Event 339703 - NULL target for ACTION_T_SUMMON_UNIQUE(49), target-type 1
2015-10-20 18:59:56 ERROR CreatureEventAI: NULL target for ACTION_T_CAST creature entry 3269 casting spell id 20793
[/code]
As you can see, no SQL errors after over 24hrs, and the only change was the timeout.

I agree with your suggestion of identifying what is causing these connections to open and time-out. I also like the idea of handling a server-side closing of the connection and then re-establishing said connection or leaving it closed based on need. I do not believe we need a large pool of connections idling open to the server. Maybe we could implement something where we have one for the server plus one per character playing or something. I believe finding the issue is more important though, and the rest could be done down the road.

Now, as new people come to MaNGOS and start trying to build and tinker with their own servers they may hit this issue. I suggest that we explain in the directions that most major database servers have changed the timeout from 8hrs to 10min and they need to change it to at least 15min (wait_timeout = 900 in my.cnf on MySQL/MariaDB) to avoid connections timing out. It could also prevent erroneous bug reports due to connections timing out.

As for me, I will see if I can figure out why some connections are being made and either being queued for over ten minutes or why they're made and not used. I do believe a ten minute queue is a bit excessive, but this is my first time looking at how connections are handled. I've tinkered with the code which loads and saves toons while working on DB structure, but not the connection code.

Ok, you are true. Connection pooling is something businesses don't do.

And it's back. Setting the time-out to 900 made it work for almost three days. Now I am getting crud like this in my output again.
[code]
WORLD: Received opcode CMSG_LOGOUT_REQUEST, security 0
The value of player Tyree at save:
HP is: 422 MP is: 1500
AGILITY is: 26.000000 STRENGTH is: 24.00000
0
INTELLECT is: 94.000000 SPIRIT is: 100.0000
00
STAMINA is: 41.000000
Armor is: 531 Block is: 0.000000
HolyRes is: 1 FireRes is: 1
NatureRes is: 1 FrostRes is: 1
ShadowRes is: 1 ArcaneRes is: 1
MIN_DAMAGE is: 11.200000 MAX_DAMAGE is: 19.200001
MIN_OFFHAND_DAMAGE is: 0.000000 MAX_OFFHAND_DAMAGE is: 0.000000
MIN_RANGED_DAMAGE is: 101.371429 MAX_RANGED_DAMAGE is: 251.371429
ATTACK_TIME is: 2200 RANGE_ATTACK_TIME is: 1200
PLAYER: UpdateHonor
Player: channels cleaned up!
Aura 22 now is remove mode 5
Aura 29 now is remove mode 5
Aura 34 now is remove mode 5
Aura 35 now is remove mode 5
Aura 22 now is remove mode 5
Aura 107 now is remove mode 5
Aura 108 now is remove mode 5
Aura 88 now is remove mode 5
Aura 116 now is remove mode 5
Aura 168 now is remove mode 5
Aura 98 now is remove mode 5
Aura 98 now is remove mode 5
SQL: can not execute 'DELETE FROM characters WHERE guid = ?'
SQL ERROR: Lost connection to MySQL server during query
Player: channels cleaned up!
SESSION: Sent SMSG_LOGOUT_COMPLETE Message
SQL: can not execute 'UPDATE characters SET online = 0 WHERE account = ?'
SQL ERROR: Lost connection to MySQL server during query
Build enum data for char guid 7 from account 4.
WorldSocket::handle_input: Peer has closed connection
[/code]
SQL server has not been touched. It just decided today that it was going to screw up again. The AH is also empty again. I can NOT get it to repopulate, even if I restart the server. It's like the server "learned" my time-out was 900 so it is now waiting 1200 or something. Also, during startup the AHBot appears to work but it is not again.
[code]
Sorting and cleaning items for AHBot seller...
[AuctionHouseBot seller will use 3675 items to fill auction house (according you
r config choices)
Items loaded Grey White Green Blue Purple Orange Yellow
Consumable 0 93 0 0 0 0 0
Container 0 21 1 0 0 0 0
Weapon 81 60 187 189 5 0 0
Jewelry(OBSOLETE) 0 0 0 0 0 0 0
Armor 227 200 1302 261 22 0 0
Reagent 2 24 6 2 0 0 0
Projectile 0 1 1 1 0 0 0
Trade Goods 2 158 39 11 3 0 0
Generic(OBSOLETE) 0 0 0 0 0 0 0
Recipe 0 12 275 63 14 0 0
Money(OBSOLETE) 0 0 0 0 0 0 0
Quiver 0 0 0 0 0 0 0
Quest 0 13 0 0 0 0 0
Key 0 2 1 0 0 0 0
Permanent(OBSOLETE) 0 0 0 0 0 0 0
Miscellaneous 278 83 2 33 0 0 0

AHBot seller configuration data loaded and initilized


[Eluna]: Successfully loaded `lua_scripts/extensions/ObjectVariables.ext`
[Eluna]: Successfully loaded `lua_scripts/extensions/StackTracePlus/StackTracePlus.ext`
[Eluna]: Successfully loaded `lua_scripts/extensions/_Misc.ext`
[Eluna]: Executed 3 Lua scripts in 2 ms
------------------------
WORLD: World initialized
------------------------

SERVER STARTUP TIME: 0 minutes 3 seconds

[New Thread 0x7fffefa75700 (LWP 980)]
[New Thread 0x7fffef274700 (LWP 981)]

mangos>Max allowed socket connections 1024
[New Thread 0x7fffeea73700 (LWP 982)]
Network Thread Starting
[New Thread 0x7fffee272700 (LWP 983)]
Network Thread Starting
Map::GetHitPosition vmaps corrects gained with dynamic objects! new dest coords are X:-10584.904297 Y:-1196.747314 Z:28.749500
ahbot status
CLI command under processing...
Table `command` overwrite for command 'honor show' default security (2) by 1
Table `command` overwrite for command 'modify honor' default security (1) by 2
==========================================================
| | Alliance | Horde | Neutral | Total |
|--------------------------------------------------------|
| Count | 0 | 0 | 0 | 0 |
==========================================================
Command: ahbot status [Account: 0 from Console]
mangos>
[/code]
As you can see, it populates in memory but never actually has it in-game. Upping my time-out from 600 to 900 fixed it before, but we should not have to keep upping the time-out to make the server work. Something is very wrong with our SQL connection code. I am looking into it but I need to understand it before I can draw conclusions about it. This may take me a while.

*UPDATE*

I decided to try doubling both "net_read_timeout" and "net_write_timeout". Now only time will tell. I am also going to monitor my process list (SHOW PROCESSLIST;) and see if I get many idle connections.

*UPDATE*

The issue appears to be that the server won't close connections when nobody is on and then reopen them when a player logs in, so they time-out and MaNGOS doesn't realize it.
[code]
MariaDB [(none)]> SHOW PROCESSLIST;
+-----+------+-------------------+------------+---------+------+-------+------------------+----------+
| Id | User | Host | db | Command | Time | State | Info | Progress |
+-----+------+-------------------+------------+---------+------+-------+------------------+----------+
| 161 | zero | x.y.z.211:46768 | realms | Sleep | 706 | | NULL | 0.000 |
| 162 | zero | x.y.z.211:46769 | realms | Sleep | 706 | | NULL | 0.000 |
| 163 | zero | x.y.z.211:46770 | zero_world | Sleep | 854 | | NULL | 0.000 |
| 164 | zero | x.y.z.211:46771 | zero_world | Sleep | 863 | | NULL | 0.000 |
| 165 | zero | x.y.z.211:46772 | zero_toons | Sleep | 686 | | NULL | 0.000 |
| 166 | zero | x.y.z.211:46773 | zero_toons | Sleep | 534 | | NULL | 0.000 |
| 167 | zero | x.y.z.211:46774 | realms | Sleep | 706 | | NULL | 0.000 |
| 168 | zero | x.y.z.211:46775 | realms | Sleep | 258 | | NULL | 0.000 |
| 169 | root | localhost | NULL | Query | 0 | NULL | SHOW PROCESSLIST | 0.000 |
+-----+------+-------------------+------------+---------+------+-------+------------------+----------+
9 rows in set (0.01 sec)
[/code]
The time keeps increasing. These will get dropped if nothing occurs soon.

*UPDATE*

It happened. By the time I posted my last update, several connections reached the timeout.
[code]
MariaDB [(none)]> SHOW PROCESSLIST;
+-----+------+-------------------+------------+---------+------+-------+------------------+----------+
| Id | User | Host | db | Command | Time | State | Info | Progress |
+-----+------+-------------------+------------+---------+------+-------+------------------+----------+
| 165 | zero | x.y.z.211:46772 | zero_toons | Sleep | 898 | | NULL | 0.000 |
| 166 | zero | x.y.z.211:46773 | zero_toons | Sleep | 746 | | NULL | 0.000 |
| 168 | zero | x.y.z.211:46775 | realms | Sleep | 470 | | NULL | 0.000 |
| 169 | root | localhost | NULL | Query | 0 | NULL | SHOW PROCESSLIST | 0.000 |
+-----+------+-------------------+------------+---------+------+-------+------------------+----------+
4 rows in set (0.00 sec)
[/code]
Now if it is unable to save or function in another way, we have found the issue.

*UPDATE*

After I finished posting THAT update, I checked again...
[code]
MariaDB [(none)]> SHOW PROCESSLIST;
+-----+------+-------------------+--------+---------+------+-------+------------------+----------+
| Id | User | Host | db | Command | Time | State | Info | Progress |
+-----+------+-------------------+--------+---------+------+-------+------------------+----------+
| 168 | zero | x.y.z.211:46775 | realms | Sleep | 48 | | NULL | 0.000 |
| 169 | root | localhost | NULL | Query | 0 | NULL | SHOW PROCESSLIST | 0.000 |
+-----+------+-------------------+--------+---------+------+-------+------------------+----------+
2 rows in set (0.00 sec)
[/code]
Our connections being forced to close and MaNGOS obviously does not know this. Now we need to do two things. First, only open a connection when we need it if nobody is on. Second, detect these aborted connections and handle it!

Also, note how the only open connection keeps doing something which is resetting its timer...

Alright, I believe I just figured it out! If I start my server and play, all is good. However, if I start it and let the connections time-out, then login and play, I get loads of SQL connection errors, players are NOT saved, and nothing appears to work correctly. I then tried restarting the server. Everything works until the connection times out. This leads me to two conclusions.

First, we are not closing idle connections. This is a no-no.
Second, after the connection is closed by the DB engine, MaNGOS does not realize it so it is sending queries to the now closed connections. This is why it instantly fails when a player logs in or out.

So, how do we fix this? It is going to take some collaboration to correct this issue. We have to fix BOTH of the issues above.

First, we need to agree on a way to handle idle connections. It is a best practice to close them using "mysql_close". Our time-out should be less than the MySQL default of 600 seconds (ten minutes) or the DB engine will close them forcefully. My thinking is that on a populated server (one to a million players) the connections will be used frequently enough that we never hit our time-out and all will be good. If nobody is on, then it doesn't matter if they close.

Second, the simple fix for the DB engine closing the connection would be to check it before any query is run. I do not remember the code off the top of my head, but the last time I did a C++, Windows app that used MySQL, I used some object method which checked to see if the connection was good before I ran my query. If not, I simply recreated it and updated the handle to the connection. We should be able to do this here. This solution is a band-aid. We should NOT be keeping six or eight idle connections open when nobody is on.

One thing that bothers me however, is that once all of the connections timed out, I had my fiancée login. Several connections WERE created again, including four to the realm database and one to the characters database. However, despite there being a connection to the character DB, I got SQL errors. This could mean that despite the connection being there, some of the server code is not being updated to the new connection handle, preventing it from using the connected, valid connection. I have no idea how to easily troubleshoot that.

I got off work early today so I came home and did some testing. If I start the server and join within fifteen minutes the server is fine for as long as you play. Once the last player quits, you have roughly fifteen minutes until the DB server breaks connections. After that, you can no longer save your character or update server info such as population. Here are my test cases.

Started server, joined, played for an hour
Quit, joined five minutes later, played for thirty minutes
Quit, waited EXACTLY sixteen minutes (one minute beyond timeout), joined, had SQL errors

Started the server, waited sixteen minutes, joined, had SQL errors

Started the server, waited fourteen minutes, joined, played for thirty minutes
Quit, waited fourteen minutes, joined, played fifteen minutes
Quit, waited sixteen minutes, joined, had SQL errors

What I am getting at, is that this is reproducible. Once your SQL timeout is reached and the connection is broken, we are hosed. However, if you have upgraded MySQL over the years your default will be 28,800 seconds, or eight hours as it was when you first installed it. This means that if you run a server all the time, such as Covenant, as long as a single player logs into the server prior to the timeout, the timeout is reset and the server chugs along. However, with a new install on a new system with a recent MariaDB or MySQL, this problem rears its ugly nose.

I am currently digging into the MySQL client code in MaNGOS, but it is slow-going. Not my coding style and I need to understand the program-flow before I make changes, but my plan is to simply detect a closed connection and, if it is closed, reopen it prior to doing the query.

Although it will certainly fix the issue, I don't think this is a good idea. You will only hack-fix the code by doing this everywhere.

If you want to fix it properly, you need to figure out the complete DB-connection strategy of Mangos and, if there's none, put one in place.

I'm convinced that with a simple singleton-pattern, you can achieve it efficiently.

Actually the correct fix would be to determine the connection timeout when connected to the DB server and set all connections to maybe half of that. If a query is sent before the local timeout expires, reset the timer. If not, it does a graceful close via mysql_close. What I am proposing is not a hack, but something fairly standard in business apps. Just check the connection prior to sending a query. I am still going over the MySQL code in MaNGOS, but my assumption is that this will only need to be applied in one location. That is to say, in one class. I am currently still figuring out program-flow through the classes, but hope to have it figured out by Friday and begin working on a fix shortly thereafter.

I am not saying something else than you though...

"DB-Connection strategy" actually means a real DB layer with connection management... Never said you don't have to check if you connection is open.

Ah OK, I misunderstood you.

My reason for dropping by is that the SQL issue caused a server to go completely down. I am already working on a fix, but this server is bone-stock. I have a test server I do my own modifications on.
[code]
Map::GetHitPosition vmaps corrects gained with static objects! new dest coords are X:-10585.773438 Y:-1194.180786 Z:28.749500
[ ] 0%
SQL ERROR: wrong amount of parameters (2 instead of 0)
SQL ERROR: statement: UPDATE characters SET stored_honorable_kills = stored_honorable_kills + %u WHERE guid = %u
/usr/src/zero/server/src/shared/Database/SqlPreparedStatement.cpp:68: Error: Assertion in Execute failed: false
Stack Trace:
/home//zero/bin/mangosd(_ZN9ObjectMgr15FlushRankPointsEj+0x23a) [0xaaa654]
/home//zero/bin/mangosd(_ZN5World22ServerMaintenanceStartEv+0x68) [0xd644aa]
/home//zero/bin/mangosd(_ZN5World6UpdateEj+0x378) [0xd62ede]
/home//zero/bin/mangosd(_ZN13WorldRunnable3runEv+0x64) [0xa3a254]
/home//zero/bin/mangosd(_ZN9ACE_Based6Thread10ThreadTaskEPv+0x37) [0xeb5a59]
/usr/lib/libACE-6.0.3.so(_ZN21ACE_OS_Thread_Adapter6invokeEv+0xa6) [0x7ffff7745636]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50) [0x7ffff5edab50]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ffff5c2495d]
mangosd: /usr/src/zero/server/src/shared/Database/SqlPreparedStatement.cpp:68: bool SqlStatement::Execute(): Assertion `"false" && 0' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffefa75700 (LWP 10736)]
0x00007ffff5b7b165 in raise () from /lib/x86_64-linux-gnu/libc.so.6
[/code]
No issues prior to that. I did hide my username in the dump above, but I changed nothing else. This may be a different issue though. I had killed some Alliance NPCs earlier and managed to nail two dishonorable kills in the process. Perhaps when it was trying to update my honor this bug reared its head.

At least, you have the file where it happened.

In mangosd.conf you have a line reading MaxPingTime. Set this to a value lower than your MySQL`s wait_timeout and I bet all these problems will dissapear (the default value for MaxPingTime is 30min, which was ok for a wait_timeout of 28800).

I have not seen MaxPingTime referenced in the SQL classes I saw, but I admit I have not been over every line. I do see it in the config file. I will set it to five minutes (half the default time-out now) and see how things run. Thank you for pointing this out. If it works, I will simply do a PR with an updated MaxPingTime default.

That fixed it. I reset the MySQL configuration to a ten-minute time-out and set MaxPingTime to 5, and it has been running for 36hrs without a hitch. I will do a PR to fix the configuration file this evening. Credit goes to H0zen for mentioning the setting. I do now see how it affects the connection manager in the code, but this is good. I still believe that we need a way to detect broken connections however, to avoid a player losing an hour of play in the event something else causes a connection to break, but this is not high priority by any means.

I am closing this. I did a PR which was already merged on this issue.