From e8d18cb014fc2123518056fe5dbf9d8f17360da6 Mon Sep 17 00:00:00 2001 From: Akkadius Date: Sun, 1 Nov 2015 20:29:51 -0600 Subject: [PATCH] Made many performance optimizing oriented code changes in the source - Added Rate limit the rate in which signals are processed for NPC's (.5 seconds instead of .01 seconds) Added Perl Export Settings which should heavily reduce the Perl footprint - Normally when any sub EVENT_ gets triggered, all kinds of variables have to get exported every single time an event is triggered and this can make Perl very slow when events are triggered constantly - The two most taxing variable exports are the item variables ($itemcount{} $hasitem{} $oncursor{}) and qglobals ($qglobals{}) - qglobals can pose to be an issue quickly when global qglobals build up, it is highly recommend to use the GetGlobal() and SetGlobal() methods instead as they don't reference the hashmap $qglobals{} that is rebuilt every single time a sub event is triggered - A stress test conducted with 10,000 samples shows an excess of time taken to export variables: http://i.imgur.com/NEpW1tS.png - After the Perl Export Settings table is implemented, and all exports are shut off you see the following test result: http://i.imgur.com/Du5hth9.png - The difference of eliminating uneeded exports brings the overhead and footprint of 10,000 triggers from 54 seconds to 2 seconds - In a 10,000 sample test (10,000 sub event triggers), exporting item variables adds 12 seconds alone, when item variables are only needed in EVENT_ITEM and EVENT_SAY a majority of the time if at all - In a 10,000 sample test (10,000 sub event triggers), exporting qglobals with approximately 1,000 global qglobals in the database creates about 11-20 seconds of delay on its own (Depending on hardware of course) - I've written a parser that has determined which of these exports are needed in which sub routines and have turned off all of the unneeded exports in sub routines that do not need them and used it to create the default table that will be installed in the database. - The export table is called 'perl_event_export_settings' and it resembles the following structure and contains all current 81 EVENTS - If an entry doesn't exist in this table and a new subroutine is added to the source, all exports will be on by default for that routine +----------+-----------------------------------------+-----------------+------------+-------------+-------------+--------------+ | event_id | event_description | export_qglobals | export_mob | export_zone | export_item | export_event | +----------+-----------------------------------------+-----------------+------------+-------------+-------------+--------------+ | 0 | EVENT_SAY | 1 | 1 | 1 | 1 | 1 | | 1 | EVENT_ITEM | 1 | 1 | 1 | 0 | 1 | | 2 | EVENT_DEATH | 1 | 1 | 1 | 0 | 1 | | 3 | EVENT_SPAWN | 1 | 1 | 1 | 0 | 1 | | 4 | EVENT_ATTACK | 0 | 1 | 1 | 0 | 1 | | 5 | EVENT_COMBAT | 1 | 1 | 1 | 0 | 1 | +----------+-----------------------------------------+-----------------+------------+-------------+-------------+--------------+ - If a change is made to this table while the server is live and running, you can hot reload all zone process settings via: #reloadperlexportsettings - For those who wonder what "exports" are, they are reference to variables that are made available at runtime of the sub event, such as: (export_qglobals) (Heavy) : $qglobals https://github.com/EQEmu/Server/blob/master/zone/embparser.cpp#L916 (export_item) (Heavy) : $itemcount{} $hasitem{} $oncursor{} https://github.com/EQEmu/Server/blob/master/zone/embparser.cpp#L1103 (export_zone) : $zoneid, $instanceid, $zoneln etc. https://github.com/EQEmu/Server/blob/master/zone/embparser.cpp#L1083 (export_mob) : $x, $y, $z, $h, $hpratio etc. https://github.com/EQEmu/Server/blob/master/zone/embparser.cpp#L1032 (export_event) : (event specific) IE: EVENT_SAY ($text) https://github.com/EQEmu/Server/blob/master/zone/embparser.cpp#L1141 --- changelog.txt | 42 +++++++ common/version.h | 2 +- utils/sql/db_update_manifest.txt | 1 + .../2015_11_01_perl_event_export_settings.sql | 116 ++++++++++++++++++ 4 files changed, 160 insertions(+), 1 deletion(-) create mode 100644 utils/sql/git/required/2015_11_01_perl_event_export_settings.sql diff --git a/changelog.txt b/changelog.txt index e3f74ce08..7aa6ee7e1 100644 --- a/changelog.txt +++ b/changelog.txt @@ -1,5 +1,47 @@ EQEMu Changelog (Started on Sept 24, 2003 15:50) ------------------------------------------------------- +== 11/1/2015 == +Akkadius: Made many performance optimizing oriented code changes in the source + - Added Rate limit the rate in which signals are processed for NPC's (.5 seconds instead of .01 seconds) +Akkadius: Added Perl Export Settings which should heavily reduce the Perl footprint + - Normally when any sub EVENT_ gets triggered, all kinds of variables have to get exported every single time an event is triggered and + this can make Perl very slow when events are triggered constantly + - The two most taxing variable exports are the item variables ($itemcount{} $hasitem{} $oncursor{}) and qglobals ($qglobals{}) + - qglobals can pose to be an issue quickly when global qglobals build up, it is highly recommend to use the GetGlobal() and SetGlobal() + methods instead as they don't reference the hashmap $qglobals{} that is rebuilt every single time a sub event is triggered + - A stress test conducted with 10,000 samples shows an excess of time taken to export variables: http://i.imgur.com/NEpW1tS.png + - After the Perl Export Settings table is implemented, and all exports are shut off you see the following test result: + http://i.imgur.com/Du5hth9.png + - The difference of eliminating uneeded exports brings the overhead and footprint of 10,000 triggers from 54 seconds to 2 seconds + - In a 10,000 sample test (10,000 sub event triggers), exporting item variables adds 12 seconds alone, when item variables are only needed in + EVENT_ITEM and EVENT_SAY a majority of the time if at all + - In a 10,000 sample test (10,000 sub event triggers), exporting qglobals with approximately 1,000 global qglobals in the database creates + about 11-20 seconds of delay on its own (Depending on hardware of course) + - I've written a parser that has determined which of these exports are needed in which sub routines and have turned off all of the unneeded + exports in sub routines that do not need them and used it to create the default table that will be installed in the database. + - The export table is called 'perl_event_export_settings' and it resembles the following structure and contains all current 81 EVENTS + - If an entry doesn't exist in this table and a new subroutine is added to the source, all exports will be on by default for that routine + + +----------+-----------------------------------------+-----------------+------------+-------------+-------------+--------------+ + | event_id | event_description | export_qglobals | export_mob | export_zone | export_item | export_event | + +----------+-----------------------------------------+-----------------+------------+-------------+-------------+--------------+ + | 0 | EVENT_SAY | 1 | 1 | 1 | 1 | 1 | + | 1 | EVENT_ITEM | 1 | 1 | 1 | 0 | 1 | + | 2 | EVENT_DEATH | 1 | 1 | 1 | 0 | 1 | + | 3 | EVENT_SPAWN | 1 | 1 | 1 | 0 | 1 | + | 4 | EVENT_ATTACK | 0 | 1 | 1 | 0 | 1 | + | 5 | EVENT_COMBAT | 1 | 1 | 1 | 0 | 1 | + +----------+-----------------------------------------+-----------------+------------+-------------+-------------+--------------+ + + - If a change is made to this table while the server is live and running, you can hot reload all zone process settings via: + #reloadperlexportsettings + - For those who wonder what "exports" are, they are reference to variables that are made available at runtime of the sub event, such as: + (export_qglobals) (Heavy) : $qglobals https://github.com/EQEmu/Server/blob/master/zone/embparser.cpp#L916 + (export_item) (Heavy) : $itemcount{} $hasitem{} $oncursor{} https://github.com/EQEmu/Server/blob/master/zone/embparser.cpp#L1103 + (export_zone) : $zoneid, $instanceid, $zoneln etc. https://github.com/EQEmu/Server/blob/master/zone/embparser.cpp#L1083 + (export_mob) : $x, $y, $z, $h, $hpratio etc. https://github.com/EQEmu/Server/blob/master/zone/embparser.cpp#L1032 + (export_event) : (event specific) IE: EVENT_SAY ($text) https://github.com/EQEmu/Server/blob/master/zone/embparser.cpp#L1141 + == 10/16/2015 == Uleat: Added command '#bot clearfollowdistance [ | spawned | all ]' to coincide with the activation of the load/save feature for follow_distance diff --git a/common/version.h b/common/version.h index d8a6b6077..3a3969ec1 100644 --- a/common/version.h +++ b/common/version.h @@ -30,7 +30,7 @@ Manifest: https://github.com/EQEmu/Server/blob/master/utils/sql/db_update_manifest.txt */ -#define CURRENT_BINARY_DATABASE_VERSION 9087 +#define CURRENT_BINARY_DATABASE_VERSION 9088 #ifdef BOTS #define CURRENT_BINARY_BOTS_DATABASE_VERSION 9000 #else diff --git a/utils/sql/db_update_manifest.txt b/utils/sql/db_update_manifest.txt index 239fdbb67..2ffc2f452 100644 --- a/utils/sql/db_update_manifest.txt +++ b/utils/sql/db_update_manifest.txt @@ -341,6 +341,7 @@ 9085|2015_07_01_Marquee_Rule.sql|SELECT * FROM `rule_values` WHERE `rule_name` LIKE '%Character:MarqueeHPUpdates%'|empty| 9086|2015_07_02_aa_rework.sql|SHOW TABLES LIKE 'aa_ranks'|empty| 9087|2015_09_25_inventory_snapshots.sql|SHOW TABLES LIKE 'inventory_snapshots'|empty| +9088|2015_11_01_perl_event_export_settings.sql|SHOW TABLES LIKE 'perl_event_export_settings'|empty| # Upgrade conditions: # This won't be needed after this system is implemented, but it is used database that are not diff --git a/utils/sql/git/required/2015_11_01_perl_event_export_settings.sql b/utils/sql/git/required/2015_11_01_perl_event_export_settings.sql new file mode 100644 index 000000000..1a06535d0 --- /dev/null +++ b/utils/sql/git/required/2015_11_01_perl_event_export_settings.sql @@ -0,0 +1,116 @@ +/* +Navicat MySQL Data Transfer + +Source Server : localhost +Source Server Version : 50505 +Source Host : localhost:3306 +Source Database : peq + +Target Server Type : MYSQL +Target Server Version : 50505 +File Encoding : 65001 + +Date: 2015-11-01 20:28:21 +*/ + +SET FOREIGN_KEY_CHECKS=0; + +-- ---------------------------- +-- Table structure for perl_event_export_settings +-- ---------------------------- +DROP TABLE IF EXISTS `perl_event_export_settings`; +CREATE TABLE `perl_event_export_settings` ( + `event_id` int(11) NOT NULL, + `event_description` varchar(150) DEFAULT NULL, + `export_qglobals` smallint(11) DEFAULT '0', + `export_mob` smallint(11) DEFAULT '0', + `export_zone` smallint(11) DEFAULT '0', + `export_item` smallint(11) DEFAULT '0', + `export_event` smallint(11) DEFAULT '0', + PRIMARY KEY (`event_id`) +) ENGINE=InnoDB DEFAULT CHARSET=latin1; + +-- ---------------------------- +-- Records of perl_event_export_settings +-- ---------------------------- +INSERT INTO `perl_event_export_settings` VALUES ('0', 'EVENT_SAY', '1', '1', '1', '1', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('1', 'EVENT_ITEM', '1', '1', '1', '1', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('2', 'EVENT_DEATH', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('3', 'EVENT_SPAWN', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('4', 'EVENT_ATTACK', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('5', 'EVENT_COMBAT', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('6', 'EVENT_AGGRO', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('7', 'EVENT_SLAY', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('8', 'EVENT_NPC_SLAY', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('9', 'EVENT_WAYPOINT_ARRIVE', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('10', 'EVENT_WAYPOINT_DEPART', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('11', 'EVENT_TIMER', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('12', 'EVENT_SIGNAL', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('13', 'EVENT_HP', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('14', 'EVENT_ENTER', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('15', 'EVENT_EXIT', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('16', 'EVENT_ENTERZONE', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('17', 'EVENT_CLICKDOOR', '1', '1', '1', '1', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('18', 'EVENT_LOOT', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('19', 'EVENT_ZONE', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('20', 'EVENT_LEVEL_UP', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('21', 'EVENT_KILLED_MERIT', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('22', 'EVENT_CAST_ON', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('23', 'EVENT_TASKACCEPTED', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('24', 'EVENT_TASK_STAGE_COMPLETE', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('25', 'EVENT_TASK_UPDATE', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('26', 'EVENT_TASK_COMPLETE', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('27', 'EVENT_TASK_FAIL', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('28', 'EVENT_AGGRO_SAY', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('29', 'EVENT_PLAYER_PICKUP', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('30', 'EVENT_POPUPRESPONSE', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('31', 'EVENT_ENVIRONMENTAL_DAMAGE', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('32', 'EVENT_PROXIMITY_SAY', '1', '1', '1', '1', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('33', 'EVENT_CAST', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('34', 'EVENT_CAST_BEGIN', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('35', 'EVENT_SCALE_CALC', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('36', 'EVENT_ITEM_ENTER_ZONE', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('37', 'EVENT_TARGET_CHANGE', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('38', 'EVENT_HATE_LIST', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('39', 'EVENT_SPELL_EFFECT_CLIENT', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('40', 'EVENT_SPELL_EFFECT_NPC', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('41', 'EVENT_SPELL_EFFECT_BUFF_TIC_CLIENT', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('42', 'EVENT_SPELL_EFFECT_BUFF_TIC_NPC', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('43', 'EVENT_SPELL_FADE', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('44', 'EVENT_SPELL_EFFECT_TRANSLOCATE_COMPLETE', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('45', 'EVENT_COMBINE_SUCCESS', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('46', 'EVENT_COMBINE_FAILURE', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('47', 'EVENT_ITEM_CLICK', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('48', 'EVENT_ITEM_CLICK_CAST', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('49', 'EVENT_GROUP_CHANGE', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('50', 'EVENT_FORAGE_SUCCESS', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('51', 'EVENT_FORAGE_FAILURE', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('52', 'EVENT_FISH_START', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('53', 'EVENT_FISH_SUCCESS', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('54', 'EVENT_FISH_FAILURE', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('55', 'EVENT_CLICK_OBJECT', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('56', 'EVENT_DISCOVER_ITEM', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('57', 'EVENT_DISCONNECT', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('58', 'EVENT_CONNECT', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('59', 'EVENT_ITEM_TICK', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('60', 'EVENT_DUEL_WIN', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('61', 'EVENT_DUEL_LOSE', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('62', 'EVENT_ENCOUNTER_LOAD', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('63', 'EVENT_ENCOUNTER_UNLOAD', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('64', 'EVENT_SAY', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('65', 'EVENT_DROP_ITEM', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('66', 'EVENT_DESTROY_ITEM', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('67', 'EVENT_FEIGN_DEATH', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('68', 'EVENT_WEAPON_PROC', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('69', 'EVENT_EQUIP_ITEM', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('70', 'EVENT_UNEQUIP_ITEM', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('71', 'EVENT_AUGMENT_ITEM', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('72', 'EVENT_UNAUGMENT_ITEM', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('73', 'EVENT_AUGMENT_INSERT', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('74', 'EVENT_AUGMENT_REMOVE', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('75', 'EVENT_ENTER_AREA', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('76', 'EVENT_LEAVE_AREA', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('77', 'EVENT_RESPAWN', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('78', 'EVENT_DEATH_COMPLETE', '1', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('79', 'EVENT_UNHANDLED_OPCODE', '0', '1', '1', '0', '1'); +INSERT INTO `perl_event_export_settings` VALUES ('80', 'EVENT_TICK', '0', '1', '1', '0', '1');