[Release] 22.31.1

* [Linux] Add symbols to release builds

* Update linux-build.sh

.drone.yml

@@ -15,7 +15,7 @@ volumes:
 
 steps:
   - name: Build Linux X64
-    image: akkadius/eqemu-server:v11
+    image: akkadius/eqemu-server:v13
     environment:
       GITHUB_TOKEN:
         from_secret: GH_RELEASE_GITHUB_API_TOKEN

CHANGELOG.md

@@ -1,3 +1,403 @@
+## [22.31.1] - 10/31/2023
+
+### Fixes
+
+* Hotfix issue with blocked spells not loading properly @Akkadius 2023-10-31
+
+## [22.31.0] - 10/29/2023
+
+### Crash
+
+* Fix crash when client pointer does not exist during #hotfix ([#3661](https://github.com/EQEmu/Server/pull/3661)) @Akkadius 2023-10-29
+* Fix spell in AESpell related to beacons ([#3659](https://github.com/EQEmu/Server/pull/3659)) @Akkadius 2023-10-29
+
+### Database
+
+* Add id to variables table ([#3658](https://github.com/EQEmu/Server/pull/3658)) @Akkadius 2023-10-29
+
+### Linux
+
+* Add symbols to release builds ([#3660](https://github.com/EQEmu/Server/pull/3660)) @Akkadius 2023-10-29
+
+### Perl
+
+* Revert " Reload perl quests on zone bootup " ([#3648](https://github.com/EQEmu/Server/pull/3648)) @Akkadius 2023-10-26
+
+### Trading
+
+* Fix part 3 of Issue 932. ([#3654](https://github.com/EQEmu/Server/pull/3654)) @noudess 2023-10-29
+
+## [22.30.2] - 10/26/2023
+
+### Fixes
+
+Revert Perl regression in #3648 causing scripts to not reliably initialize on zone bootup. @Akkadius 2023-10-26
+
+## [22.30.1] - 10/24/2023
+
+### Fixes
+
+* Fix empty InsertMany in bot starting items. ([#3653](https://github.com/EQEmu/Server/pull/3653)) @Kinglykrab 2023-10-24
+
+## [22.30.0] - 10/23/2023
+
+### API
+
+* Implement Zone Sidecar ([#3635](https://github.com/EQEmu/Server/pull/3635)) @Akkadius 2023-10-24
+
+### Commands
+
+* Move #suspend from content database ([#3651](https://github.com/EQEmu/Server/pull/3651)) @joligario 2023-10-24
+
+### Fixes
+
+* Fix Bot Starting Items SQL ([#3649](https://github.com/EQEmu/Server/pull/3649)) @Kinglykrab 2023-10-23
+
+### Perl
+
+* Implement eqemu-perl for Linux ([#3652](https://github.com/EQEmu/Server/pull/3652)) @Akkadius 2023-10-24
+* Reload perl quests on zone bootup ([#3648](https://github.com/EQEmu/Server/pull/3648)) @hgtw 2023-10-24
+
+### Pets
+
+* Disallow effect of alliance line when cast on pets. ([#3650](https://github.com/EQEmu/Server/pull/3650)) @noudess 2023-10-24
+
+## [22.29.1] - 10/21/2023
+
+### DB
+
+* Fix manifest for blocked spells ([#3646](https://github.com/EQEmu/Server/pull/3646)) @joligario 2023-10-21
+
+### Fixes
+
+* Fix issue with subcommand settings not working ([#3643](https://github.com/EQEmu/Server/pull/3643)) @Kinglykrab 2023-10-21
+* Hotfix command without hotfix name ([#3644](https://github.com/EQEmu/Server/pull/3644)) @joligario 2023-10-21
+* Verifying mail keys when none exist ([#3645](https://github.com/EQEmu/Server/pull/3645)) @joligario 2023-10-21
+
+## [22.29.0] - 10/20/2023
+
+### Feature
+
+* Add Expansion and Content Flag support to Blocked Spells ([#3638](https://github.com/EQEmu/Server/pull/3638)) @Kinglykrab 2023-10-20
+
+### Fixes
+
+* Fix crash when checking Bot Group/Raid membership ([#3641](https://github.com/EQEmu/Server/pull/3641)) @Aeadoin 2023-10-20
+
+### Perl
+
+* Static linker fix on Linux ([#3642](https://github.com/EQEmu/Server/pull/3642)) @Akkadius 2023-10-20
+
+### Rules
+
+* Add rule to configure max number of procs per round Combat:MaxProcs ([#3640](https://github.com/EQEmu/Server/pull/3640)) @Akkadius 2023-10-20
+
+## [22.28.1] - 10/20/2023
+
+### Build
+
+* Perl Linux build fix
+
+## [22.28.0] - 10/15/2023
+
+### Bots
+
+* Adjust Bot Movement Speed ([#3615](https://github.com/EQEmu/Server/pull/3615)) @Kinglykrab 2023-10-14
+* Fix bot removal on zone, regression from #3611 ([#3631](https://github.com/EQEmu/Server/pull/3631)) @Akkadius 2023-10-16
+
+### Crash
+
+* Fix Crash with #summon ([#3618](https://github.com/EQEmu/Server/pull/3618)) @Kinglykrab 2023-10-14
+* Fix crash in Mob::ShowBuffs ([#3632](https://github.com/EQEmu/Server/pull/3632)) @Akkadius 2023-10-16
+* Resolve crash when assigning empty raid note. ([#3628](https://github.com/EQEmu/Server/pull/3628)) @Aeadoin 2023-10-15
+
+### Feature
+
+* Add Extra Kick Classes ([#3613](https://github.com/EQEmu/Server/pull/3613)) @Kinglykrab 2023-10-11
+* Add Immune to Assassinate Special Ability ([#3622](https://github.com/EQEmu/Server/pull/3622)) @Kinglykrab 2023-10-14
+* Add Immune to Headshot Special Ability ([#3624](https://github.com/EQEmu/Server/pull/3624)) @Kinglykrab 2023-10-14
+* Update Raid Functions for Titanium and Underfoot ([#3524](https://github.com/EQEmu/Server/pull/3524)) @neckkola 2023-10-14
+
+### Fixes
+
+* Fix #cast defaulting to cast time ([#3617](https://github.com/EQEmu/Server/pull/3617)) @Kinglykrab 2023-10-14
+
+### Parser Fix
+
+* Fix SendIllusion Spire parsing ([#3623](https://github.com/EQEmu/Server/pull/3623)) @Kinglykrab 2023-10-14
+
+### Quest API
+
+* Add GrantAllAAPoints() to Perl/Lua and Modify #grantaa ([#3616](https://github.com/EQEmu/Server/pull/3616)) @Kinglykrab 2023-10-14
+* Add target ID and spell exports to events ([#3620](https://github.com/EQEmu/Server/pull/3620)) @Kinglykrab 2023-10-15
+
+### Scripts
+
+* Update 13th Floor importer ([#3630](https://github.com/EQEmu/Server/pull/3630)) @joligario 2023-10-16
+* Update 13th Floor script for legacy research tome bagtypes ([#3621](https://github.com/EQEmu/Server/pull/3621)) @joligario 2023-10-14
+
+## [22.27.0] - 10/07/2023
+
+### Crash
+
+* Bot member zoned crash fix ([#3607](https://github.com/EQEmu/Server/pull/3607)) @Akkadius 2023-10-07
+* Fix #summon crash ([#3608](https://github.com/EQEmu/Server/pull/3608)) @Akkadius 2023-10-07
+* Fix CanUseAlternateAdvancementRank crash ([#3609](https://github.com/EQEmu/Server/pull/3609)) @Akkadius 2023-10-07
+* Fix crash in #movechar ([#3612](https://github.com/EQEmu/Server/pull/3612)) @Akkadius 2023-10-07
+* Fix crash in CastSpell Quest API input cast ([#3610](https://github.com/EQEmu/Server/pull/3610)) @Akkadius 2023-10-07
+* Fix dangling pointer crash observed in SendHPPacketsFrom ([#3611](https://github.com/EQEmu/Server/pull/3611)) @Akkadius 2023-10-07
+* Fix rarer crash with File::Makedir ([#3606](https://github.com/EQEmu/Server/pull/3606)) @Akkadius 2023-10-07
+
+### Fixes
+
+* Add Validation to #find, #set, and #show args ([#3598](https://github.com/EQEmu/Server/pull/3598)) @Kinglykrab 2023-09-18
+* Ensure Linux builds report failures  @Akkadius 2023-10-03
+* Fix #show group_info Popup ([#3605](https://github.com/EQEmu/Server/pull/3605)) @Kinglykrab 2023-10-04
+* Fix swarm pet names to use '_' instead of ' ' ([#3601](https://github.com/EQEmu/Server/pull/3601)) @noudess 2023-09-19
+* Invis vs. Undead/Animal Breaks Charm for Pets ([#3587](https://github.com/EQEmu/Server/pull/3587)) @crdunwel 2023-09-19
+
+### Logs
+
+* Change pathing log messages from Error to Pathing. ([#3604](https://github.com/EQEmu/Server/pull/3604)) @joligario 2023-09-29
+
+### Quest API
+
+* Add Caster ID Parameter to FindBuff in Perl/Lua ([#3590](https://github.com/EQEmu/Server/pull/3590)) @Kinglykrab 2023-09-29
+
+## [22.26.2] - 09/18/2023
+
+### Fixes
+
+* Fix an issue with schema versioning for the AA update
+
+## [22.26.1] - 09/17/2023
+
+### Fixes
+
+* Add Validation to #find, #set, and #show args ([#3598](https://github.com/EQEmu/Server/pull/3598)) @Kinglykrab 2023-09-17
+
+## [22.26.0] - 09/17/2023
+
+### Bug
+
+* Additional Wild Ramp param was causing an overflow it appears at least on local testing. ([#3589](https://github.com/EQEmu/Server/pull/3589)) @fryguy503 2023-09-11
+
+### Commands
+
+* npc_edit faction and #setfaction duplicate and incorrect. ([#3577](https://github.com/EQEmu/Server/pull/3577)) @noudess 2023-09-03
+
+### Feature
+
+* Add Defensive Proc Rules for Level Gap Penalty ([#3580](https://github.com/EQEmu/Server/pull/3580)) @Kinglykrab 2023-09-17
+* Add opcodes for Cast and Scribe book buttons ([#3578](https://github.com/EQEmu/Server/pull/3578)) @hgtw 2023-09-17
+* Cleanup Group Split Money Logic ([#3583](https://github.com/EQEmu/Server/pull/3583)) @crdunwel 2023-09-17
+
+### Fixes
+
+* AA System Fixes ([#3572](https://github.com/EQEmu/Server/pull/3572)) @KimLS 2023-09-17
+* Check for Song Skill Increase on Bard Pulse ([#3586](https://github.com/EQEmu/Server/pull/3586)) @crdunwel 2023-09-17
+* Fix #set faction/#setfaction Command ([#3575](https://github.com/EQEmu/Server/pull/3575)) @Kinglykrab 2023-09-02
+* Fix Killed XYZH support in EVENT_DEATH in Perl. ([#3591](https://github.com/EQEmu/Server/pull/3591)) @Kinglykrab 2023-09-17
+* Fix entity ID on death with #castspell ([#3592](https://github.com/EQEmu/Server/pull/3592)) @Kinglykrab 2023-09-17
+* Zoning logging edge case safety  @Akkadius 2023-08-29
+
+### Logging
+
+* Add logging to track down Wild Ramp issue ([#3588](https://github.com/EQEmu/Server/pull/3588)) @fryguy503 2023-09-11
+
+### Messages
+
+* Remove duplicate message on tracking begin ([#3574](https://github.com/EQEmu/Server/pull/3574)) @noudess 2023-09-11
+* Swarm pet normal damage messages were missing ([#3594](https://github.com/EQEmu/Server/pull/3594)) @noudess 2023-09-17
+
+### Quest API
+
+* Add ApplySpell() and SetBuffDuration() overloads to Perl/Lua ([#3576](https://github.com/EQEmu/Server/pull/3576)) @Kinglykrab 2023-09-17
+* Add XYZ/XYZH Overloads to Cross Zone Move Methods ([#3581](https://github.com/EQEmu/Server/pull/3581)) @Kinglykrab 2023-09-17
+
+## [22.25.0] - 08/28/2023
+
+### Bots
+
+* Add Support for Mana Pool AA Bonuses. ([#3571](https://github.com/EQEmu/Server/pull/3571)) @Aeadoin 2023-08-21
+* Remove In-Game Command References to Bot Groups ([#3545](https://github.com/EQEmu/Server/pull/3545)) @Aeadoin 2023-08-13
+
+### Bug
+
+* Escape item name in trader audit. ([#3540](https://github.com/EQEmu/Server/pull/3540)) @fryguy503 2023-08-13
+
+### CMake
+
+* Add cmake option to re-enable MSVC warnings ([#3537](https://github.com/EQEmu/Server/pull/3537)) @hgtw 2023-08-13
+* Define perlbind option for all targets ([#3538](https://github.com/EQEmu/Server/pull/3538)) @hgtw 2023-08-13
+
+### Charm
+
+* Fix to update target windows on charm on/off ([#3549](https://github.com/EQEmu/Server/pull/3549)) @noudess 2023-08-20
+
+### Combat Messages
+
+* Fix issue where pet proc damage was not showing up ([#3551](https://github.com/EQEmu/Server/pull/3551)) @noudess 2023-08-24
+
+### Database
+
+* Change primary key entry to NOT NULL ([#3559](https://github.com/EQEmu/Server/pull/3559)) @joligario 2023-08-20
+* Extend dumper CLI utility to export static instance data ([#3562](https://github.com/EQEmu/Server/pull/3562)) @Akkadius 2023-08-20
+
+### Expansions
+
+* Expansion settings tweaks ([#3556](https://github.com/EQEmu/Server/pull/3556)) @Akkadius 2023-08-20
+
+### Feature
+
+* Add adjustability for AERampage Range. ([#3548](https://github.com/EQEmu/Server/pull/3548)) @fryguy503 2023-08-20
+* Change #reload zone to reload zone headers globally. ([#3557](https://github.com/EQEmu/Server/pull/3557)) @Kinglykrab 2023-08-20
+* Change money type to all lower case as EQ live money shares and split has it that way. ([#3550](https://github.com/EQEmu/Server/pull/3550)) @regneq 2023-08-18
+
+### Fixes
+
+* Add character_stats_record to player tables  @Akkadius 2023-08-11
+* Bots no longer drop group on death, and raid fixes. ([#3542](https://github.com/EQEmu/Server/pull/3542)) @Aeadoin 2023-08-13
+* Bots will now load AAs properly when spawned. ([#3544](https://github.com/EQEmu/Server/pull/3544)) @Aeadoin 2023-08-13
+* Clearing target window on CHARM wear off had a side effect ([#3570](https://github.com/EQEmu/Server/pull/3570)) @noudess 2023-08-21
+* Fix #motd/#set motd Command ([#3558](https://github.com/EQEmu/Server/pull/3558)) @Kinglykrab 2023-08-20
+* Fix Bot::CheckDataBucket to work with Owner Buckets. ([#3552](https://github.com/EQEmu/Server/pull/3552)) @Aeadoin 2023-08-18
+* Fix to zoning logging exception  @Akkadius 2023-08-29
+* Instance GetUnusedInstanceID crash fox  @Akkadius 2023-08-21
+* Minor adjustment to formula calc position to fix modifier bug. ([#3565](https://github.com/EQEmu/Server/pull/3565)) @Valorith 2023-08-21
+
+### Instances
+
+* Honor reserved instances ([#3563](https://github.com/EQEmu/Server/pull/3563)) @joligario 2023-08-21
+* Refine id selection ([#3568](https://github.com/EQEmu/Server/pull/3568)) @joligario 2023-08-29
+
+### Logging
+
+* Add detailed zoning logging ([#3555](https://github.com/EQEmu/Server/pull/3555)) @Akkadius 2023-08-20
+
+### Quest API
+
+* Reload content flags globally when a content flag is set ([#3564](https://github.com/EQEmu/Server/pull/3564)) @Akkadius 2023-08-21
+
+### Rules
+
+* Add FinalRaidExpMultiplier Rule ([#3554](https://github.com/EQEmu/Server/pull/3554)) @Valorith 2023-08-20
+* Add a rule to adjust the randomization range for Wizard\Caster Merc innate critical ratio. ([#3543](https://github.com/EQEmu/Server/pull/3543)) @catapultam-habeo 2023-08-13
+* Add rule to restrict hand in of quest items to quest flagged NPCs. ([#3536](https://github.com/EQEmu/Server/pull/3536)) @Valorith 2023-08-13
+* Add rule to toggle pets accepting quest items ([#3533](https://github.com/EQEmu/Server/pull/3533)) @Valorith 2023-08-06
+* Also reload rules in world when #reload rules invoked ([#3566](https://github.com/EQEmu/Server/pull/3566)) @Akkadius 2023-08-21
+* Correct explanation of Bots:ManaRegen ([#3569](https://github.com/EQEmu/Server/pull/3569)) @Aeadoin 2023-08-21
+
+### Spawns
+
+* Fixes a rarer issue where spawn2 is not being properly content filtered  @Akkadius 2023-08-13
+
+## [22.24.0] - 08/05/2023
+
+### Character
+
+* Record character stats to `character_stats_record` table ([#3522](https://github.com/EQEmu/Server/pull/3522)) @Akkadius 2023-08-05
+
+### Code
+
+* Cleanup #view zone_loot Command ([#3523](https://github.com/EQEmu/Server/pull/3523)) @Kinglykrab 2023-08-02
+* Remove Strings::Commify from all identifier values ([#3528](https://github.com/EQEmu/Server/pull/3528)) @Kinglykrab 2023-08-02
+
+### Feature
+
+* Add Support for Drakkin Heritage Illusions ([#3521](https://github.com/EQEmu/Server/pull/3521)) @Kinglykrab 2023-08-01
+
+### Fixes
+
+* Bug fix for raid mark NPC across zones ([#3525](https://github.com/EQEmu/Server/pull/3525)) @neckkola 2023-08-05
+* Bugs table should not target content database ([#3535](https://github.com/EQEmu/Server/pull/3535)) @Akkadius 2023-08-05
+* Fix +/- 0.1 XYZ Door Manipulation ([#3527](https://github.com/EQEmu/Server/pull/3527)) @Kinglykrab 2023-08-02
+* Fix issue with mob scanning when trying to use EVENT_SPAWN ([#3529](https://github.com/EQEmu/Server/pull/3529)) @Akkadius 2023-08-04
+
+### Quest API
+
+* Adjust GetCloseMobList calls internally ([#3530](https://github.com/EQEmu/Server/pull/3530)) @Akkadius 2023-08-04
+
+## [22.23.0] - 07/31/2023
+
+### Databuckets
+
+* Improvements to distributed cache, reload commands ([#3519](https://github.com/EQEmu/Server/pull/3519)) @Akkadius 2023-08-01
+
+### Fixes
+
+* Fix #gm top level alias for #set gm ([#3517](https://github.com/EQEmu/Server/pull/3517)) @Kinglykrab 2023-07-30
+* Fix Appearance Issues ([#3520](https://github.com/EQEmu/Server/pull/3520)) @Kinglykrab 2023-08-01
+* Fix NPC Cast Events not parsing properly. ([#3518](https://github.com/EQEmu/Server/pull/3518)) @Kinglykrab 2023-08-01
+
+### Scaling/Bug Fix
+
+* Scaling where min and max damage was bugged ([#3514](https://github.com/EQEmu/Server/pull/3514)) @noudess 2023-08-01
+
+## [22.22.1] - 07/30/2023
+
+### Database
+
+* Hotfix: Add command_subsettings to server tables  @Akkadius 2023-07-29
+
+### Doors
+
+* Add door blacklist ([#3516](https://github.com/EQEmu/Server/pull/3516)) @Akkadius 2023-07-30
+
+## [22.22.0] - 07/27/2023
+
+### Code
+
+* Fix casing in corpse money and decay time. ([#3511](https://github.com/EQEmu/Server/pull/3511)) @Kinglykrab 2023-07-23
+
+### Crash Fix
+
+* Guard against Spells:MaxTotalSlotsPET being set above client allowed maximum. ([#3507](https://github.com/EQEmu/Server/pull/3507)) @Valorith 2023-07-22
+
+### Data Buckets
+
+* Distributed Databucket Caching ([#3500](https://github.com/EQEmu/Server/pull/3500)) @Kinglykrab 2023-07-24
+
+### Database
+
+* Fix console output in database:dump --dump-output-to-console  @Akkadius 2023-07-25
+
+### Fixes
+
+* Escape search string in #find item ([#3510](https://github.com/EQEmu/Server/pull/3510)) @Kinglykrab 2023-07-22
+
+### Quest API
+
+* Add GetMobTypeIdentifier() to Perl/Lua ([#3512](https://github.com/EQEmu/Server/pull/3512)) @Kinglykrab 2023-07-28
+
+### Saylink
+
+* Fix cases where saylinks were not being cached ([#3508](https://github.com/EQEmu/Server/pull/3508)) @Akkadius 2023-07-20
+
+## [22.21.2] - 07/19/2023
+
+### Databuckets
+
+* Fix rarer same bucket name scoping overlap issue ([#3509](https://github.com/EQEmu/Server/pull/3509)) @Akkadius 2023-07-20
+
+## [22.21.1] - 07/18/2023
+
+### Bug
+
+* Show Petition and Show Petition_Info fix ([#3503](https://github.com/EQEmu/Server/pull/3503)) @fryguy503 2023-07-18
+
+### Code
+
+* Remove arbitrary teleport blocking in Tutorial and Load zones ([#3506](https://github.com/EQEmu/Server/pull/3506)) @Valorith 2023-07-19
+
+### Databuckets
+
+* Fix issue with expired databuckets not being expired and returned properly ([#3504](https://github.com/EQEmu/Server/pull/3504)) @Akkadius 2023-07-18
+
+### Fixes
+
+* #set title_suffix Argument Position ([#3505](https://github.com/EQEmu/Server/pull/3505)) @Kinglykrab 2023-07-18
+
 ## [22.21.0] - 07/18/2023
 
 ### Data Buckets

CMakeLists.txt

@@ -23,7 +23,12 @@ IF (EQEMU_BUILD_STATIC)
 	SET(CMAKE_FIND_LIBRARY_SUFFIXES ".lib" ".a")
 	MESSAGE(STATUS "Building with static linking")
 	SET(CMAKE_EXE_LINKER_FLAGS "-static-libgcc -static-libstdc++")
-ENDIF(EQEMU_BUILD_STATIC)
+	IF (UNIX)
+		SET(PERL_LIBRARY "/opt/eqemu-perl/lib/5.32.1/x86_64-linux-thread-multi/CORE/libperl.so")
+		SET(PERL_INCLUDE_PATH "/opt/eqemu-perl/lib/5.32.1/x86_64-linux-thread-multi/CORE/")
+		SET(PERL_EXECUTABLE "/opt/eqemu-perl/bin/perl")
+	ENDIF ()
+ENDIF (EQEMU_BUILD_STATIC)
 
 IF(MSVC)
 	ADD_DEFINITIONS(-D_CRT_SECURE_NO_WARNINGS)
@@ -32,7 +37,11 @@ IF(MSVC)
 	ADD_DEFINITIONS(-D_HAS_AUTO_PTR_ETC) # for Luabind on C++17
 
 	SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /MP")
-	ADD_DEFINITIONS( "/W0 /D_CRT_SECURE_NO_WARNINGS /wd4005 /wd4996 /nologo /Os")
+
+	OPTION(EQEMU_DISABLE_MSVC_WARNINGS "Disable MSVC compile warnings." ON)
+	IF(EQEMU_DISABLE_MSVC_WARNINGS)
+		ADD_DEFINITIONS( "/W0 /D_CRT_SECURE_NO_WARNINGS /wd4005 /wd4996 /nologo /Os")
+	ENDIF(EQEMU_DISABLE_MSVC_WARNINGS)
 ELSE(MSVC)
 	ADD_DEFINITIONS(-DHAS_UNION_SEMUN)
 ENDIF(MSVC)
@@ -129,6 +138,13 @@ ELSE()
 	MESSAGE(STATUS "* mbedTLS:                               MISSING *")
 ENDIF()
 
+MESSAGE(STATUS "PERL_INCLUDE_PATH: ${PERL_INCLUDE_PATH}")
+MESSAGE(STATUS "PERL_LIBRARY: ${PERL_LIBRARY}")
+MESSAGE(STATUS "PERL_INCLUDE_DIR: ${PERL_INCLUDE_DIR}")
+MESSAGE(STATUS "PERL_INCLUDE_DIRS: ${PERL_INCLUDE_DIRS}")
+MESSAGE(STATUS "PERL_LIBRARIES: ${PERL_LIBRARIES}")
+MESSAGE(STATUS "PERL_VERSION: ${PERL_VERSION}")
+
 MESSAGE(STATUS "**************************************************")
 
 #options
@@ -384,6 +400,10 @@ IF(PERL_LIBRARY_ENABLED)
 		INCLUDE_DIRECTORIES(SYSTEM "${PERL_LIBRARY_INCLUDE}")
 		ADD_DEFINITIONS(-DEMBPERL)
 		ADD_DEFINITIONS(-DEMBPERL_PLUGIN)
+		ADD_DEFINITIONS(-DPERLBIND_NO_STRICT_SCALAR_TYPES)
+		IF (UNIX AND EQEMU_BUILD_STATIC)
+			SET(SERVER_LIBS ${SERVER_LIBS} libcrypt.a)
+		ENDIF ()
 	ENDIF()
 ENDIF()
 

common/CMakeLists.txt

@@ -70,6 +70,7 @@ SET(common_sources
     perl_eqdb.cpp
     perl_eqdb_res.cpp
     process/process.cpp
+    process.cpp
     proc_launcher.cpp
     profanity_manager.cpp
     ptimer.cpp
@@ -90,6 +91,7 @@ SET(common_sources
     timer.cpp
     unix.cpp
     platform.cpp
+    json/json.hpp
     json/jsoncpp.cpp
     zone_store.cpp
     net/console_server.cpp
@@ -583,12 +585,14 @@ SET(common_headers
     path_manager.cpp
     platform.h
     process/process.h
+    process.h
     proc_launcher.h
     profanity_manager.h
     profiler.h
     ptimer.h
     queue.h
     races.h
+	raid.h
     random.h
     rdtsc.h
     rulesys.h

common/cli/eqemu_command_handler.cpp

@@ -39,15 +39,15 @@ namespace EQEmuCommand {
 	{
 		if (cmd[{"-d", "--debug"}]) {
 			std::cout << "Positional args:\n";
-			for (auto &pos_arg : cmd.pos_args())
+			for (auto &pos_arg: cmd.pos_args())
 				std::cout << '\t' << pos_arg << std::endl;
 
 			std::cout << "\nFlags:\n";
-			for (auto &flag : cmd.flags())
+			for (auto &flag: cmd.flags())
 				std::cout << '\t' << flag << std::endl;
 
 			std::cout << "\nParameters:\n";
-			for (auto &param : cmd.params())
+			for (auto &param: cmd.params())
 				std::cout << '\t' << param.first << " : " << param.second << std::endl;
 		}
 	}
@@ -69,8 +69,8 @@ namespace EQEmuCommand {
 	{
 		bool arguments_filled = true;
 
-		int index = 2;
-		for (auto &arg : arguments) {
+		int       index = 2;
+		for (auto &arg: arguments) {
 			if (cmd(arg).str().empty() && cmd(index).str().empty()) {
 				arguments_filled = false;
 			}
@@ -79,12 +79,12 @@ namespace EQEmuCommand {
 
 		if (!arguments_filled || (argc == 2 && !cmd[{"-h", "--help"}]) || (argc == 3 && cmd[{"-h", "--help"}])) {
 			std::string arguments_string;
-			for (auto &arg : arguments) {
+			for (auto   &arg: arguments) {
 				arguments_string += " " + arg;
 			}
 
 			std::string options_string;
-			for (auto   &opt : options) {
+			for (auto   &opt: options) {
 				options_string += "  " + opt + "\n";
 			}
 
@@ -124,14 +124,6 @@ namespace EQEmuCommand {
 	)
 	{
 		std::string description;
-		bool        ran_command = false;
-		for (auto   &it: in_function_map) {
-			if (it.first == argv[1]) {
-				(it.second)(argc, argv, cmd, description);
-				ran_command = true;
-			}
-		}
-
 		if (cmd[{"-h", "--help"}]) {
 			std::cout << std::endl;
 			std::cout <<
@@ -142,9 +134,7 @@ namespace EQEmuCommand {
 					  << std::endl
 					  << std::endl;
 
-			/**
-			 * Get max command length for padding length
-			 */
+			// Get max command length for padding length
 			int max_command_length = 0;
 
 			for (auto &it: in_function_map) {
@@ -155,18 +145,14 @@ namespace EQEmuCommand {
 				}
 			}
 
-			/**
-			 * Display command menu
-			 */
+			// Display command menu
 			std::string command_section;
 			for (auto   &it: in_function_map) {
 				description.clear();
 
 				(it.second)(argc, argv, cmd, description);
 
-				/**
-				 * Print section header
-				 */
+				// Print section header
 				std::string command_prefix = it.first.substr(0, it.first.find(":"));
 
 				if (command_prefix.find("test") != std::string::npos) {
@@ -178,9 +164,7 @@ namespace EQEmuCommand {
 					std::cout << termcolor::reset << command_prefix << std::endl;
 				}
 
-				/**
-				 * Print commands
-				 */
+				// Print commands
 				std::stringstream command;
 				command << termcolor::colorize << termcolor::yellow << it.first << termcolor::reset;
 				printf("  %-*s %s\n", max_command_length, command.str().c_str(), description.c_str());
@@ -191,6 +175,15 @@ namespace EQEmuCommand {
 			std::exit(0);
 		}
 
+		bool ran_command = false;
+
+		for (auto &it: in_function_map) {
+			if (it.first == argv[1]) {
+				(it.second)(argc, argv, cmd, description);
+				ran_command = true;
+			}
+		}
+
 		if (ran_command) {
 			std::exit(0);
 		}

common/database/database_dump_service.cpp

@@ -279,6 +279,11 @@ void DatabaseDumpService::DatabaseDump()
 		}
 	}
 
+	if (IsDumpStaticInstanceData()) {
+		tables_to_dump += "instance_list";
+		options += " --no-create-info --where=\"instance_list.is_global > 0 and instance_list.never_expires > 0\"";
+	}
+
 	if (!dump_descriptor.empty()) {
 		SetDumpFileName(GetDumpFileName() + dump_descriptor);
 	}
@@ -324,7 +329,9 @@ void DatabaseDumpService::DatabaseDump()
 		}
 	}
 
-	LogSys.LoadLogSettingsDefaults();
+	if (!IsDumpOutputToConsole()) {
+		LogSys.LoadLogSettingsDefaults();
+	}
 
 	if (!pipe_file.empty()) {
 		std::string file = fmt::format("{}.sql", GetDumpFileNameWithPath());
@@ -604,3 +611,13 @@ void DatabaseDumpService::RemoveCredentialsFile()
 		std::filesystem::remove(CREDENTIALS_FILE);
 	}
 }
+
+bool DatabaseDumpService::IsDumpStaticInstanceData()
+{
+	return dump_static_instance_data;
+}
+
+void DatabaseDumpService::SetDumpStaticInstanceData(bool b)
+{
+	dump_static_instance_data = b;
+}

common/database/database_dump_service.h

@@ -58,6 +58,9 @@ public:
 	bool IsDumpMercTables() const;
 	void SetDumpMercTables(bool dump_bot_tables);
 
+	void SetDumpStaticInstanceData(bool b);
+	bool IsDumpStaticInstanceData();
+
 private:
 	bool        dump_all_tables             = false;
 	bool        dump_state_tables           = false;
@@ -73,6 +76,8 @@ private:
 	bool        dump_drop_table_syntax_only = false;
 	bool        dump_bot_tables             = false;
 	bool        dump_merc_tables            = false;
+	bool        dump_static_instance_data   = false;
+
 	std::string dump_path;
 	std::string dump_file_name;
 

common/database/database_update_manifest.cpp

@@ -4825,6 +4825,176 @@ UPDATE data_buckets SET character_id = SUBSTRING_INDEX(SUBSTRING_INDEX( `key`, '
 UPDATE data_buckets SET npc_id = SUBSTRING_INDEX(SUBSTRING_INDEX( `key`, '-', 2 ), '-', -1), `key` = SUBSTR(SUBSTRING_INDEX(`key`, SUBSTRING_INDEX( `key`, '-', 2 ), -1), 2) WHERE `key` LIKE 'npc-%';
 UPDATE data_buckets SET bot_id = SUBSTRING_INDEX(SUBSTRING_INDEX( `key`, '-', 2 ), '-', -1), `key` = SUBSTR(SUBSTRING_INDEX(`key`, SUBSTRING_INDEX( `key`, '-', 2 ), -1), 2) WHERE `key` LIKE 'bot-%';
 
+)"
+	},
+	ManifestEntry{
+		.version = 9234,
+		.description = "2023_07_27_update_raid_details.sql",
+		.check = "SHOW COLUMNS FROM `raid_details` LIKE 'marked_npc_1_entity_id';",
+		.condition = "empty",
+		.match = "",
+		.sql = R"(ALTER TABLE `raid_details`
+			CHANGE COLUMN `marked_npc_1` `marked_npc_1_entity_id` INT UNSIGNED NOT NULL DEFAULT '0' AFTER `motd`,
+			ADD COLUMN `marked_npc_1_zone_id` INT UNSIGNED NOT NULL DEFAULT '0' AFTER `marked_npc_1_entity_id`,
+			ADD COLUMN `marked_npc_1_instance_id` INT UNSIGNED NOT NULL DEFAULT '0' AFTER `marked_npc_1_zone_id`,
+			CHANGE COLUMN `marked_npc_2` `marked_npc_2_entity_id` INT UNSIGNED NOT NULL DEFAULT '0' AFTER `marked_npc_1_instance_id`,
+			ADD COLUMN `marked_npc_2_zone_id` INT UNSIGNED NOT NULL DEFAULT '0' AFTER `marked_npc_2_entity_id`,
+			ADD COLUMN `marked_npc_2_instance_id` INT UNSIGNED NOT NULL DEFAULT '0' AFTER `marked_npc_2_zone_id`,
+			CHANGE COLUMN `marked_npc_3` `marked_npc_3_entity_id` INT UNSIGNED NOT NULL DEFAULT '0' AFTER `marked_npc_2_instance_id`,
+			ADD COLUMN `marked_npc_3_zone_id` INT UNSIGNED NOT NULL DEFAULT '0' AFTER `marked_npc_3_entity_id`,
+			ADD COLUMN `marked_npc_3_instance_id` INT UNSIGNED NOT NULL DEFAULT '0' AFTER `marked_npc_3_zone_id`;
+		)"
+    },
+	ManifestEntry{
+		.version = 9235,
+		.description = "2023_07_31_character_stats_record.sql",
+		.check = "SHOW TABLES LIKE 'character_stats_record'",
+		.condition = "empty",
+		.match = "",
+		.sql = R"(
+
+CREATE TABLE `character_stats_record`  (
+  `character_id` int NOT NULL,
+  `name` varchar(100) NULL,
+  `status` int NULL DEFAULT 0,
+  `level` int NULL DEFAULT 0,
+  `class` int NULL DEFAULT 0,
+  `race` int NULL DEFAULT 0,
+  `aa_points` int NULL DEFAULT 0,
+  `hp` bigint NULL DEFAULT 0,
+  `mana` bigint NULL DEFAULT 0,
+  `endurance` bigint NULL DEFAULT 0,
+  `ac` int NULL DEFAULT 0,
+  `strength` int NULL DEFAULT 0,
+  `stamina` int NULL DEFAULT 0,
+  `dexterity` int NULL DEFAULT 0,
+  `agility` int NULL DEFAULT 0,
+  `intelligence` int NULL DEFAULT 0,
+  `wisdom` int NULL DEFAULT 0,
+  `charisma` int NULL DEFAULT 0,
+  `magic_resist` int NULL DEFAULT 0,
+  `fire_resist` int NULL DEFAULT 0,
+  `cold_resist` int NULL DEFAULT 0,
+  `poison_resist` int NULL DEFAULT 0,
+  `disease_resist` int NULL DEFAULT 0,
+  `corruption_resist` int NULL DEFAULT 0,
+  `heroic_strength` int NULL DEFAULT 0,
+  `heroic_stamina` int NULL DEFAULT 0,
+  `heroic_dexterity` int NULL DEFAULT 0,
+  `heroic_agility` int NULL DEFAULT 0,
+  `heroic_intelligence` int NULL DEFAULT 0,
+  `heroic_wisdom` int NULL DEFAULT 0,
+  `heroic_charisma` int NULL DEFAULT 0,
+  `heroic_magic_resist` int NULL DEFAULT 0,
+  `heroic_fire_resist` int NULL DEFAULT 0,
+  `heroic_cold_resist` int NULL DEFAULT 0,
+  `heroic_poison_resist` int NULL DEFAULT 0,
+  `heroic_disease_resist` int NULL DEFAULT 0,
+  `heroic_corruption_resist` int NULL DEFAULT 0,
+  `haste` int NULL DEFAULT 0,
+  `accuracy` int NULL DEFAULT 0,
+  `attack` int NULL DEFAULT 0,
+  `avoidance` int NULL DEFAULT 0,
+  `clairvoyance` int NULL DEFAULT 0,
+  `combat_effects` int NULL DEFAULT 0,
+  `damage_shield_mitigation` int NULL DEFAULT 0,
+  `damage_shield` int NULL DEFAULT 0,
+  `dot_shielding` int NULL DEFAULT 0,
+  `hp_regen` int NULL DEFAULT 0,
+  `mana_regen` int NULL DEFAULT 0,
+  `endurance_regen` int NULL DEFAULT 0,
+  `shielding` int NULL DEFAULT 0,
+  `spell_damage` int NULL DEFAULT 0,
+  `spell_shielding` int NULL DEFAULT 0,
+  `strikethrough` int NULL DEFAULT 0,
+  `stun_resist` int NULL DEFAULT 0,
+  `backstab` int NULL DEFAULT 0,
+  `wind` int NULL DEFAULT 0,
+  `brass` int NULL DEFAULT 0,
+  `string` int NULL DEFAULT 0,
+  `percussion` int NULL DEFAULT 0,
+  `singing` int NULL DEFAULT 0,
+  `baking` int NULL DEFAULT 0,
+  `alchemy` int NULL DEFAULT 0,
+  `tailoring` int NULL DEFAULT 0,
+  `blacksmithing` int NULL DEFAULT 0,
+  `fletching` int NULL DEFAULT 0,
+  `brewing` int NULL DEFAULT 0,
+  `jewelry` int NULL DEFAULT 0,
+  `pottery` int NULL DEFAULT 0,
+  `research` int NULL DEFAULT 0,
+  `alcohol` int NULL DEFAULT 0,
+  `fishing` int NULL DEFAULT 0,
+  `tinkering` int NULL DEFAULT 0,
+  `created_at` datetime DEFAULT NULL,
+  `updated_at` datetime DEFAULT NULL,
+  PRIMARY KEY (`character_id`)
+);
+)"
+	},
+	ManifestEntry{
+		.version = 9236,
+		.description = "2023_08_24_aa_ability_auto_grant.sql",
+		.check = "SHOW COLUMNS FROM `aa_ability` LIKE 'auto_grant_enabled';",
+		.condition = "empty",
+		.match = "",
+		.sql = R"(
+ALTER TABLE `aa_ability` ADD COLUMN `auto_grant_enabled` TINYINT(4) NOT NULL DEFAULT '0' AFTER `reset_on_death`;
+UPDATE `aa_ability` SET `auto_grant_enabled` = 1 WHERE `grant_only` = 0 AND `charges` = 0 AND `category` = -1;
+)"
+	},
+	ManifestEntry{
+		.version = 9237,
+		.description = "2023_10_15_import_13th_floor.sql",
+		.check = "SHOW COLUMNS FROM `items` LIKE 'bardeffect';",
+		.condition = "contains",
+		.match = "mediumint",
+		.sql = R"(
+ALTER TABLE `items`
+ MODIFY COLUMN `scriptfileid`        MEDIUMINT(6)     NOT NULL DEFAULT 0,
+ MODIFY COLUMN `powersourcecapacity` MEDIUMINT(7)     NOT NULL DEFAULT 0,
+ MODIFY COLUMN `augdistiller`        INT(11) UNSIGNED NOT NULL DEFAULT 0,
+ MODIFY COLUMN `scrollunk1`          INT(11) UNSIGNED NOT NULL DEFAULT 0,
+ MODIFY COLUMN `bardeffect`          MEDIUMINT(6)     NOT NULL DEFAULT 0;
+)"
+	},
+	ManifestEntry{
+		.version = 9238,
+		.description = "2023_10_18_tradeskill_add_learned_by_item_id.sql",
+		.check = "SHOW COLUMNS FROM `tradeskill_recipe` LIKE 'learned_by_item_id'",
+		.condition = "empty",
+		.match = "",
+		.sql = R"(
+ALTER TABLE `tradeskill_recipe`
+	ADD COLUMN `learned_by_item_id` int(11) NOT NULL DEFAULT 0 AFTER `must_learn`;
+)"
+	},
+	ManifestEntry{
+		.version = 9239,
+		.description = "2023_10_18_blocked_spells_expansions_content_flags.sql",
+		.check = "SHOW COLUMNS FROM `blocked_spells` LIKE 'min_expansion'",
+		.condition = "empty",
+		.match = "",
+		.sql = R"(
+ALTER TABLE `blocked_spells`
+ADD COLUMN `min_expansion` tinyint(4) NOT NULL DEFAULT -1 AFTER `description`,
+ADD COLUMN `max_expansion` tinyint(4) NOT NULL DEFAULT -1 AFTER `min_expansion`,
+ADD COLUMN `content_flags` varchar(100) CHARACTER SET latin1 COLLATE latin1_swedish_ci NULL DEFAULT NULL AFTER `max_expansion`,
+ADD COLUMN `content_flags_disabled` varchar(100) CHARACTER SET latin1 COLLATE latin1_swedish_ci NULL DEFAULT NULL AFTER `content_flags`;
+)"
+	},
+	ManifestEntry{
+		.version = 9240,
+		.description = "2023_10_29_variables_id.sql",
+		.check = "SHOW COLUMNS FROM `variables` LIKE 'id'",
+		.condition = "empty",
+		.match = "",
+		.sql = R"(
+ALTER TABLE `variables`
+ADD COLUMN `id` int(11) NOT NULL AUTO_INCREMENT FIRST,
+DROP PRIMARY KEY,
+ADD PRIMARY KEY (`id`) USING BTREE,
+ADD UNIQUE INDEX(`varname`);
 )"
 	},
 

common/database/database_update_manifest_bots.cpp

@@ -61,6 +61,29 @@ DROP TABLE IF EXISTS `bot_group_members`;
 SET FOREIGN_KEY_CHECKS = 1;
 )",
 	},
+	ManifestEntry{
+		.version = 9040,
+		.description = "2023_11_16_bot_starting_items.sql",
+		.check = "SHOW TABLES LIKE 'bot_starting_items'",
+		.condition = "empty",
+		.match = "",
+		.sql = R"(
+CREATE TABLE `bot_starting_items`  (
+`id` int(11) UNSIGNED NOT NULL AUTO_INCREMENT,
+`races` int(11) UNSIGNED NOT NULL DEFAULT 0,
+`classes` int(11) UNSIGNED NOT NULL DEFAULT 0,
+`item_id` int(11) UNSIGNED NOT NULL DEFAULT 0,
+`item_charges` tinyint(3) UNSIGNED NOT NULL DEFAULT 1,
+`min_status` tinyint(3) UNSIGNED NOT NULL DEFAULT 0,
+`slot_id` mediumint(9) NOT NULL DEFAULT -1,
+`min_expansion` tinyint(4) NOT NULL DEFAULT -1,
+`max_expansion` tinyint(4) NOT NULL DEFAULT -1,
+`content_flags` varchar(100) CHARACTER SET latin1 COLLATE latin1_swedish_ci NULL DEFAULT NULL,
+`content_flags_disabled` varchar(100) CHARACTER SET latin1 COLLATE latin1_swedish_ci NULL DEFAULT NULL,
+PRIMARY KEY (`id`)
+) ENGINE = InnoDB CHARACTER SET = latin1 COLLATE = latin1_swedish_ci;
+)"
+	}
 // -- template; copy/paste this when you need to create a new entry
 //	ManifestEntry{
 //		.version = 9228,

common/database_instances.cpp

@@ -131,85 +131,118 @@ bool Database::CreateInstance(uint16 instance_id, uint32 zone_id, uint32 version
 bool Database::GetUnusedInstanceID(uint16 &instance_id)
 {
 	uint32 max_reserved_instance_id = RuleI(Instances, ReservedInstances);
-	uint32 max                      = 32000;
+	uint32 max_instance_id          = 32000;
 
+	// sanity check reserved
+	if (max_reserved_instance_id >= max_instance_id) {
+		instance_id = 0;
+		return false;
+	}
+
+	// recycle instances
+	if (RuleB(Instances, RecycleInstanceIds)) {
+
+		//query to get first unused id above reserved
+		auto query = fmt::format(
+			SQL(
+				SELECT id
+				FROM instance_list
+				WHERE id = {};
+			),
+			max_reserved_instance_id + 1
+		);
+
+		auto results = QueryDatabase(query);
+
+		// could not successfully query - bail out
+		if (!results.Success()) {
+			instance_id = 0;
+			return false;
+		}
+
+		// first id is available
+		if (results.RowCount() == 0) {
+			instance_id = max_reserved_instance_id + 1;
+			return true;
+		}
+
+		// now look for next available above reserved
+		query = fmt::format(
+			SQL(
+				SELECT MIN(i.id + 1) AS next_available
+				FROM instance_list i
+				LEFT JOIN instance_list i2 ON i.id + 1 = i2.id
+				WHERE i.id >= {}
+				AND i2.id IS NULL;
+			),
+			max_reserved_instance_id
+		);
+
+		results = QueryDatabase(query);
+
+		// could not successfully query - bail out
+		if (!results.Success()) {
+			instance_id = 0;
+			return false;
+		}
+
+		// did not retrieve any rows - bail out
+		if (results.RowCount() == 0) {
+			instance_id = 0;
+			return false;
+		}
+
+		auto row = results.begin();
+
+		// check that id is within limits
+		if (row[0] && Strings::ToInt(row[0]) <= max_instance_id) {
+			instance_id = Strings::ToInt(row[0]);
+			return true;
+		}
+
+		// no available instance ids
+		instance_id = 0;
+		return false;
+	}
+
+	// get max unused id above reserved
 	auto query = fmt::format(
 		"SELECT IFNULL(MAX(id), {}) + 1 FROM instance_list WHERE id > {}",
 		max_reserved_instance_id,
 		max_reserved_instance_id
 	);
 
-	if (RuleB(Instances, RecycleInstanceIds)) {
-		query = (
-			SQL(
-				SELECT i.id + 1 AS next_available
-				FROM instance_list i
-				LEFT JOIN instance_list i2 ON i2.id = i.id + 1
-				WHERE i2.id IS NULL
-				ORDER BY i.id
-				LIMIT 0, 1;
-
-			)
-		);
-	}
-
 	auto results = QueryDatabase(query);
 
+	// could not successfully query - bail out
 	if (!results.Success()) {
 		instance_id = 0;
 		return false;
 	}
 
+	// did not retrieve any rows - bail out
 	if (results.RowCount() == 0) {
-		instance_id = max_reserved_instance_id;
-		return true;
+		instance_id = 0;
+		return false;
 	}
 
 	auto row = results.begin();
 
-	if (Strings::ToInt(row[0]) <= max) {
+	// no instances currently used
+	if (!row[0]) {
+		instance_id = max_reserved_instance_id + 1;
+		return true;
+	}
+
+	// check that id is within limits
+	if (Strings::ToInt(row[0]) <= max_instance_id) {
 		instance_id = Strings::ToInt(row[0]);
-
 		return true;
 	}
 
-	if (instance_id < max_reserved_instance_id) {
-		instance_id = max_reserved_instance_id;
-		return true;
-	}
-
-	query   = fmt::format("SELECT id FROM instance_list where id > {} ORDER BY id", max_reserved_instance_id);
-	results = QueryDatabase(query);
-
-	if (!results.Success()) {
-		instance_id = 0;
-		return false;
-	}
-
-	if (results.RowCount() == 0) {
-		instance_id = 0;
-		return false;
-	}
-
-	max_reserved_instance_id++;
-
-	for (auto row : results) {
-		if (max_reserved_instance_id < Strings::ToUnsignedInt(row[0])) {
-			instance_id = max_reserved_instance_id;
-			return true;
-		}
-
-		if (max_reserved_instance_id > max) {
-			instance_id = 0;
-			return false;
-		}
-
-		max_reserved_instance_id++;
-	}
-
-	instance_id = max_reserved_instance_id;
-
-	return true;
+	// no available instance ids
+	instance_id = 0;
+	return false;
 }
 
 bool Database::IsGlobalInstance(uint16 instance_id)

common/database_schema.h

@@ -66,6 +66,7 @@ namespace DatabaseSchema {
 			{"character_potionbelt",           "id"},
 			{"character_skills",               "id"},
 			{"character_spells",               "id"},
+			{"character_stats_record",         "character_id"},
 			{"character_task_timers",          "character_id"},
 			{"character_tasks",                "charid"},
 			{"character_tribute",              "character_id"},
@@ -134,6 +135,7 @@ namespace DatabaseSchema {
 			"character_potionbelt",
 			"character_skills",
 			"character_spells",
+			"character_stats_record",
 			"character_task_timers",
 			"character_tasks",
 			"character_tribute",
@@ -258,6 +260,7 @@ namespace DatabaseSchema {
 			"chatchannels",
 			"chatchannel_reserved_names",
 			"command_settings",
+			"command_subsettings",
 			"content_flags",
 			"db_str",
 			"eqtime",

common/emu_oplist.h

@@ -62,6 +62,7 @@ N(OP_BeginCast),
 N(OP_Bind_Wound),
 N(OP_BlockedBuffs),
 N(OP_BoardBoat),
+N(OP_BookButton),
 N(OP_Buff),
 N(OP_BuffCreate),
 N(OP_BuffRemoveRequest),

common/eq_packet_structs.h

@@ -2559,7 +2559,10 @@ struct GMEmoteZone_Struct {
 struct BookText_Struct {
 	uint8 window;	// where to display the text (0xFF means new window)
 	uint8 type;		//type: 0=scroll, 1=book, 2=item info.. prolly others.
-	uint32 invslot;	// Only used in SoF and later clients.
+	int16 invslot;  // Only used in SoF and later clients.
+	int32 target_id;
+	int8 can_cast;
+	int8 can_scribe;
 	char booktext[1]; // Variable Length
 };
 // This is the request to read a book.
@@ -2568,11 +2571,18 @@ struct BookText_Struct {
 struct BookRequest_Struct {
 	uint8 window;	// where to display the text (0xFF means new window)
 	uint8 type;		//type: 0=scroll, 1=book, 2=item info.. prolly others.
-	uint32 invslot;	// Only used in Sof and later clients;
-	int16 subslot; // The subslot inside of a bag if it is inside one.
+	int16 invslot;  // Only used in Sof and later clients;
+	int32 target_id;
 	char txtfile[20];
 };
 
+// used by Scribe and CastSpell book buttons
+struct BookButton_Struct
+{
+	int16 invslot;   // server slot
+	int32 target_id;
+};
+
 /*
 ** Object/Ground Spawn struct
 ** Used for Forges, Ovens, ground spawns, items dropped to ground, etc
@@ -4154,7 +4164,6 @@ struct RaidGeneral_Struct {
 /*68*/	uint32		unknown1;
 /*72*/	char		leader_name[64];
 /*136*/	uint32		parameter;
-/*200*/	char		note[64];
 };
 
 struct RaidAddMember_Struct {
@@ -4165,9 +4174,14 @@ struct RaidAddMember_Struct {
 /*139*/	uint8 flags[5]; //no idea if these are needed...
 };
 
+struct RaidNote_Struct {
+/*000*/ RaidGeneral_Struct general;
+/*140*/ char note[64];
+};
+
 struct RaidMOTD_Struct {
-/*000*/ RaidGeneral_Struct general; // leader_name and action only used
-/*136*/ char motd[0]; // max size is 1024, but reply is variable
+/*000*/ RaidGeneral_Struct general;
+/*140*/ char motd[1024];
 };
 
 struct RaidLeadershipUpdate_Struct {

common/eqemu_logsys.h

@@ -137,6 +137,8 @@ namespace Logs {
 		Bugs,
 		QuestErrors,
 		PlayerEvents,
+		DataBuckets,
+		Zoning,
 		MaxCategoryID /* Don't Remove this */
 	};
 
@@ -233,6 +235,8 @@ namespace Logs {
 		"Bugs",
 		"QuestErrors",
 		"PlayerEvents",
+		"DataBuckets",
+		"Zoning",
 	};
 }
 

common/eqemu_logsys_log_aliases.h

@@ -794,6 +794,26 @@
         OutF(LogSys, Logs::Detail, Logs::PlayerEvents, __FILE__, __func__, __LINE__, message, ##__VA_ARGS__);\
 } while (0)
 
+#define LogDataBuckets(message, ...) do {\
+    if (LogSys.IsLogEnabled(Logs::General, Logs::DataBuckets))\
+        OutF(LogSys, Logs::General, Logs::DataBuckets, __FILE__, __func__, __LINE__, message, ##__VA_ARGS__);\
+} while (0)
+
+#define LogDataBucketsDetail(message, ...) do {\
+    if (LogSys.IsLogEnabled(Logs::Detail, Logs::DataBuckets))\
+        OutF(LogSys, Logs::Detail, Logs::DataBuckets, __FILE__, __func__, __LINE__, message, ##__VA_ARGS__);\
+} while (0)
+
+#define LogZoning(message, ...) do {\
+    if (LogSys.IsLogEnabled(Logs::General, Logs::Zoning))\
+        OutF(LogSys, Logs::General, Logs::Zoning, __FILE__, __func__, __LINE__, message, ##__VA_ARGS__);\
+} while (0)
+
+#define LogZoningDetail(message, ...) do {\
+    if (LogSys.IsLogEnabled(Logs::Detail, Logs::Zoning))\
+        OutF(LogSys, Logs::Detail, Logs::Zoning, __FILE__, __func__, __LINE__, message, ##__VA_ARGS__);\
+} while (0)
+
 #define Log(debug_level, log_category, message, ...) do {\
     if (LogSys.IsLogEnabled(debug_level, log_category))\
         LogSys.Out(debug_level, log_category, __FILE__, __func__, __LINE__, message, ##__VA_ARGS__);\

common/features.h

@@ -250,9 +250,7 @@ enum {
 	commandMovecharSelfOnly       = 80,  //below this == only self move allowed
 	commandMovecharToSpecials     = 200, //ability to send people to cshom/load zones
 	commandCastSpecials           = 100, //can cast special spells
-	commandInstacast              = 100, //insta-cast all #casted spells
 	commandDoAnimOthers           = 100, //can #doanim on others
-	commandLockZones              = 101, //can lock or unlock zones
 	commandEditPlayerCorpses      = 150, //can Edit Player Corpses
 	commandInterrogateInv         = 100, //below this == only log on error state and self-only target dump
 	commandInvSnapshot            = 150  //ability to clear/restore snapshots

common/file.cpp

@@ -55,8 +55,14 @@ bool File::Exists(const std::string &name)
  */
 void File::Makedir(const std::string &directory_name)
 {
-	fs::create_directory(directory_name);
-	fs::permissions(directory_name, fs::perms::owner_all);
+	try {
+		fs::create_directory(directory_name);
+		fs::permissions(directory_name, fs::perms::owner_all);
+	}
+	catch (const fs::filesystem_error &ex) {
+		std::cout << "Failed to create directory: " << directory_name << std::endl;
+		std::cout << ex.what() << std::endl;
+	}
 }
 
 std::string File::FindEqemuConfigPath()

common/item_data.cpp

@@ -230,6 +230,10 @@ bool EQ::ItemData::IsTypeShield() const
 	return (ItemType == item::ItemTypeShield);
 }
 
+bool EQ::ItemData::IsQuestItem() const {
+	return QuestItemFlag;
+}
+
 bool EQ::ItemData::CheckLoreConflict(const ItemData* l_item, const ItemData* r_item)
 {
 	if (!l_item || !r_item)

common/item_data.h

@@ -546,6 +546,7 @@ namespace EQ
 		bool IsType1HWeapon() const;
 		bool IsType2HWeapon() const;
 		bool IsTypeShield() const;
+		bool IsQuestItem() const;
 
 		static bool CheckLoreConflict(const ItemData* l_item, const ItemData* r_item);
 		bool CheckLoreConflict(const ItemData* item) const { return CheckLoreConflict(this, item); }

common/patches/rof.cpp

@@ -34,6 +34,7 @@
 #include "../rulesys.h"
 #include "../path_manager.h"
 #include "../races.h"
+#include "../raid.h"
 
 #include <iostream>
 #include <sstream>
@@ -2608,88 +2609,124 @@ namespace RoF
 
 	ENCODE(OP_RaidJoin)
 	{
-		EQApplicationPacket *inapp = *p;
-		unsigned char * __emu_buffer = inapp->pBuffer;
-		RaidCreate_Struct *raid_create = (RaidCreate_Struct*)__emu_buffer;
+		EQApplicationPacket* inapp = *p;
+		*p = nullptr;
+		unsigned char* __emu_buffer = inapp->pBuffer;
+		RaidCreate_Struct* emu = (RaidCreate_Struct*)__emu_buffer;
 
-		auto outapp_create = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
-		structs::RaidGeneral_Struct *general = (structs::RaidGeneral_Struct*)outapp_create->pBuffer;
+		auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
+		structs::RaidGeneral_Struct* general = (structs::RaidGeneral_Struct*)outapp->pBuffer;
 
-		general->action = 8;
-		general->parameter = 1;
-		strn0cpy(general->leader_name, raid_create->leader_name, 64);
-		strn0cpy(general->player_name, raid_create->leader_name, 64);
+		general->action = raidCreate;
+		general->parameter = RaidCommandAcceptInvite;
+		strn0cpy(general->leader_name, emu->leader_name, sizeof(emu->leader_name));
+		strn0cpy(general->player_name, emu->leader_name, sizeof(emu->leader_name));
+		
+		dest->FastQueuePacket(&outapp);
 
-		dest->FastQueuePacket(&outapp_create);
 		safe_delete(inapp);
+
 	}
 
 	ENCODE(OP_RaidUpdate)
 	{
-		EQApplicationPacket *inapp = *p;
+		EQApplicationPacket* inapp = *p;
 		*p = nullptr;
-		unsigned char * __emu_buffer = inapp->pBuffer;
-		RaidGeneral_Struct *raid_gen = (RaidGeneral_Struct*)__emu_buffer;
+		unsigned char* __emu_buffer = inapp->pBuffer;
+		RaidGeneral_Struct* raid_gen = (RaidGeneral_Struct*)__emu_buffer;
 
-		if (raid_gen->action == 0) // raid add has longer length than other raid updates
+		switch (raid_gen->action)
 		{
-			RaidAddMember_Struct* in_add_member = (RaidAddMember_Struct*)__emu_buffer;
+		case raidAdd:
+		{
+			RaidAddMember_Struct* emu = (RaidAddMember_Struct*)__emu_buffer;
 
 			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidAddMember_Struct));
-			structs::RaidAddMember_Struct *add_member = (structs::RaidAddMember_Struct*)outapp->pBuffer;
+			structs::RaidAddMember_Struct* eq = (structs::RaidAddMember_Struct*)outapp->pBuffer;
 
-			add_member->raidGen.action = in_add_member->raidGen.action;
-			add_member->raidGen.parameter = in_add_member->raidGen.parameter;
-			strn0cpy(add_member->raidGen.leader_name, in_add_member->raidGen.leader_name, 64);
-			strn0cpy(add_member->raidGen.player_name, in_add_member->raidGen.player_name, 64);
-			add_member->_class = in_add_member->_class;
-			add_member->level = in_add_member->level;
-			add_member->isGroupLeader = in_add_member->isGroupLeader;
-			add_member->flags[0] = in_add_member->flags[0];
-			add_member->flags[1] = in_add_member->flags[1];
-			add_member->flags[2] = in_add_member->flags[2];
-			add_member->flags[3] = in_add_member->flags[3];
-			add_member->flags[4] = in_add_member->flags[4];
-			dest->FastQueuePacket(&outapp);
-		}
-		else if (raid_gen->action == 35)
-		{
-			RaidMOTD_Struct *inmotd = (RaidMOTD_Struct *)__emu_buffer;
-			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidMOTD_Struct) +
-										 strlen(inmotd->motd) + 1);
-			structs::RaidMOTD_Struct *outmotd = (structs::RaidMOTD_Struct *)outapp->pBuffer;
+			OUT(raidGen.action);
+			OUT(raidGen.parameter);
+			OUT_str(raidGen.leader_name);
+			OUT_str(raidGen.player_name);
+			OUT(_class);
+			OUT(level);
+			OUT(isGroupLeader);
+			OUT(flags[0]);
+			OUT(flags[1]);
+			OUT(flags[2]);
+			OUT(flags[3]);
+			OUT(flags[4]);
 
-			outmotd->general.action = inmotd->general.action;
-			strn0cpy(outmotd->general.player_name, inmotd->general.player_name, 64);
-			strn0cpy(outmotd->motd, inmotd->motd, strlen(inmotd->motd) + 1);
 			dest->FastQueuePacket(&outapp);
+			break;
 		}
-		else if (raid_gen->action == 14 || raid_gen->action == 30)
+		case raidSetMotd:
 		{
-			RaidLeadershipUpdate_Struct *inlaa = (RaidLeadershipUpdate_Struct *)__emu_buffer;
-			auto outapp =
-			    new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidLeadershipUpdate_Struct));
-			structs::RaidLeadershipUpdate_Struct *outlaa = (structs::RaidLeadershipUpdate_Struct *)outapp->pBuffer;
+			RaidMOTD_Struct* emu = (RaidMOTD_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidMOTD_Struct));
+			structs::RaidMOTD_Struct* eq = (structs::RaidMOTD_Struct*)outapp->pBuffer;
+
+			OUT(general.action);
+			OUT_str(general.player_name);
+			OUT_str(general.leader_name);
+			OUT_str(motd);
 
-			outlaa->action = inlaa->action;
-			strn0cpy(outlaa->player_name, inlaa->player_name, 64);
-			strn0cpy(outlaa->leader_name, inlaa->leader_name, 64);
-			memcpy(&outlaa->raid, &inlaa->raid, sizeof(RaidLeadershipAA_Struct));
 			dest->FastQueuePacket(&outapp);
+			break;
 		}
-		else
+		case raidSetLeaderAbilities:
+		case raidMakeLeader:
 		{
-			RaidGeneral_Struct* in_raid_general = (RaidGeneral_Struct*)__emu_buffer;
+			RaidLeadershipUpdate_Struct* emu = (RaidLeadershipUpdate_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidLeadershipUpdate_Struct));
+			structs::RaidLeadershipUpdate_Struct* eq = (structs::RaidLeadershipUpdate_Struct*)outapp->pBuffer;
+
+			OUT(action);
+			OUT_str(player_name);
+			OUT_str(leader_name);
+			memcpy(&eq->raid, &emu->raid, sizeof(RaidLeadershipAA_Struct));
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		case raidSetNote:
+		{
+			auto emu = (RaidNote_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidNote_Struct));
+			auto eq = (structs::RaidNote_Struct*)outapp->pBuffer;
+
+			OUT(general.action);
+			OUT_str(general.leader_name);
+			OUT_str(general.player_name);
+			OUT_str(note);
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		case raidNoRaid:
+		{
+			dest->QueuePacket(inapp);
+			break;
+		}
+		default:
+		{
+			RaidGeneral_Struct* emu = (RaidGeneral_Struct*)__emu_buffer;
 
 			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
-			structs::RaidGeneral_Struct *raid_general = (structs::RaidGeneral_Struct*)outapp->pBuffer;
-			strn0cpy(raid_general->leader_name, in_raid_general->leader_name, 64);
-			strn0cpy(raid_general->player_name, in_raid_general->player_name, 64);
-			raid_general->action = in_raid_general->action;
-			raid_general->parameter = in_raid_general->parameter;
-			dest->FastQueuePacket(&outapp);
-		}
+			structs::RaidGeneral_Struct* eq = (structs::RaidGeneral_Struct*)outapp->pBuffer;
 
+			OUT(action);
+			OUT(parameter);
+			OUT_str(leader_name);
+			OUT_str(player_name);
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		}
 		safe_delete(inapp);
 	}
 
@@ -4861,37 +4898,47 @@ namespace RoF
 	{
 		DECODE_LENGTH_ATLEAST(structs::RaidGeneral_Struct);
 
-		// This is a switch on the RaidGeneral action
-		switch (*(uint32 *)__packet->pBuffer) {
-			case 35: { // raidMOTD
-				// we don't have a nice macro for this
-				structs::RaidMOTD_Struct *__eq_buffer = (structs::RaidMOTD_Struct *)__packet->pBuffer;
-				__eq_buffer->motd[1023] = '\0';
-				size_t motd_size = strlen(__eq_buffer->motd) + 1;
-				__packet->size = sizeof(RaidMOTD_Struct) + motd_size;
-				__packet->pBuffer = new unsigned char[__packet->size];
-				RaidMOTD_Struct *emu = (RaidMOTD_Struct *)__packet->pBuffer;
-				structs::RaidMOTD_Struct *eq = (structs::RaidMOTD_Struct *)__eq_buffer;
-				strn0cpy(emu->general.player_name, eq->general.player_name, 64);
-				strn0cpy(emu->motd, eq->motd, motd_size);
-				IN(general.action);
-				IN(general.parameter);
-				FINISH_DIRECT_DECODE();
-				break;
-			 }
-			case 36: { // raidPlayerNote unhandled
-				break;
-			}
-			default: {
-				DECODE_LENGTH_EXACT(structs::RaidGeneral_Struct);
-				SETUP_DIRECT_DECODE(RaidGeneral_Struct, structs::RaidGeneral_Struct);
-				strn0cpy(emu->leader_name, eq->leader_name, 64);
-				strn0cpy(emu->player_name, eq->player_name, 64);
-				IN(action);
-				IN(parameter);
-				FINISH_DIRECT_DECODE();
-				break;
-			}
+		RaidGeneral_Struct* rgs = (RaidGeneral_Struct*)__packet->pBuffer;
+
+		switch (rgs->action)
+		{
+		case raidSetMotd:
+		{
+			SETUP_VAR_DECODE(RaidMOTD_Struct, structs::RaidMOTD_Struct, motd);
+
+			IN(general.action);
+			IN(general.parameter);
+			IN_str(general.leader_name);
+			IN_str(general.player_name);
+			IN_str(motd);
+
+			FINISH_VAR_DECODE();
+			break;
+		}
+		case raidSetNote:
+		{
+			SETUP_VAR_DECODE(RaidNote_Struct, structs::RaidNote_Struct, note);
+
+			IN(general.action);
+			IN(general.parameter);
+			IN_str(general.leader_name);
+			IN_str(general.player_name);
+			IN_str(note);
+
+			FINISH_VAR_DECODE();
+			break;
+		}
+		default:
+		{
+			SETUP_DIRECT_DECODE(RaidGeneral_Struct, structs::RaidGeneral_Struct);
+			IN(action);
+			IN(parameter);
+			IN_str(leader_name);
+			IN_str(player_name);
+
+			FINISH_DIRECT_DECODE();
+			break;
+		}
 		}
 	}
 

common/patches/rof2.cpp

@@ -35,7 +35,7 @@
 #include "../path_manager.h"
 #include "../classes.h"
 #include "../races.h"
-#include "../../zone/raids.h"
+#include "../raid.h"
 
 #include <iostream>
 #include <sstream>
@@ -2678,100 +2678,124 @@ namespace RoF2
 
 	ENCODE(OP_RaidJoin)
 	{
-		EQApplicationPacket *inapp = *p;
-		unsigned char * __emu_buffer = inapp->pBuffer;
-		RaidCreate_Struct *raid_create = (RaidCreate_Struct*)__emu_buffer;
+		EQApplicationPacket* inapp = *p;
+		*p = nullptr;
+		unsigned char* __emu_buffer = inapp->pBuffer;
+		RaidCreate_Struct* emu = (RaidCreate_Struct*)__emu_buffer;
 
-		auto outapp_create = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
-		structs::RaidGeneral_Struct *general = (structs::RaidGeneral_Struct*)outapp_create->pBuffer;
+		auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
+		structs::RaidGeneral_Struct* general = (structs::RaidGeneral_Struct*)outapp->pBuffer;
 
-		general->action = 8;
-		general->parameter = 1;
-		strn0cpy(general->leader_name, raid_create->leader_name, 64);
-		strn0cpy(general->player_name, raid_create->leader_name, 64);
+		general->action = raidCreate;
+		general->parameter = RaidCommandAcceptInvite;
+		strn0cpy(general->leader_name, emu->leader_name, sizeof(emu->leader_name));
+		strn0cpy(general->player_name, emu->leader_name, sizeof(emu->leader_name));
+
+		dest->FastQueuePacket(&outapp);
 
-		dest->FastQueuePacket(&outapp_create);
 		safe_delete(inapp);
+
 	}
 
 	ENCODE(OP_RaidUpdate)
 	{
-		EQApplicationPacket *inapp = *p;
+		EQApplicationPacket* inapp = *p;
 		*p = nullptr;
-		unsigned char * __emu_buffer = inapp->pBuffer;
-		RaidGeneral_Struct *raid_gen = (RaidGeneral_Struct*)__emu_buffer;
+		unsigned char* __emu_buffer = inapp->pBuffer;
+		RaidGeneral_Struct* raid_gen = (RaidGeneral_Struct*)__emu_buffer;
 
-		if (raid_gen->action == 0) // raid add has longer length than other raid updates
+		switch (raid_gen->action)
 		{
-			RaidAddMember_Struct* in_add_member = (RaidAddMember_Struct*)__emu_buffer;
+		case raidAdd: 
+		{
+			RaidAddMember_Struct* emu = (RaidAddMember_Struct*)__emu_buffer;
 
 			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidAddMember_Struct));
-			structs::RaidAddMember_Struct *add_member = (structs::RaidAddMember_Struct*)outapp->pBuffer;
+			structs::RaidAddMember_Struct* eq = (structs::RaidAddMember_Struct*)outapp->pBuffer;
 
-			add_member->raidGen.action = in_add_member->raidGen.action;
-			add_member->raidGen.parameter = in_add_member->raidGen.parameter;
-			strn0cpy(add_member->raidGen.leader_name, in_add_member->raidGen.leader_name, 64);
-			strn0cpy(add_member->raidGen.player_name, in_add_member->raidGen.player_name, 64);
-			add_member->_class = in_add_member->_class;
-			add_member->level = in_add_member->level;
-			add_member->isGroupLeader = in_add_member->isGroupLeader;
-			add_member->flags[0] = in_add_member->flags[0];
-			add_member->flags[1] = in_add_member->flags[1];
-			add_member->flags[2] = in_add_member->flags[2];
-			add_member->flags[3] = in_add_member->flags[3];
-			add_member->flags[4] = in_add_member->flags[4];
-			dest->FastQueuePacket(&outapp);
-		}
-		else if (raid_gen->action == 35)
-		{
-			RaidMOTD_Struct *inmotd = (RaidMOTD_Struct *)__emu_buffer;
-			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidMOTD_Struct) +
-										 strlen(inmotd->motd) + 1);
-			structs::RaidMOTD_Struct *outmotd = (structs::RaidMOTD_Struct *)outapp->pBuffer;
+			OUT(raidGen.action);
+			OUT(raidGen.parameter);
+			OUT_str(raidGen.leader_name);
+			OUT_str(raidGen.player_name);
+			OUT(_class);
+			OUT(level);
+			OUT(isGroupLeader);
+			OUT(flags[0]);
+			OUT(flags[1]);
+			OUT(flags[2]);
+			OUT(flags[3]);
+			OUT(flags[4]);
 
-			outmotd->general.action = inmotd->general.action;
-			strn0cpy(outmotd->general.player_name, inmotd->general.player_name, 64);
-			strn0cpy(outmotd->motd, inmotd->motd, strlen(inmotd->motd) + 1);
 			dest->FastQueuePacket(&outapp);
+			break;
 		}
-		else if (raid_gen->action == 14 || raid_gen->action == 30)
+		case raidSetMotd:
 		{
-			RaidLeadershipUpdate_Struct *inlaa = (RaidLeadershipUpdate_Struct *)__emu_buffer;
-			auto outapp =
-				new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidLeadershipUpdate_Struct));
-			structs::RaidLeadershipUpdate_Struct *outlaa = (structs::RaidLeadershipUpdate_Struct *)outapp->pBuffer;
+			RaidMOTD_Struct* emu = (RaidMOTD_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidMOTD_Struct));
+			structs::RaidMOTD_Struct* eq = (structs::RaidMOTD_Struct*)outapp->pBuffer;
+
+			OUT(general.action);
+			OUT_str(general.player_name);
+			OUT_str(general.leader_name);
+			OUT_str(motd);
 
-			outlaa->action = inlaa->action;
-			strn0cpy(outlaa->player_name, inlaa->player_name, 64);
-			strn0cpy(outlaa->leader_name, inlaa->leader_name, 64);
-			memcpy(&outlaa->raid, &inlaa->raid, sizeof(RaidLeadershipAA_Struct));
 			dest->FastQueuePacket(&outapp);
+			break;
 		}
-		else if (raid_gen->action == raidSetNote)
+		case raidSetLeaderAbilities:
+		case raidMakeLeader:
 		{
-			auto in_note = (RaidGeneral_Struct*)__emu_buffer;
-			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(RaidGeneral_Struct));
-			auto note = (RaidGeneral_Struct*)outapp->pBuffer;
-			note->action = raidSetNote;
-			strn0cpy(note->leader_name, in_note->leader_name, sizeof(note->leader_name));
-			strn0cpy(note->player_name, in_note->player_name, sizeof(note->leader_name));
-			strn0cpy(note->note, in_note->note, sizeof(note->note));
-			dest->QueuePacket(outapp);
-			safe_delete(outapp);
+			RaidLeadershipUpdate_Struct* emu = (RaidLeadershipUpdate_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidLeadershipUpdate_Struct));
+			structs::RaidLeadershipUpdate_Struct* eq = (structs::RaidLeadershipUpdate_Struct*)outapp->pBuffer;
+
+			OUT(action);
+			OUT_str(player_name);
+			OUT_str(leader_name);
+			memcpy(&eq->raid, &emu->raid, sizeof(RaidLeadershipAA_Struct));
+
+			dest->FastQueuePacket(&outapp);
+			break;
 		}
-		else
+		case raidSetNote:
 		{
-			RaidGeneral_Struct* in_raid_general = (RaidGeneral_Struct*)__emu_buffer;
+			auto emu = (RaidNote_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidNote_Struct));
+			auto eq = (structs::RaidNote_Struct*)outapp->pBuffer;
+
+			OUT(general.action);
+			OUT_str(general.leader_name);
+			OUT_str(general.player_name);
+			OUT_str(note);
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		case raidNoRaid:
+		{
+			dest->QueuePacket(inapp);
+			break;
+		}
+		default:
+		{
+			RaidGeneral_Struct* emu = (RaidGeneral_Struct*)__emu_buffer;
 
 			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
-			structs::RaidGeneral_Struct *raid_general = (structs::RaidGeneral_Struct*)outapp->pBuffer;
-			strn0cpy(raid_general->leader_name, in_raid_general->leader_name, 64);
-			strn0cpy(raid_general->player_name, in_raid_general->player_name, 64);
-			raid_general->action = in_raid_general->action;
-			raid_general->parameter = in_raid_general->parameter;
-			dest->FastQueuePacket(&outapp);
-		}
+			structs::RaidGeneral_Struct* eq = (structs::RaidGeneral_Struct*)outapp->pBuffer;
 
+			OUT(action);
+			OUT(parameter);
+			OUT_str(leader_name);
+			OUT_str(player_name);
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		}
 		safe_delete(inapp);
 	}
 
@@ -2785,7 +2809,10 @@ namespace RoF2
 		else
 			eq->window = emu->window;
 		OUT(type);
-		OUT(invslot);
+		eq->invslot = ServerToRoF2TypelessSlot(emu->invslot, invtype::typePossessions);
+		OUT(target_id);
+		OUT(can_cast);
+		OUT(can_scribe);
 		strn0cpy(eq->txtfile, emu->booktext, sizeof(eq->txtfile));
 
 		FINISH_ENCODE();
@@ -4411,6 +4438,17 @@ namespace RoF2
 		FINISH_DIRECT_DECODE();
 	}
 
+	DECODE(OP_BookButton)
+	{
+		DECODE_LENGTH_EXACT(structs::BookButton_Struct);
+		SETUP_DIRECT_DECODE(BookButton_Struct, structs::BookButton_Struct);
+
+		emu->invslot = static_cast<int16_t>(RoF2ToServerTypelessSlot(eq->slot, invtype::typePossessions));
+		IN(target_id);
+
+		FINISH_DIRECT_DECODE();
+	}
+
 	DECODE(OP_Buff)
 	{
 		DECODE_LENGTH_EXACT(structs::SpellBuffPacket_Struct);
@@ -5077,37 +5115,47 @@ namespace RoF2
 	{
 		DECODE_LENGTH_ATLEAST(structs::RaidGeneral_Struct);
 
-		// This is a switch on the RaidGeneral action
-		switch (*(uint32 *)__packet->pBuffer) {
-			case 35: { // raidMOTD
-				// we don't have a nice macro for this
-				structs::RaidMOTD_Struct *__eq_buffer = (structs::RaidMOTD_Struct *)__packet->pBuffer;
-				__eq_buffer->motd[1023] = '\0';
-				size_t motd_size = strlen(__eq_buffer->motd) + 1;
-				__packet->size = sizeof(RaidMOTD_Struct) + motd_size;
-				__packet->pBuffer = new unsigned char[__packet->size];
-				RaidMOTD_Struct *emu = (RaidMOTD_Struct *)__packet->pBuffer;
-				structs::RaidMOTD_Struct *eq = (structs::RaidMOTD_Struct *)__eq_buffer;
-				strn0cpy(emu->general.player_name, eq->general.player_name, 64);
-				strn0cpy(emu->motd, eq->motd, motd_size);
-				IN(general.action);
-				IN(general.parameter);
-				FINISH_DIRECT_DECODE();
-				break;
-			 }
-			case 36: { // raidPlayerNote unhandled
-				break;
-			}
-			default: {
-				DECODE_LENGTH_EXACT(structs::RaidGeneral_Struct);
-				SETUP_DIRECT_DECODE(RaidGeneral_Struct, structs::RaidGeneral_Struct);
-				strn0cpy(emu->leader_name, eq->leader_name, 64);
-				strn0cpy(emu->player_name, eq->player_name, 64);
-				IN(action);
-				IN(parameter);
-				FINISH_DIRECT_DECODE();
-				break;
-			}
+		RaidGeneral_Struct* rgs = (RaidGeneral_Struct*)__packet->pBuffer;
+
+		switch (rgs->action)
+		{
+		case raidSetMotd:
+		{
+			SETUP_VAR_DECODE(RaidMOTD_Struct, structs::RaidMOTD_Struct, motd);
+
+			IN(general.action);
+			IN(general.parameter);
+			IN_str(general.leader_name);
+			IN_str(general.player_name);
+			IN_str(motd);
+
+			FINISH_VAR_DECODE();
+			break;
+		}
+		case raidSetNote:
+		{
+			SETUP_VAR_DECODE(RaidNote_Struct, structs::RaidNote_Struct, note);
+
+			IN(general.action);
+			IN(general.parameter);
+			IN_str(general.leader_name);
+			IN_str(general.player_name);
+			IN_str(note);
+
+			FINISH_VAR_DECODE();
+			break;
+		}
+		default:
+		{
+			SETUP_DIRECT_DECODE(RaidGeneral_Struct, structs::RaidGeneral_Struct);
+			IN(action);
+			IN(parameter);
+			IN_str(leader_name);
+			IN_str(player_name);
+
+			FINISH_DIRECT_DECODE();
+			break;
+		}
 		}
 	}
 
@@ -5117,8 +5165,8 @@ namespace RoF2
 		SETUP_DIRECT_DECODE(BookRequest_Struct, structs::BookRequest_Struct);
 
 		IN(type);
-		IN(invslot);
-		IN(subslot);
+		emu->invslot = static_cast<int16_t>(RoF2ToServerTypelessSlot(eq->invslot, invtype::typePossessions));
+		IN(target_id);
 		emu->window = (uint8)eq->window;
 		strn0cpy(emu->txtfile, eq->txtfile, sizeof(emu->txtfile));
 

common/patches/rof2_ops.h

@@ -150,6 +150,7 @@ D(OP_AugmentInfo)
 D(OP_AugmentItem)
 D(OP_BazaarSearch)
 D(OP_BlockedBuffs)
+D(OP_BookButton)
 D(OP_Buff)
 D(OP_BuffRemoveRequest)
 D(OP_CastSpell)

common/patches/rof2_structs.h

@@ -2868,15 +2868,23 @@ struct BookText_Struct {
 // This is just a "text file" on the server
 // or in our case, the 'name' column in our books table.
 struct BookRequest_Struct {
-/*0000*/	uint32 window;		// where to display the text (0xFFFFFFFF means new window).
-/*0004*/	uint16 invslot;		// Is the slot, but the RoF2 conversion causes it to fail.  Turned to 0 since it isnt required anyway.
-/*0006*/	int16 subslot;		// Inventory sub-slot (0-x)
-/*0008*/	uint16 unknown006;	// Seen FFFF
-/*0010*/	uint16 unknown008;	// seen 0000
-/*0012*/	uint32 type;		// 0 = Scroll, 1 = Book, 2 = Item Info. Possibly others
-/*0016*/	uint32 unknown0012;
-/*0020*/	uint16 unknown0016;
-/*0022*/	char txtfile[8194];
+/*0000*/ uint32 window;         // where to display the text (0xFFFFFFFF means new window).
+/*0004*/ TypelessInventorySlot_Struct invslot; // book ItemIndex (with int16_t alignment padding)
+/*0012*/ uint32 type;           // 0 = Scroll, 1 = Book, 2 = Item Info. Possibly others
+/*0016*/ uint32 target_id;      // client's target when using the book
+/*0020*/ uint8 can_cast;        // show Cast Spell button in book window
+/*0021*/ uint8 can_scribe;      // show Scribe button in book window
+/*0022*/ char txtfile[8194];
+/*8216*/
+};
+
+// used by Scribe and CastSpell book buttons
+struct BookButton_Struct
+{
+/*0000*/ TypelessInventorySlot_Struct slot; // book ItemIndex (with int16_t alignment padding)
+/*0008*/ int32 target_id; // client's target when using the book button
+/*0012*/ int32 unused;    // always 0 from button packets
+/*0016*/
 };
 
 /*
@@ -4190,9 +4198,14 @@ struct RaidAddMember_Struct {
 /*139*/	uint8 flags[5]; //no idea if these are needed...
 };
 
+struct RaidNote_Struct {
+/*000*/ RaidGeneral_Struct general;
+/*140*/ char note[64];
+};
+
 struct RaidMOTD_Struct {
-/*000*/ RaidGeneral_Struct general; // leader_name and action only used
-/*140*/ char motd[0]; // max size 1024, but reply is variable
+/*000*/ RaidGeneral_Struct general;
+/*140*/ char motd[1024];
 };
 
 struct RaidLeadershipUpdate_Struct {

common/patches/rof_structs.h

@@ -4136,9 +4136,14 @@ struct RaidAddMember_Struct {
 /*139*/	uint8 flags[5]; //no idea if these are needed...
 };
 
+struct RaidNote_Struct {
+/*000*/ RaidGeneral_Struct general;
+/*140*/ char note[64];
+};
+
 struct RaidMOTD_Struct {
 /*000*/ RaidGeneral_Struct general; // leader_name and action only used
-/*140*/ char motd[0]; // max size 1024, but reply is variable
+/*140*/ char motd[1024]; // max size is 1024, but reply is variable
 };
 
 struct RaidLeadershipUpdate_Struct {

common/patches/sod.cpp

@@ -34,6 +34,7 @@
 #include "../rulesys.h"
 #include "../path_manager.h"
 #include "../races.h"
+#include "../raid.h"
 
 #include <iostream>
 #include <sstream>
@@ -1686,88 +1687,124 @@ namespace SoD
 
 	ENCODE(OP_RaidJoin)
 	{
-		EQApplicationPacket *inapp = *p;
-		unsigned char * __emu_buffer = inapp->pBuffer;
-		RaidCreate_Struct *raid_create = (RaidCreate_Struct*)__emu_buffer;
+		EQApplicationPacket* inapp = *p;
+		*p = nullptr;
+		unsigned char* __emu_buffer = inapp->pBuffer;
+		RaidCreate_Struct* emu = (RaidCreate_Struct*)__emu_buffer;
 
-		auto outapp_create = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
-		structs::RaidGeneral_Struct *general = (structs::RaidGeneral_Struct*)outapp_create->pBuffer;
+		auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
+		structs::RaidGeneral_Struct* general = (structs::RaidGeneral_Struct*)outapp->pBuffer;
 
-		general->action = 8;
-		general->parameter = 1;
-		strn0cpy(general->leader_name, raid_create->leader_name, 64);
-		strn0cpy(general->player_name, raid_create->leader_name, 64);
+		general->action = raidCreate;
+		general->parameter = RaidCommandAcceptInvite;
+		strn0cpy(general->leader_name, emu->leader_name, sizeof(emu->leader_name));
+		strn0cpy(general->player_name, emu->leader_name, sizeof(emu->leader_name));
+
+		dest->FastQueuePacket(&outapp);
 
-		dest->FastQueuePacket(&outapp_create);
 		safe_delete(inapp);
+
 	}
 
 	ENCODE(OP_RaidUpdate)
 	{
-		EQApplicationPacket *inapp = *p;
+		EQApplicationPacket* inapp = *p;
 		*p = nullptr;
-		unsigned char * __emu_buffer = inapp->pBuffer;
-		RaidGeneral_Struct *raid_gen = (RaidGeneral_Struct*)__emu_buffer;
+		unsigned char* __emu_buffer = inapp->pBuffer;
+		RaidGeneral_Struct* raid_gen = (RaidGeneral_Struct*)__emu_buffer;
 
-		if (raid_gen->action == 0) // raid add has longer length than other raid updates
+		switch (raid_gen->action)
 		{
-			RaidAddMember_Struct* in_add_member = (RaidAddMember_Struct*)__emu_buffer;
+		case raidAdd:
+		{
+			RaidAddMember_Struct* emu = (RaidAddMember_Struct*)__emu_buffer;
 
 			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidAddMember_Struct));
-			structs::RaidAddMember_Struct *add_member = (structs::RaidAddMember_Struct*)outapp->pBuffer;
+			structs::RaidAddMember_Struct* eq = (structs::RaidAddMember_Struct*)outapp->pBuffer;
 
-			add_member->raidGen.action = in_add_member->raidGen.action;
-			add_member->raidGen.parameter = in_add_member->raidGen.parameter;
-			strn0cpy(add_member->raidGen.leader_name, in_add_member->raidGen.leader_name, 64);
-			strn0cpy(add_member->raidGen.player_name, in_add_member->raidGen.player_name, 64);
-			add_member->_class = in_add_member->_class;
-			add_member->level = in_add_member->level;
-			add_member->isGroupLeader = in_add_member->isGroupLeader;
-			add_member->flags[0] = in_add_member->flags[0];
-			add_member->flags[1] = in_add_member->flags[1];
-			add_member->flags[2] = in_add_member->flags[2];
-			add_member->flags[3] = in_add_member->flags[3];
-			add_member->flags[4] = in_add_member->flags[4];
-			dest->FastQueuePacket(&outapp);
-		}
-		else if (raid_gen->action == 35)
-		{
-			RaidMOTD_Struct *inmotd = (RaidMOTD_Struct *)__emu_buffer;
-			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidMOTD_Struct) +
-										 strlen(inmotd->motd) + 1);
-			structs::RaidMOTD_Struct *outmotd = (structs::RaidMOTD_Struct *)outapp->pBuffer;
+			OUT(raidGen.action);
+			OUT(raidGen.parameter);
+			OUT_str(raidGen.leader_name);
+			OUT_str(raidGen.player_name);
+			OUT(_class);
+			OUT(level);
+			OUT(isGroupLeader);
+			OUT(flags[0]);
+			OUT(flags[1]);
+			OUT(flags[2]);
+			OUT(flags[3]);
+			OUT(flags[4]);
 
-			outmotd->general.action = inmotd->general.action;
-			strn0cpy(outmotd->general.player_name, inmotd->general.player_name, 64);
-			strn0cpy(outmotd->motd, inmotd->motd, strlen(inmotd->motd) + 1);
 			dest->FastQueuePacket(&outapp);
+			break;
 		}
-		else if (raid_gen->action == 14 || raid_gen->action == 30)
+		case raidSetMotd:
 		{
-			RaidLeadershipUpdate_Struct *inlaa = (RaidLeadershipUpdate_Struct *)__emu_buffer;
-			auto outapp =
-			    new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidLeadershipUpdate_Struct));
-			structs::RaidLeadershipUpdate_Struct *outlaa = (structs::RaidLeadershipUpdate_Struct *)outapp->pBuffer;
+			RaidMOTD_Struct* emu = (RaidMOTD_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidMOTD_Struct));
+			structs::RaidMOTD_Struct* eq = (structs::RaidMOTD_Struct*)outapp->pBuffer;
+
+			OUT(general.action);
+			OUT_str(general.player_name);
+			OUT_str(general.leader_name);
+			OUT_str(motd);
 
-			outlaa->action = inlaa->action;
-			strn0cpy(outlaa->player_name, inlaa->player_name, 64);
-			strn0cpy(outlaa->leader_name, inlaa->leader_name, 64);
-			memcpy(&outlaa->raid, &inlaa->raid, sizeof(RaidLeadershipAA_Struct));
 			dest->FastQueuePacket(&outapp);
+			break;
 		}
-		else
+		case raidSetLeaderAbilities:
+		case raidMakeLeader:
 		{
-			RaidGeneral_Struct* in_raid_general = (RaidGeneral_Struct*)__emu_buffer;
+			RaidLeadershipUpdate_Struct* emu = (RaidLeadershipUpdate_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidLeadershipUpdate_Struct));
+			structs::RaidLeadershipUpdate_Struct* eq = (structs::RaidLeadershipUpdate_Struct*)outapp->pBuffer;
+
+			OUT(action);
+			OUT_str(player_name);
+			OUT_str(leader_name);
+			memcpy(&eq->raid, &emu->raid, sizeof(RaidLeadershipAA_Struct));
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		case raidSetNote:
+		{
+			auto emu = (RaidNote_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidNote_Struct));
+			auto eq = (structs::RaidNote_Struct*)outapp->pBuffer;
+
+			OUT(general.action);
+			OUT_str(general.leader_name);
+			OUT_str(general.player_name);
+			OUT_str(note);
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		case raidNoRaid:
+		{
+			dest->QueuePacket(inapp);
+			break;
+		}
+		default:
+		{
+			RaidGeneral_Struct* emu = (RaidGeneral_Struct*)__emu_buffer;
 
 			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
-			structs::RaidGeneral_Struct *raid_general = (structs::RaidGeneral_Struct*)outapp->pBuffer;
-			strn0cpy(raid_general->leader_name, in_raid_general->leader_name, 64);
-			strn0cpy(raid_general->player_name, in_raid_general->player_name, 64);
-			raid_general->action = in_raid_general->action;
-			raid_general->parameter = in_raid_general->parameter;
-			dest->FastQueuePacket(&outapp);
-		}
+			structs::RaidGeneral_Struct* eq = (structs::RaidGeneral_Struct*)outapp->pBuffer;
 
+			OUT(action);
+			OUT(parameter);
+			OUT_str(leader_name);
+			OUT_str(player_name);
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		}
 		safe_delete(inapp);
 	}
 
@@ -1782,6 +1819,9 @@ namespace SoD
 			eq->window = emu->window;
 		OUT(type);
 		eq->invslot = ServerToSoDSlot(emu->invslot);
+		OUT(target_id);
+		OUT(can_cast);
+		OUT(can_scribe);
 		strn0cpy(eq->txtfile, emu->booktext, sizeof(eq->txtfile));
 
 		FINISH_ENCODE();
@@ -2817,6 +2857,17 @@ namespace SoD
 		FINISH_DIRECT_DECODE();
 	}
 
+	DECODE(OP_BookButton)
+	{
+		DECODE_LENGTH_EXACT(structs::BookButton_Struct);
+		SETUP_DIRECT_DECODE(BookButton_Struct, structs::BookButton_Struct);
+
+		emu->invslot = static_cast<int16_t>(SoDToServerSlot(eq->invslot));
+		IN(target_id);
+
+		FINISH_DIRECT_DECODE();
+	}
+
 	DECODE(OP_Buff)
 	{
 		DECODE_LENGTH_EXACT(structs::SpellBuffPacket_Struct);
@@ -3324,37 +3375,47 @@ namespace SoD
 	{
 		DECODE_LENGTH_ATLEAST(structs::RaidGeneral_Struct);
 
-		// This is a switch on the RaidGeneral action
-		switch (*(uint32 *)__packet->pBuffer) {
-			case 35: { // raidMOTD
-				// we don't have a nice macro for this
-				structs::RaidMOTD_Struct *__eq_buffer = (structs::RaidMOTD_Struct *)__packet->pBuffer;
-				__eq_buffer->motd[1023] = '\0';
-				size_t motd_size = strlen(__eq_buffer->motd) + 1;
-				__packet->size = sizeof(RaidMOTD_Struct) + motd_size;
-				__packet->pBuffer = new unsigned char[__packet->size];
-				RaidMOTD_Struct *emu = (RaidMOTD_Struct *)__packet->pBuffer;
-				structs::RaidMOTD_Struct *eq = (structs::RaidMOTD_Struct *)__eq_buffer;
-				strn0cpy(emu->general.player_name, eq->general.player_name, 64);
-				strn0cpy(emu->motd, eq->motd, motd_size);
-				IN(general.action);
-				IN(general.parameter);
-				FINISH_DIRECT_DECODE();
-				break;
-			}
-			case 36: { // raidPlayerNote unhandled
-				break;
-			}
-			default: {
-				DECODE_LENGTH_EXACT(structs::RaidGeneral_Struct);
-				SETUP_DIRECT_DECODE(RaidGeneral_Struct, structs::RaidGeneral_Struct);
-				strn0cpy(emu->leader_name, eq->leader_name, 64);
-				strn0cpy(emu->player_name, eq->player_name, 64);
-				IN(action);
-				IN(parameter);
-				FINISH_DIRECT_DECODE();
-				break;
-			}
+		RaidGeneral_Struct* rgs = (RaidGeneral_Struct*)__packet->pBuffer;
+
+		switch (rgs->action)
+		{
+		case raidSetMotd:
+		{
+			SETUP_VAR_DECODE(RaidMOTD_Struct, structs::RaidMOTD_Struct, motd);
+
+			IN(general.action);
+			IN(general.parameter);
+			IN_str(general.leader_name);
+			IN_str(general.player_name);
+			IN_str(motd);
+
+			FINISH_VAR_DECODE();
+			break;
+		}
+		case raidSetNote:
+		{
+			SETUP_VAR_DECODE(RaidNote_Struct, structs::RaidNote_Struct, note);
+
+			IN(general.action);
+			IN(general.parameter);
+			IN_str(general.leader_name);
+			IN_str(general.player_name);
+			IN_str(note);
+
+			FINISH_VAR_DECODE();
+			break;
+		}
+		default:
+		{
+			SETUP_DIRECT_DECODE(RaidGeneral_Struct, structs::RaidGeneral_Struct);
+			IN(action);
+			IN(parameter);
+			IN_str(leader_name);
+			IN_str(player_name);
+
+			FINISH_DIRECT_DECODE();
+			break;
+		}
 		}
 	}
 
@@ -3364,7 +3425,8 @@ namespace SoD
 		SETUP_DIRECT_DECODE(BookRequest_Struct, structs::BookRequest_Struct);
 
 		IN(type);
-		emu->invslot = SoDToServerSlot(eq->invslot);
+		emu->invslot = static_cast<int16_t>(SoDToServerSlot(eq->invslot));
+		IN(target_id);
 		emu->window = (uint8)eq->window;
 		strn0cpy(emu->txtfile, eq->txtfile, sizeof(emu->txtfile));
 

common/patches/sod_ops.h

@@ -103,6 +103,7 @@ D(OP_ApplyPoison)
 D(OP_AugmentInfo)
 D(OP_AugmentItem)
 D(OP_BazaarSearch)
+D(OP_BookButton)
 D(OP_Buff)
 D(OP_CastSpell)
 D(OP_ChannelMessage)

common/patches/sod_structs.h

@@ -2351,12 +2351,21 @@ struct BookText_Struct {
 // This is just a "text file" on the server
 // or in our case, the 'name' column in our books table.
 struct BookRequest_Struct {
-/*0000*/	uint32 window;		// where to display the text (0xFFFFFFFF means new window).
-/*0004*/	uint32 invslot;		// The inventory slot the book is in. Not used, but echoed in the response packet.
-/*0008*/	uint32 type;		// 0 = Scroll, 1 = Book, 2 = Item Info. Possibly others
-/*0012*/	uint32 unknown0012;
-/*0016*/	uint16 unknown0016;
-/*0018*/	char txtfile[8194];
+/*0000*/ uint32 window;     // where to display the text (0xFFFFFFFF means new window).
+/*0004*/ uint32 invslot;    // The inventory slot the book is in
+/*0008*/ uint32 type;       // 0 = Scroll, 1 = Book, 2 = Item Info. Possibly others
+/*0012*/ uint32 target_id;
+/*0016*/ uint8 can_cast;
+/*0017*/ uint8 can_scribe;
+/*0018*/ char txtfile[8194];
+};
+
+// used by Scribe and CastSpell book buttons
+struct BookButton_Struct
+{
+/*0000*/ int32 invslot;
+/*0004*/ int32 target_id; // client's target when using the book
+/*0008*/ int32 unused;    // always 0 from button packets
 };
 
 /*
@@ -3583,9 +3592,14 @@ struct RaidAddMember_Struct {
 /*139*/	uint8 flags[5]; //no idea if these are needed...
 };
 
+struct RaidNote_Struct {
+/*000*/ RaidGeneral_Struct general;
+/*140*/ char note[64];
+};
+
 struct RaidMOTD_Struct {
 /*000*/ RaidGeneral_Struct general; // leader_name and action only used
-/*140*/ char motd[0]; // max size 1024, but reply is variable
+/*140*/ char motd[1024]; // max size is 1024, but reply is variable
 };
 
 struct RaidLeadershipUpdate_Struct {

common/patches/sof.cpp

@@ -33,6 +33,7 @@
 #include "sof_structs.h"
 #include "../rulesys.h"
 #include "../path_manager.h"
+#include "../raid.h"
 
 #include <iostream>
 #include <sstream>
@@ -1356,88 +1357,124 @@ namespace SoF
 
 	ENCODE(OP_RaidJoin)
 	{
-		EQApplicationPacket *inapp = *p;
-		unsigned char * __emu_buffer = inapp->pBuffer;
-		RaidCreate_Struct *raid_create = (RaidCreate_Struct*)__emu_buffer;
+		EQApplicationPacket* inapp = *p;
+		*p = nullptr;
+		unsigned char* __emu_buffer = inapp->pBuffer;
+		RaidCreate_Struct* emu = (RaidCreate_Struct*)__emu_buffer;
 
-		auto outapp_create = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
-		structs::RaidGeneral_Struct *general = (structs::RaidGeneral_Struct*)outapp_create->pBuffer;
+		auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
+		structs::RaidGeneral_Struct* general = (structs::RaidGeneral_Struct*)outapp->pBuffer;
 
-		general->action = 8;
-		general->parameter = 1;
-		strn0cpy(general->leader_name, raid_create->leader_name, 64);
-		strn0cpy(general->player_name, raid_create->leader_name, 64);
+		general->action = raidCreate;
+		general->parameter = RaidCommandAcceptInvite;
+		strn0cpy(general->leader_name, emu->leader_name, sizeof(emu->leader_name));
+		strn0cpy(general->player_name, emu->leader_name, sizeof(emu->leader_name));
+
+		dest->FastQueuePacket(&outapp);
 
-		dest->FastQueuePacket(&outapp_create);
 		safe_delete(inapp);
+
 	}
 
 	ENCODE(OP_RaidUpdate)
 	{
-		EQApplicationPacket *inapp = *p;
+		EQApplicationPacket* inapp = *p;
 		*p = nullptr;
-		unsigned char * __emu_buffer = inapp->pBuffer;
-		RaidGeneral_Struct *raid_gen = (RaidGeneral_Struct*)__emu_buffer;
+		unsigned char* __emu_buffer = inapp->pBuffer;
+		RaidGeneral_Struct* raid_gen = (RaidGeneral_Struct*)__emu_buffer;
 
-		if (raid_gen->action == 0) // raid add has longer length than other raid updates
+		switch (raid_gen->action)
 		{
-			RaidAddMember_Struct* in_add_member = (RaidAddMember_Struct*)__emu_buffer;
+		case raidAdd:
+		{
+			RaidAddMember_Struct* emu = (RaidAddMember_Struct*)__emu_buffer;
 
 			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidAddMember_Struct));
-			structs::RaidAddMember_Struct *add_member = (structs::RaidAddMember_Struct*)outapp->pBuffer;
+			structs::RaidAddMember_Struct* eq = (structs::RaidAddMember_Struct*)outapp->pBuffer;
 
-			add_member->raidGen.action = in_add_member->raidGen.action;
-			add_member->raidGen.parameter = in_add_member->raidGen.parameter;
-			strn0cpy(add_member->raidGen.leader_name, in_add_member->raidGen.leader_name, 64);
-			strn0cpy(add_member->raidGen.player_name, in_add_member->raidGen.player_name, 64);
-			add_member->_class = in_add_member->_class;
-			add_member->level = in_add_member->level;
-			add_member->isGroupLeader = in_add_member->isGroupLeader;
-			add_member->flags[0] = in_add_member->flags[0];
-			add_member->flags[1] = in_add_member->flags[1];
-			add_member->flags[2] = in_add_member->flags[2];
-			add_member->flags[3] = in_add_member->flags[3];
-			add_member->flags[4] = in_add_member->flags[4];
-			dest->FastQueuePacket(&outapp);
-		}
-		else if (raid_gen->action == 35)
-		{
-			RaidMOTD_Struct *inmotd = (RaidMOTD_Struct *)__emu_buffer;
-			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidMOTD_Struct) +
-										 strlen(inmotd->motd) + 1);
-			structs::RaidMOTD_Struct *outmotd = (structs::RaidMOTD_Struct *)outapp->pBuffer;
+			OUT(raidGen.action);
+			OUT(raidGen.parameter);
+			OUT_str(raidGen.leader_name);
+			OUT_str(raidGen.player_name);
+			OUT(_class);
+			OUT(level);
+			OUT(isGroupLeader);
+			OUT(flags[0]);
+			OUT(flags[1]);
+			OUT(flags[2]);
+			OUT(flags[3]);
+			OUT(flags[4]);
 
-			outmotd->general.action = inmotd->general.action;
-			strn0cpy(outmotd->general.player_name, inmotd->general.player_name, 64);
-			strn0cpy(outmotd->motd, inmotd->motd, strlen(inmotd->motd) + 1);
 			dest->FastQueuePacket(&outapp);
+			break;
 		}
-		else if (raid_gen->action == 14 || raid_gen->action == 30)
+		case raidSetMotd:
 		{
-			RaidLeadershipUpdate_Struct *inlaa = (RaidLeadershipUpdate_Struct *)__emu_buffer;
-			auto outapp =
-			    new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidLeadershipUpdate_Struct));
-			structs::RaidLeadershipUpdate_Struct *outlaa = (structs::RaidLeadershipUpdate_Struct *)outapp->pBuffer;
+			RaidMOTD_Struct* emu = (RaidMOTD_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidMOTD_Struct));
+			structs::RaidMOTD_Struct* eq = (structs::RaidMOTD_Struct*)outapp->pBuffer;
+
+			OUT(general.action);
+			OUT_str(general.player_name);
+			OUT_str(general.leader_name);
+			OUT_str(motd);
 
-			outlaa->action = inlaa->action;
-			strn0cpy(outlaa->player_name, inlaa->player_name, 64);
-			strn0cpy(outlaa->leader_name, inlaa->leader_name, 64);
-			memcpy(&outlaa->raid, &inlaa->raid, sizeof(RaidLeadershipAA_Struct));
 			dest->FastQueuePacket(&outapp);
+			break;
 		}
-		else
+		case raidSetLeaderAbilities:
+		case raidMakeLeader:
 		{
-			RaidGeneral_Struct* in_raid_general = (RaidGeneral_Struct*)__emu_buffer;
+			RaidLeadershipUpdate_Struct* emu = (RaidLeadershipUpdate_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidLeadershipUpdate_Struct));
+			structs::RaidLeadershipUpdate_Struct* eq = (structs::RaidLeadershipUpdate_Struct*)outapp->pBuffer;
+
+			OUT(action);
+			OUT_str(player_name);
+			OUT_str(leader_name);
+			memcpy(&eq->raid, &emu->raid, sizeof(RaidLeadershipAA_Struct));
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		case raidSetNote:
+		{
+			auto emu = (RaidNote_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidNote_Struct));
+			auto eq = (structs::RaidNote_Struct*)outapp->pBuffer;
+
+			OUT(general.action);
+			OUT_str(general.leader_name);
+			OUT_str(general.player_name);
+			OUT_str(note);
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		case raidNoRaid:
+		{
+			dest->QueuePacket(inapp);
+			break;
+		}
+		default:
+		{
+			RaidGeneral_Struct* emu = (RaidGeneral_Struct*)__emu_buffer;
 
 			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
-			structs::RaidGeneral_Struct *raid_general = (structs::RaidGeneral_Struct*)outapp->pBuffer;
-			strn0cpy(raid_general->leader_name, in_raid_general->leader_name, 64);
-			strn0cpy(raid_general->player_name, in_raid_general->player_name, 64);
-			raid_general->action = in_raid_general->action;
-			raid_general->parameter = in_raid_general->parameter;
-			dest->FastQueuePacket(&outapp);
-		}
+			structs::RaidGeneral_Struct* eq = (structs::RaidGeneral_Struct*)outapp->pBuffer;
 
+			OUT(action);
+			OUT(parameter);
+			OUT_str(leader_name);
+			OUT_str(player_name);
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		}
 		safe_delete(inapp);
 	}
 
@@ -1452,6 +1489,9 @@ namespace SoF
 			eq->window = emu->window;
 		OUT(type);
 		eq->invslot = ServerToSoFSlot(emu->invslot);
+		OUT(target_id);
+		OUT(can_cast);
+		OUT(can_scribe);
 		strn0cpy(eq->txtfile, emu->booktext, sizeof(eq->txtfile));
 
 		FINISH_ENCODE();
@@ -2261,6 +2301,17 @@ namespace SoF
 		FINISH_DIRECT_DECODE();
 	}
 
+	DECODE(OP_BookButton)
+	{
+		DECODE_LENGTH_EXACT(structs::BookButton_Struct);
+		SETUP_DIRECT_DECODE(BookButton_Struct, structs::BookButton_Struct);
+
+		emu->invslot = static_cast<int16_t>(SoFToServerSlot(eq->invslot));
+		IN(target_id);
+
+		FINISH_DIRECT_DECODE();
+	}
+
 	DECODE(OP_Buff)
 	{
 		DECODE_LENGTH_EXACT(structs::SpellBuffPacket_Struct);
@@ -2729,37 +2780,47 @@ namespace SoF
 	{
 		DECODE_LENGTH_ATLEAST(structs::RaidGeneral_Struct);
 
-		// This is a switch on the RaidGeneral action
-		switch (*(uint32 *)__packet->pBuffer) {
-			case 35: { // raidMOTD
-				// we don't have a nice macro for this
-				structs::RaidMOTD_Struct *__eq_buffer = (structs::RaidMOTD_Struct *)__packet->pBuffer;
-				__eq_buffer->motd[1023] = '\0';
-				size_t motd_size = strlen(__eq_buffer->motd) + 1;
-				__packet->size = sizeof(RaidMOTD_Struct) + motd_size;
-				__packet->pBuffer = new unsigned char[__packet->size];
-				RaidMOTD_Struct *emu = (RaidMOTD_Struct *)__packet->pBuffer;
-				structs::RaidMOTD_Struct *eq = (structs::RaidMOTD_Struct *)__eq_buffer;
-				strn0cpy(emu->general.player_name, eq->general.player_name, 64);
-				strn0cpy(emu->motd, eq->motd, motd_size);
-				IN(general.action);
-				IN(general.parameter);
-				FINISH_DIRECT_DECODE();
-				break;
-			}
-			case 36: { // raidPlayerNote unhandled
-				break;
-			}
-			default: {
-				DECODE_LENGTH_EXACT(structs::RaidGeneral_Struct);
-				SETUP_DIRECT_DECODE(RaidGeneral_Struct, structs::RaidGeneral_Struct);
-				strn0cpy(emu->leader_name, eq->leader_name, 64);
-				strn0cpy(emu->player_name, eq->player_name, 64);
-				IN(action);
-				IN(parameter);
-				FINISH_DIRECT_DECODE();
-				break;
-			}
+		RaidGeneral_Struct* rgs = (RaidGeneral_Struct*)__packet->pBuffer;
+
+		switch (rgs->action)
+		{
+		case raidSetMotd:
+		{
+			SETUP_VAR_DECODE(RaidMOTD_Struct, structs::RaidMOTD_Struct, motd);
+
+			IN(general.action);
+			IN(general.parameter);
+			IN_str(general.leader_name);
+			IN_str(general.player_name);
+			IN_str(motd);
+
+			FINISH_VAR_DECODE();
+			break;
+		}
+		case raidSetNote:
+		{
+			SETUP_VAR_DECODE(RaidNote_Struct, structs::RaidNote_Struct, note);
+
+			IN(general.action);
+			IN(general.parameter);
+			IN_str(general.leader_name);
+			IN_str(general.player_name);
+			IN_str(note);
+
+			FINISH_VAR_DECODE();
+			break;
+		}
+		default:
+		{
+			SETUP_DIRECT_DECODE(RaidGeneral_Struct, structs::RaidGeneral_Struct);
+			IN(action);
+			IN(parameter);
+			IN_str(leader_name);
+			IN_str(player_name);
+
+			FINISH_DIRECT_DECODE();
+			break;
+		}
 		}
 	}
 
@@ -2769,7 +2830,8 @@ namespace SoF
 		SETUP_DIRECT_DECODE(BookRequest_Struct, structs::BookRequest_Struct);
 
 		IN(type);
-		emu->invslot = SoFToServerSlot(eq->invslot);
+		emu->invslot = static_cast<int16_t>(SoFToServerSlot(eq->invslot));
+		IN(target_id);
 		emu->window = (uint8)eq->window;
 		strn0cpy(emu->txtfile, eq->txtfile, sizeof(emu->txtfile));
 

common/patches/sof_ops.h

@@ -94,6 +94,7 @@ D(OP_AltCurrencySellSelection)
 D(OP_ApplyPoison)
 D(OP_AugmentInfo)
 D(OP_AugmentItem)
+D(OP_BookButton)
 D(OP_Buff)
 D(OP_Bug)
 D(OP_CastSpell)

common/patches/sof_structs.h

@@ -2321,12 +2321,21 @@ struct BookText_Struct {
 // This is just a "text file" on the server
 // or in our case, the 'name' column in our books table.
 struct BookRequest_Struct {
-/*0000*/	uint32 window;		// where to display the text (0xFFFFFFFF means new window).
-/*0004*/	uint32 invslot;		// The inventory slot the book is in. Not used, but echoed in the response packet.
-/*0008*/	uint32 type;		// 0 = Scroll, 1 = Book, 2 = Item Info. Possibly others
-/*0012*/	uint32 unknown0012;
-/*0016*/	uint16 unknown0016;
-/*0018*/	char txtfile[8194];
+/*0000*/ uint32 window;     // where to display the text (0xFFFFFFFF means new window).
+/*0004*/ uint32 invslot;    // The inventory slot the book is in
+/*0008*/ uint32 type;       // 0 = Scroll, 1 = Book, 2 = Item Info. Possibly others
+/*0012*/ uint32 target_id;
+/*0016*/ uint8 can_cast;
+/*0017*/ uint8 can_scribe;
+/*0018*/ char txtfile[8194];
+};
+
+// used by Scribe and CastSpell book buttons
+struct BookButton_Struct
+{
+/*0000*/ int32 invslot;
+/*0004*/ int32 target_id; // client's target when using the book
+/*0008*/ int32 unused;    // always 0 from button packets
 };
 
 /*
@@ -3508,9 +3517,14 @@ struct RaidAddMember_Struct {
 /*139*/	uint8 flags[5]; //no idea if these are needed...
 };
 
+struct RaidNote_Struct {
+/*000*/ RaidGeneral_Struct general;
+/*140*/ char note[64];
+};
+
 struct RaidMOTD_Struct {
 /*000*/ RaidGeneral_Struct general; // leader_name and action only used
-/*140*/ char motd[0]; // max size 1024, but reply is variable
+/*140*/ char motd[1024]; // max size is 1024, but reply is variable
 };
 
 struct RaidLeadershipUpdate_Struct {

common/patches/ss_define.h

@@ -128,6 +128,15 @@
 	emu_struct *emu = (emu_struct *) __packet->pBuffer; \
 	eq_struct *eq = (eq_struct *) __eq_buffer;
 
+#define SETUP_VAR_DECODE(emu_struct, eq_struct, var_field) \
+	unsigned char *__eq_buffer = __packet->pBuffer; \
+	eq_struct* in = (eq_struct*)__packet->pBuffer; \
+	auto size = strlen(in->var_field); \
+	__packet->size = sizeof(emu_struct) + size + 1; \
+	__packet->pBuffer = new unsigned char[__packet->size]; \
+	emu_struct *emu = (emu_struct *) __packet->pBuffer; \
+	eq_struct *eq = (eq_struct *) __eq_buffer;
+
 #define MEMSET_IN(emu_struct) \
 	memset(__packet->pBuffer, 0, sizeof(emu_struct));
 
@@ -146,6 +155,9 @@
 	delete[] __eq_buffer; \
 	p->SetOpcode(OP_Unknown);
 
+#define FINISH_VAR_DECODE() \
+	delete[] __eq_buffer; 
+
 //call to finish an encoder using SETUP_DIRECT_DECODE
 #define FINISH_DIRECT_DECODE() \
 	delete[] __eq_buffer;

common/patches/titanium.cpp

@@ -33,6 +33,7 @@
 #include "../item_instance.h"
 #include "titanium_structs.h"
 #include "../path_manager.h"
+#include "../raid.h"
 
 #include <sstream>
 
@@ -1245,6 +1246,119 @@ namespace Titanium
 		FINISH_ENCODE();
 	}
 
+	ENCODE(OP_MarkRaidNPC)
+	{
+		ENCODE_LENGTH_EXACT(MarkNPC_Struct);
+		SETUP_DIRECT_ENCODE(MarkNPC_Struct, MarkNPC_Struct);
+	
+		EQApplicationPacket* outapp = new EQApplicationPacket(OP_MarkNPC, sizeof(MarkNPC_Struct));
+		MarkNPC_Struct* mnpcs = (MarkNPC_Struct*)outapp->pBuffer;
+		mnpcs->TargetID = emu->TargetID;
+		mnpcs->Number = emu->Number;
+		dest->QueuePacket(outapp);
+		safe_delete(outapp);
+
+		FINISH_ENCODE();
+	}
+
+	ENCODE(OP_RaidUpdate)
+	{
+		EQApplicationPacket* inapp = *p;
+		*p = nullptr;
+		unsigned char* __emu_buffer = inapp->pBuffer;
+		RaidGeneral_Struct* raid_gen = (RaidGeneral_Struct*)__emu_buffer;
+		
+		switch (raid_gen->action)
+		{
+		case raidAdd: 
+		{
+			RaidAddMember_Struct* emu = (RaidAddMember_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidAddMember_Struct));
+			structs::RaidAddMember_Struct* eq = (structs::RaidAddMember_Struct*)outapp->pBuffer;
+
+			OUT(raidGen.action);
+			OUT(raidGen.parameter);
+			OUT_str(raidGen.leader_name);
+			OUT_str(raidGen.player_name);
+			OUT(_class);
+			OUT(level);
+			OUT(isGroupLeader);
+
+			dest->FastQueuePacket(&outapp);
+			break;
+
+		}
+		case raidSetMotd:
+		{
+			RaidMOTD_Struct* emu = (RaidMOTD_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidMOTD_Struct));
+			structs::RaidMOTD_Struct* eq = (structs::RaidMOTD_Struct*)outapp->pBuffer;
+
+			OUT(general.action);
+			OUT_str(general.player_name);
+			OUT_str(general.leader_name);
+			OUT_str(motd);
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		case raidSetLeaderAbilities:
+		case raidMakeLeader:
+		{
+			RaidLeadershipUpdate_Struct* emu = (RaidLeadershipUpdate_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidLeadershipUpdate_Struct));
+			structs::RaidLeadershipUpdate_Struct* eq = (structs::RaidLeadershipUpdate_Struct*)outapp->pBuffer;
+
+			OUT(action);
+			OUT_str(player_name);
+			OUT_str(leader_name);
+			memcpy(&eq->raid, &emu->raid, sizeof(RaidLeadershipAA_Struct));
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		case raidSetNote:
+		{
+			auto emu = (RaidNote_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidNote_Struct));
+			auto eq = (structs::RaidNote_Struct*)outapp->pBuffer;
+
+			OUT(general.action);
+			OUT_str(general.leader_name);
+			OUT_str(general.player_name);
+			OUT_str(note);
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		case raidNoRaid:
+		{
+			dest->QueuePacket(inapp);
+			break;
+		}
+		default: 
+		{
+			RaidGeneral_Struct* emu = (RaidGeneral_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
+			structs::RaidGeneral_Struct* eq = (structs::RaidGeneral_Struct*)outapp->pBuffer;
+
+			OUT(action);
+			OUT(parameter);
+			OUT_str(leader_name);
+			OUT_str(player_name);
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		}
+		safe_delete(inapp);
+	}
+
 	ENCODE(OP_ReadBook)
 	{
 		// no apparent slot translation needed
@@ -2272,6 +2386,63 @@ namespace Titanium
 		FINISH_DIRECT_DECODE();
 	}
 
+	DECODE(OP_RaidInvite)
+	{
+		DECODE_LENGTH_ATLEAST(structs::RaidGeneral_Struct);
+		RaidGeneral_Struct* rgs = (RaidGeneral_Struct*)__packet->pBuffer;
+
+		switch (rgs->action)
+		{
+		case raidSetMotd:
+		{
+			SETUP_VAR_DECODE(RaidMOTD_Struct, structs::RaidMOTD_Struct, motd);
+
+			IN(general.action);
+			IN(general.parameter);
+			IN_str(general.leader_name);
+			IN_str(general.player_name);
+			
+			auto len = 0;
+			if (__packet->size < sizeof(structs::RaidMOTD_Struct)) {
+				len = __packet->size - sizeof(structs::RaidGeneral_Struct);
+			}
+			else {
+				len = sizeof(eq->motd);
+			}
+			
+			strn0cpy(emu->motd, eq->motd, len > 1024 ? 1024 : len); 
+			emu->motd[len - 1] = '\0';
+
+			FINISH_VAR_DECODE();
+			break;
+		}
+		case raidSetNote:
+		{
+			SETUP_VAR_DECODE(RaidNote_Struct, structs::RaidNote_Struct, note);
+
+			IN(general.action);
+			IN(general.parameter);
+			IN_str(general.leader_name);
+			IN_str(general.player_name);
+			IN_str(note);
+
+			FINISH_VAR_DECODE();
+			break;
+		}		
+		default:
+		{
+			SETUP_DIRECT_DECODE(RaidGeneral_Struct, structs::RaidGeneral_Struct);
+			IN(action);
+			IN(parameter);
+			IN_str(leader_name);
+			IN_str(player_name);
+
+			FINISH_DIRECT_DECODE();
+			break;
+		}
+		}
+	}
+
 	DECODE(OP_ReadBook)
 	{
 		// no apparent slot translation needed

common/patches/titanium_ops.h

@@ -61,6 +61,8 @@ E(OP_OnLevelMessage)
 E(OP_PetBuffWindow)
 E(OP_PlayerProfile)
 E(OP_NewSpawn)
+E(OP_MarkRaidNPC)
+E(OP_RaidUpdate)
 E(OP_ReadBook)
 E(OP_RespondAA)
 E(OP_SendCharInfo)
@@ -106,6 +108,7 @@ D(OP_LoadSpellSet)
 D(OP_LootItem)
 D(OP_MoveItem)
 D(OP_PetCommands)
+D(OP_RaidInvite)
 D(OP_ReadBook)
 D(OP_SetServerFilter)
 D(OP_ShopPlayerSell)

common/patches/titanium_structs.h

@@ -3017,23 +3017,39 @@ struct leadExpUpdateStruct {
    /*0028*/ uint32 unknown0028;
 };
 
-
-
 struct RaidGeneral_Struct {
-/*00*/	uint32		action;	//=10
-/*04*/	char		player_name[64];	//should both be the player's name
-/*04*/	char		leader_name[64];
+/*000*/	uint32		action;	//=10
+/*004*/	char		player_name[64];	//should both be the player's name
+/*068*/	char		leader_name[64];
 /*132*/	uint32		parameter;
 };
 
-struct RaidAdd_Struct {
-/*000*/	uint32		action;	//=0
-/*004*/	char		player_name[64];	//should both be the player's name
-/*068*/	char		leader_name[64];
-/*132*/	uint8		_class;
-/*133*/	uint8		level;
-/*134*/	uint8		has_group;
-/*135*/	uint8		unknown135;	//seems to be 0x42 or 0
+struct RaidAddMember_Struct {
+/*000*/	RaidGeneral_Struct raidGen;
+/*136*/	uint8		_class;
+/*137*/	uint8		level;
+/*138*/	uint8		isGroupLeader;
+/*139*/	uint8		unknown139;	//seems to be 0x42 or 0
+};
+
+struct RaidNote_Struct {
+/*000*/ RaidGeneral_Struct general;
+/*136*/ char note[64];
+};
+
+struct RaidMOTD_Struct {
+/*000*/ RaidGeneral_Struct general; // leader_name and action only used
+/*136*/ char motd[1024]; // max size is 1024, but reply is variable
+};
+
+struct RaidLeadershipUpdate_Struct {
+	/*000*/	uint32 action;
+	/*004*/	char player_name[64];
+//	/*068*/	uint32 Unknown068;
+	/*072*/	char leader_name[64];
+	/*136*/	GroupLeadershipAA_Struct group; //unneeded
+	/*200*/	RaidLeadershipAA_Struct raid;
+	/*264*/	char Unknown264[128];
 };
 
 struct RaidCreate_Struct {

common/patches/uf.cpp

@@ -35,6 +35,7 @@
 #include "../path_manager.h"
 #include "../classes.h"
 #include "../races.h"
+#include "../raid.h"
 
 #include <iostream>
 #include <sstream>
@@ -1931,88 +1932,124 @@ namespace UF
 
 	ENCODE(OP_RaidJoin)
 	{
-		EQApplicationPacket *inapp = *p;
-		unsigned char * __emu_buffer = inapp->pBuffer;
-		RaidCreate_Struct *raid_create = (RaidCreate_Struct*)__emu_buffer;
+		EQApplicationPacket* inapp = *p;
+		*p = nullptr;
+		unsigned char* __emu_buffer = inapp->pBuffer;
+		RaidCreate_Struct* emu = (RaidCreate_Struct*)__emu_buffer;
 
-		auto outapp_create = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
-		structs::RaidGeneral_Struct *general = (structs::RaidGeneral_Struct*)outapp_create->pBuffer;
+		auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
+		structs::RaidGeneral_Struct* general = (structs::RaidGeneral_Struct*)outapp->pBuffer;
 
-		general->action = 8;
-		general->parameter = 1;
-		strn0cpy(general->leader_name, raid_create->leader_name, 64);
-		strn0cpy(general->player_name, raid_create->leader_name, 64);
+		general->action = raidCreate;
+		general->parameter = RaidCommandAcceptInvite;
+		strn0cpy(general->leader_name, emu->leader_name, sizeof(emu->leader_name));
+		strn0cpy(general->player_name, emu->leader_name, sizeof(emu->leader_name));
+
+		dest->FastQueuePacket(&outapp);
 
-		dest->FastQueuePacket(&outapp_create);
 		safe_delete(inapp);
+
 	}
 
 	ENCODE(OP_RaidUpdate)
 	{
-		EQApplicationPacket *inapp = *p;
+		EQApplicationPacket* inapp = *p;
 		*p = nullptr;
-		unsigned char * __emu_buffer = inapp->pBuffer;
-		RaidGeneral_Struct *raid_gen = (RaidGeneral_Struct*)__emu_buffer;
+		unsigned char* __emu_buffer = inapp->pBuffer;
+		RaidGeneral_Struct* raid_gen = (RaidGeneral_Struct*)__emu_buffer;
 
-		if (raid_gen->action == 0) // raid add has longer length than other raid updates
+		switch (raid_gen->action)
 		{
-			RaidAddMember_Struct* in_add_member = (RaidAddMember_Struct*)__emu_buffer;
+		case raidAdd:
+		{
+			RaidAddMember_Struct* emu = (RaidAddMember_Struct*)__emu_buffer;
 
 			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidAddMember_Struct));
-			structs::RaidAddMember_Struct *add_member = (structs::RaidAddMember_Struct*)outapp->pBuffer;
+			structs::RaidAddMember_Struct* eq = (structs::RaidAddMember_Struct*)outapp->pBuffer;
 
-			add_member->raidGen.action = in_add_member->raidGen.action;
-			add_member->raidGen.parameter = in_add_member->raidGen.parameter;
-			strn0cpy(add_member->raidGen.leader_name, in_add_member->raidGen.leader_name, 64);
-			strn0cpy(add_member->raidGen.player_name, in_add_member->raidGen.player_name, 64);
-			add_member->_class = in_add_member->_class;
-			add_member->level = in_add_member->level;
-			add_member->isGroupLeader = in_add_member->isGroupLeader;
-			add_member->flags[0] = in_add_member->flags[0];
-			add_member->flags[1] = in_add_member->flags[1];
-			add_member->flags[2] = in_add_member->flags[2];
-			add_member->flags[3] = in_add_member->flags[3];
-			add_member->flags[4] = in_add_member->flags[4];
-			dest->FastQueuePacket(&outapp);
-		}
-		else if (raid_gen->action == 35)
-		{
-			RaidMOTD_Struct *inmotd = (RaidMOTD_Struct *)__emu_buffer;
-			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidMOTD_Struct) +
-										 strlen(inmotd->motd) + 1);
-			structs::RaidMOTD_Struct *outmotd = (structs::RaidMOTD_Struct *)outapp->pBuffer;
+			OUT(raidGen.action);
+			OUT(raidGen.parameter);
+			OUT_str(raidGen.leader_name);
+			OUT_str(raidGen.player_name);
+			OUT(_class);
+			OUT(level);
+			OUT(isGroupLeader);
+			OUT(flags[0]);
+			OUT(flags[1]);
+			OUT(flags[2]);
+			OUT(flags[3]);
+			OUT(flags[4]);
 
-			outmotd->general.action = inmotd->general.action;
-			strn0cpy(outmotd->general.player_name, inmotd->general.player_name, 64);
-			strn0cpy(outmotd->motd, inmotd->motd, strlen(inmotd->motd) + 1);
 			dest->FastQueuePacket(&outapp);
+			break;
 		}
-		else if (raid_gen->action == 14 || raid_gen->action == 30)
+		case raidSetMotd:
 		{
-			RaidLeadershipUpdate_Struct *inlaa = (RaidLeadershipUpdate_Struct *)__emu_buffer;
-			auto outapp =
-			    new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidLeadershipUpdate_Struct));
-			structs::RaidLeadershipUpdate_Struct *outlaa = (structs::RaidLeadershipUpdate_Struct *)outapp->pBuffer;
+			RaidMOTD_Struct* emu = (RaidMOTD_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidMOTD_Struct));
+			structs::RaidMOTD_Struct* eq = (structs::RaidMOTD_Struct*)outapp->pBuffer;
+
+			OUT(general.action);
+			OUT_str(general.player_name);
+			OUT_str(general.leader_name);
+			OUT_str(motd);
 
-			outlaa->action = inlaa->action;
-			strn0cpy(outlaa->player_name, inlaa->player_name, 64);
-			strn0cpy(outlaa->leader_name, inlaa->leader_name, 64);
-			memcpy(&outlaa->raid, &inlaa->raid, sizeof(RaidLeadershipAA_Struct));
 			dest->FastQueuePacket(&outapp);
+			break;
 		}
-		else
+		case raidSetLeaderAbilities:
+		case raidMakeLeader:
 		{
-			RaidGeneral_Struct* in_raid_general = (RaidGeneral_Struct*)__emu_buffer;
+			RaidLeadershipUpdate_Struct* emu = (RaidLeadershipUpdate_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidLeadershipUpdate_Struct));
+			structs::RaidLeadershipUpdate_Struct* eq = (structs::RaidLeadershipUpdate_Struct*)outapp->pBuffer;
+
+			OUT(action);
+			OUT_str(player_name);
+			OUT_str(leader_name);
+			memcpy(&eq->raid, &emu->raid, sizeof(RaidLeadershipAA_Struct));
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		case raidSetNote:
+		{
+			auto emu = (RaidNote_Struct*)__emu_buffer;
+
+			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidNote_Struct));
+			auto eq = (structs::RaidNote_Struct*)outapp->pBuffer;
+
+			OUT(general.action);
+			OUT_str(general.leader_name);
+			OUT_str(general.player_name);
+			OUT_str(note);
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		case raidNoRaid:
+		{
+			dest->QueuePacket(inapp);
+			break;
+		}
+		default:
+		{
+			RaidGeneral_Struct* emu = (RaidGeneral_Struct*)__emu_buffer;
 
 			auto outapp = new EQApplicationPacket(OP_RaidUpdate, sizeof(structs::RaidGeneral_Struct));
-			structs::RaidGeneral_Struct *raid_general = (structs::RaidGeneral_Struct*)outapp->pBuffer;
-			strn0cpy(raid_general->leader_name, in_raid_general->leader_name, 64);
-			strn0cpy(raid_general->player_name, in_raid_general->player_name, 64);
-			raid_general->action = in_raid_general->action;
-			raid_general->parameter = in_raid_general->parameter;
-			dest->FastQueuePacket(&outapp);
-		}
+			structs::RaidGeneral_Struct* eq = (structs::RaidGeneral_Struct*)outapp->pBuffer;
 
+			OUT(action);
+			OUT(parameter);
+			OUT_str(leader_name);
+			OUT_str(player_name);
+
+			dest->FastQueuePacket(&outapp);
+			break;
+		}
+		}
 		safe_delete(inapp);
 	}
 
@@ -2027,6 +2064,9 @@ namespace UF
 			eq->window = emu->window;
 		OUT(type);
 		eq->invslot = ServerToUFSlot(emu->invslot);
+		OUT(target_id);
+		OUT(can_cast);
+		OUT(can_scribe);
 		strn0cpy(eq->txtfile, emu->booktext, sizeof(eq->txtfile));
 
 		FINISH_ENCODE();
@@ -3105,6 +3145,17 @@ namespace UF
 		FINISH_DIRECT_DECODE();
 	}
 
+	DECODE(OP_BookButton)
+	{
+		DECODE_LENGTH_EXACT(structs::BookButton_Struct);
+		SETUP_DIRECT_DECODE(BookButton_Struct, structs::BookButton_Struct);
+
+		emu->invslot = static_cast<int16_t>(UFToServerSlot(eq->invslot));
+		IN(target_id);
+
+		FINISH_DIRECT_DECODE();
+	}
+
 	DECODE(OP_Buff)
 	{
 		DECODE_LENGTH_EXACT(structs::SpellBuffPacket_Struct);
@@ -3623,39 +3674,48 @@ namespace UF
 	{
 		DECODE_LENGTH_ATLEAST(structs::RaidGeneral_Struct);
 
-		// This is a switch on the RaidGeneral action
-		switch (*(uint32 *)__packet->pBuffer) {
-			case 35: { // raidMOTD
-				// we don't have a nice macro for this
-				structs::RaidMOTD_Struct *__eq_buffer = (structs::RaidMOTD_Struct *)__packet->pBuffer;
-				__eq_buffer->motd[1023] = '\0';
-				size_t motd_size = strlen(__eq_buffer->motd) + 1;
-				__packet->size = sizeof(RaidMOTD_Struct) + motd_size;
-				__packet->pBuffer = new unsigned char[__packet->size];
-				RaidMOTD_Struct *emu = (RaidMOTD_Struct *)__packet->pBuffer;
-				structs::RaidMOTD_Struct *eq = (structs::RaidMOTD_Struct *)__eq_buffer;
-				strn0cpy(emu->general.player_name, eq->general.player_name, 64);
-				strn0cpy(emu->motd, eq->motd, motd_size);
-				IN(general.action);
-				IN(general.parameter);
-				FINISH_DIRECT_DECODE();
-				break;
-			}
-			case 36: { // raidPlayerNote unhandled
-				break;
-			}
-			default: {
-				DECODE_LENGTH_EXACT(structs::RaidGeneral_Struct);
-				SETUP_DIRECT_DECODE(RaidGeneral_Struct, structs::RaidGeneral_Struct);
-				strn0cpy(emu->leader_name, eq->leader_name, 64);
-				strn0cpy(emu->player_name, eq->player_name, 64);
-				IN(action);
-				IN(parameter);
-				FINISH_DIRECT_DECODE();
-				break;
-			}
-		}
+		RaidGeneral_Struct* rgs = (RaidGeneral_Struct*)__packet->pBuffer;
 
+		switch (rgs->action)
+		{
+		case raidSetMotd:
+		{
+			SETUP_VAR_DECODE(RaidMOTD_Struct, structs::RaidMOTD_Struct, motd);
+
+			IN(general.action);
+			IN(general.parameter);
+			IN_str(general.leader_name);
+			IN_str(general.player_name);
+			IN_str(motd);
+
+			FINISH_VAR_DECODE();
+			break;
+		}
+		case raidSetNote:
+		{
+			SETUP_VAR_DECODE(RaidNote_Struct, structs::RaidNote_Struct, note);
+
+			IN(general.action);
+			IN(general.parameter);
+			IN_str(general.leader_name);
+			IN_str(general.player_name);
+			IN_str(note);
+
+			FINISH_VAR_DECODE();
+			break;
+		}
+		default:
+		{
+			SETUP_DIRECT_DECODE(RaidGeneral_Struct, structs::RaidGeneral_Struct);
+			IN(action);
+			IN(parameter);
+			IN_str(leader_name);
+			IN_str(player_name);
+
+			FINISH_DIRECT_DECODE();
+			break;
+		}
+		}
 	}
 
 	DECODE(OP_ReadBook)
@@ -3664,7 +3724,8 @@ namespace UF
 		SETUP_DIRECT_DECODE(BookRequest_Struct, structs::BookRequest_Struct);
 
 		IN(type);
-		emu->invslot = UFToServerSlot(eq->invslot);
+		emu->invslot = static_cast<int16_t>(UFToServerSlot(eq->invslot));
+		IN(target_id);
 		emu->window = (uint8)eq->window;
 		strn0cpy(emu->txtfile, eq->txtfile, sizeof(emu->txtfile));
 

common/patches/uf_ops.h

@@ -110,6 +110,7 @@ D(OP_ApplyPoison)
 D(OP_AugmentInfo)
 D(OP_AugmentItem)
 D(OP_BazaarSearch)
+D(OP_BookButton)
 D(OP_Buff)
 D(OP_BuffRemoveRequest)
 D(OP_CastSpell)

common/patches/uf_structs.h

@@ -2400,12 +2400,21 @@ struct BookText_Struct {
 // This is just a "text file" on the server
 // or in our case, the 'name' column in our books table.
 struct BookRequest_Struct {
-/*0000*/	uint32 window;		// where to display the text (0xFFFFFFFF means new window).
-/*0004*/	uint32 invslot;		// The inventory slot the book is in. Not used, but echoed in the response packet.
-/*0008*/	uint32 type;		// 0 = Scroll, 1 = Book, 2 = Item Info. Possibly others
-/*0012*/	uint32 unknown0012;
-/*0016*/	uint16 unknown0016;
-/*0018*/	char txtfile[8194];
+/*0000*/ uint32 window;       // where to display the text (0xFFFFFFFF means new window).
+/*0004*/ uint32 invslot;      // The inventory slot the book is in
+/*0008*/ uint32 type;         // 0 = Scroll, 1 = Book, 2 = Item Info. Possibly others
+/*0012*/ uint32 target_id;
+/*0016*/ uint8 can_cast;
+/*0017*/ uint8 can_scribe;
+/*0018*/ char txtfile[8194];
+};
+
+// used by Scribe and CastSpell book buttons
+struct BookButton_Struct
+{
+/*0000*/ int32 invslot;
+/*0004*/ int32 target_id; // client's target when using the book
+/*0008*/ int32 unused;    // always 0 from button packets
 };
 
 /*
@@ -3638,9 +3647,14 @@ struct RaidAddMember_Struct {
 /*139*/	uint8 flags[5]; //no idea if these are needed...
 };
 
+struct RaidNote_Struct {
+/*000*/ RaidGeneral_Struct general;
+/*140*/ char note[64];
+};
+
 struct RaidMOTD_Struct {
-/*000*/ RaidGeneral_Struct general; // leader_name and action only used
-/*140*/ char motd[0]; // max size 1024, but reply is variable
+/*000*/ RaidGeneral_Struct general;
+/*140*/ char motd[1024];
 };
 
 struct RaidLeadershipUpdate_Struct {

common/platform.cpp

@@ -72,6 +72,8 @@ std::string GetPlatformName()
 			return "HC";
 		case EQEmuExePlatform::ExePlatformTests:
 			return "Tests";
+		case EQEmuExePlatform::ExePlatformZoneSidecar:
+			return "ZoneSidecar";
 		default:
 			return "";
 	}

common/platform.h

@@ -37,7 +37,8 @@ enum EQEmuExePlatform
 	ExePlatformClientImport,
 	ExePlatformClientExport,
 	ExePlatformHC,
-	ExePlatformTests
+	ExePlatformTests,
+	ExePlatformZoneSidecar
 };
 
 void RegisterExecutablePlatform(EQEmuExePlatform p);

common/process.cpp

@@ -0,0 +1,44 @@
+#include <string>
+#include <fstream>
+#include <algorithm>
+#include "process.h"
+
+inline std::string random_string(size_t length)
+{
+	auto        randchar = []() -> char {
+		const char   charset[] = "0123456789" "ABCDEFGHIJKLMNOPQRSTUVWXYZ" "abcdefghijklmnopqrstuvwxyz";
+		const size_t max_index = (sizeof(charset) - 1);
+		return charset[static_cast<size_t>(std::rand()) % max_index];
+	};
+	std::string str(length, 0);
+	std::generate_n(str.begin(), length, randchar);
+	return str;
+}
+
+std::string Process::execute(const std::string &cmd, bool return_result)
+{
+	std::string random     = "/tmp/" + random_string(25);
+	const char  *file_name = random.c_str();
+
+	if (return_result) {
+#ifdef _WINDOWS
+		std::system((cmd + " > " + file_name + " 2>&1").c_str());
+#else
+		std::system((cmd + " > " + file_name + " 2>&1").c_str());
+#endif
+	}
+	else {
+		std::system((cmd).c_str());
+	}
+
+	std::string result;
+
+	if (return_result) {
+		std::ifstream file(file_name);
+		result = {std::istreambuf_iterator<char>(file), std::istreambuf_iterator<char>()};
+		std::remove(file_name);
+
+	}
+
+	return result;
+}

common/process.h

@@ -0,0 +1,10 @@
+#ifndef EQEMU_PROCESS_H
+#define EQEMU_PROCESS_H
+
+class Process {
+public:
+	static std::string execute(const std::string &cmd, bool return_result = true);
+};
+
+
+#endif //EQEMU_PROCESS_H

common/races.cpp

@@ -1594,11 +1594,13 @@ float GetRaceGenderDefaultHeight(int race, int gender)
 
 	const auto size = sizeof(male_height) / sizeof(male_height[0]);
 
-	if (race >= size)
+	if (race >= size) {
 		return 6.0f;
+	}
 
-	if (gender == 1)
+	if (gender == FEMALE) {
 		return female_height[race];
+	}
 
 	return male_height[race];
 }

common/raid.h

@@ -0,0 +1,72 @@
+/*	EQEMu: Everquest Server Emulator
+	Copyright (C) 2001-2016 EQEMu Development Team (http://eqemu.org)
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; version 2 of the License.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY except by those people which sell it, which
+	are required to give you total support for your newly bought product;
+	without even the implied warranty of MERCHANTABILITY or FITNESS FOR
+	A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program; if not, write to the Free Software
+	Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+
+#ifndef RAID_H
+#define RAID_H
+
+enum {	//raid packet types:
+	raidAdd = 0,
+	raidRemove2 = 1,	//parameter=0
+	raidMemberNameChange = 2,
+	raidRemove1 = 3,	//parameter=0xFFFFFFFF
+	raidNoLongerLeader = 4,
+	raidDisband = 5,
+	raidMembers = 6,	//len 395+, details + members list
+	raidNoAssignLeadership = 7,
+	raidCreate = 8,		//len 72
+	raidUnknown = 9, // unused?
+	raidNoRaid = 10,		//parameter=0
+	raidChangeLootType = 11,
+	raidStringID = 12,
+	raidChangeGroupLeader = 13,	//136 raid leader, new group leader, group_id?
+	raidSetLeaderAbilities = 14,	//472
+	raidSetLeaderData = 15,	// 14,15 SoE names, not sure on difference, 14 packet has 0x100 bytes 15 0x214 in addition to raid general
+	raidChangeGroup = 16,	//?? len 136 old leader, new leader, 0 (preceeded with a remove2)
+	raidLock = 17,		//len 136 leader?, leader, 0
+	raidUnlock = 18,		//len 136 leader?, leader, 0
+	raidRedStringID = 19,
+	raidSetLeader = 20,	//len 388, contains 'details' struct without members; also used for "invite to raid"
+	raidMakeLeader = 30,
+	raidSetMotd = 35,
+	raidSetNote = 36,
+};
+
+
+enum { //raid command types
+	RaidCommandInviteIntoExisting = 0, //in use
+	RaidCommandAcceptInvite = 1, //in use
+	RaidCommandInvite = 3, //in use
+	RaidCommandDisband = 5, //in use
+	RaidCommandMoveGroup = 6, //in use
+	RaidCommandRemoveGroupLeader = 7,
+	RaidCommandRaidLock = 8, //in use
+	RaidCommandRaidUnlock = 9, //in use
+	RaidCommandLootType = 20, //in use
+	RaidCommandAddLooter = 21, //in use
+	RaidCommandRemoveLooter = 22, //in use
+	RaidCommandMakeLeader = 30,
+	RaidCommandInviteFail = 31, //already in raid, waiting on invite from other raid, etc
+	RaidCommandLootType2 = 32, //in use
+	RaidCommandAddLooter2 = 33, //in use
+	RaidCommandRemoveLooter2 = 34, //in use
+	RaidCommandSetMotd = 35,
+	RaidCommandSetNote = 36,
+};
+
+
+#endif

common/repositories/base/base_bot_starting_items_repository.h

@@ -0,0 +1,414 @@
+/**
+ * DO NOT MODIFY THIS FILE
+ *
+ * This repository was automatically generated and is NOT to be modified directly.
+ * Any repository modifications are meant to be made to the repository extending the base.
+ * Any modifications to base repositories are to be made by the generator only
+ *
+ * @generator ./utils/scripts/generators/repository-generator.pl
+ * @docs https://eqemu.gitbook.io/server/in-development/developer-area/repositories
+ */
+
+#ifndef EQEMU_BASE_BOT_STARTING_ITEMS_REPOSITORY_H
+#define EQEMU_BASE_BOT_STARTING_ITEMS_REPOSITORY_H
+
+#include "../../database.h"
+#include "../../strings.h"
+#include <ctime>
+
+
+class BaseBotStartingItemsRepository {
+public:
+	struct BotStartingItems {
+		uint32_t    id;
+		uint32_t    races;
+		uint32_t    classes;
+		uint32_t    item_id;
+		uint8_t     item_charges;
+		int32_t     slot_id;
+		int8_t      min_expansion;
+		int8_t      max_expansion;
+		std::string content_flags;
+		std::string content_flags_disabled;
+	};
+
+	static std::string PrimaryKey()
+	{
+		return std::string("id");
+	}
+
+	static std::vector<std::string> Columns()
+	{
+		return {
+			"id",
+			"races",
+			"classes",
+			"item_id",
+			"item_charges",
+			"slot_id",
+			"min_expansion",
+			"max_expansion",
+			"content_flags",
+			"content_flags_disabled",
+		};
+	}
+
+	static std::vector<std::string> SelectColumns()
+	{
+		return {
+			"id",
+			"races",
+			"classes",
+			"item_id",
+			"item_charges",
+			"slot_id",
+			"min_expansion",
+			"max_expansion",
+			"content_flags",
+			"content_flags_disabled",
+		};
+	}
+
+	static std::string ColumnsRaw()
+	{
+		return std::string(Strings::Implode(", ", Columns()));
+	}
+
+	static std::string SelectColumnsRaw()
+	{
+		return std::string(Strings::Implode(", ", SelectColumns()));
+	}
+
+	static std::string TableName()
+	{
+		return std::string("bot_starting_items");
+	}
+
+	static std::string BaseSelect()
+	{
+		return fmt::format(
+			"SELECT {} FROM {}",
+			SelectColumnsRaw(),
+			TableName()
+		);
+	}
+
+	static std::string BaseInsert()
+	{
+		return fmt::format(
+			"INSERT INTO {} ({}) ",
+			TableName(),
+			ColumnsRaw()
+		);
+	}
+
+	static BotStartingItems NewEntity()
+	{
+		BotStartingItems e{};
+
+		e.id                     = 0;
+		e.races                  = 0;
+		e.classes                = 0;
+		e.item_id                = 0;
+		e.item_charges           = 1;
+		e.slot_id                = -1;
+		e.min_expansion          = -1;
+		e.max_expansion          = -1;
+		e.content_flags          = "";
+		e.content_flags_disabled = "";
+
+		return e;
+	}
+
+	static BotStartingItems GetBotStartingItems(
+		const std::vector<BotStartingItems> &bot_starting_itemss,
+		int bot_starting_items_id
+	)
+	{
+		for (auto &bot_starting_items : bot_starting_itemss) {
+			if (bot_starting_items.id == bot_starting_items_id) {
+				return bot_starting_items;
+			}
+		}
+
+		return NewEntity();
+	}
+
+	static BotStartingItems FindOne(
+		Database& db,
+		int bot_starting_items_id
+	)
+	{
+		auto results = db.QueryDatabase(
+			fmt::format(
+				"{} WHERE {} = {} LIMIT 1",
+				BaseSelect(),
+				PrimaryKey(),
+				bot_starting_items_id
+			)
+		);
+
+		auto row = results.begin();
+		if (results.RowCount() == 1) {
+			BotStartingItems e{};
+
+			e.id                     = static_cast<uint32_t>(strtoul(row[0], nullptr, 10));
+			e.races                  = static_cast<uint32_t>(strtoul(row[1], nullptr, 10));
+			e.classes                = static_cast<uint32_t>(strtoul(row[2], nullptr, 10));
+			e.item_id                = static_cast<uint32_t>(strtoul(row[3], nullptr, 10));
+			e.item_charges           = static_cast<uint8_t>(strtoul(row[4], nullptr, 10));
+			e.slot_id                = static_cast<int32_t>(atoi(row[5]));
+			e.min_expansion          = static_cast<int8_t>(atoi(row[6]));
+			e.max_expansion          = static_cast<int8_t>(atoi(row[7]));
+			e.content_flags          = row[8] ? row[8] : "";
+			e.content_flags_disabled = row[9] ? row[9] : "";
+
+			return e;
+		}
+
+		return NewEntity();
+	}
+
+	static int DeleteOne(
+		Database& db,
+		int bot_starting_items_id
+	)
+	{
+		auto results = db.QueryDatabase(
+			fmt::format(
+				"DELETE FROM {} WHERE {} = {}",
+				TableName(),
+				PrimaryKey(),
+				bot_starting_items_id
+			)
+		);
+
+		return (results.Success() ? results.RowsAffected() : 0);
+	}
+
+	static int UpdateOne(
+		Database& db,
+		const BotStartingItems &e
+	)
+	{
+		std::vector<std::string> v;
+
+		auto columns = Columns();
+
+		v.push_back(columns[1] + " = " + std::to_string(e.races));
+		v.push_back(columns[2] + " = " + std::to_string(e.classes));
+		v.push_back(columns[3] + " = " + std::to_string(e.item_id));
+		v.push_back(columns[4] + " = " + std::to_string(e.item_charges));
+		v.push_back(columns[5] + " = " + std::to_string(e.slot_id));
+		v.push_back(columns[6] + " = " + std::to_string(e.min_expansion));
+		v.push_back(columns[7] + " = " + std::to_string(e.max_expansion));
+		v.push_back(columns[8] + " = '" + Strings::Escape(e.content_flags) + "'");
+		v.push_back(columns[9] + " = '" + Strings::Escape(e.content_flags_disabled) + "'");
+
+		auto results = db.QueryDatabase(
+			fmt::format(
+				"UPDATE {} SET {} WHERE {} = {}",
+				TableName(),
+				Strings::Implode(", ", v),
+				PrimaryKey(),
+				e.id
+			)
+		);
+
+		return (results.Success() ? results.RowsAffected() : 0);
+	}
+
+	static BotStartingItems InsertOne(
+		Database& db,
+		BotStartingItems e
+	)
+	{
+		std::vector<std::string> v;
+
+		v.push_back(std::to_string(e.id));
+		v.push_back(std::to_string(e.races));
+		v.push_back(std::to_string(e.classes));
+		v.push_back(std::to_string(e.item_id));
+		v.push_back(std::to_string(e.item_charges));
+		v.push_back(std::to_string(e.slot_id));
+		v.push_back(std::to_string(e.min_expansion));
+		v.push_back(std::to_string(e.max_expansion));
+		v.push_back("'" + Strings::Escape(e.content_flags) + "'");
+		v.push_back("'" + Strings::Escape(e.content_flags_disabled) + "'");
+
+		auto results = db.QueryDatabase(
+			fmt::format(
+				"{} VALUES ({})",
+				BaseInsert(),
+				Strings::Implode(",", v)
+			)
+		);
+
+		if (results.Success()) {
+			e.id = results.LastInsertedID();
+			return e;
+		}
+
+		e = NewEntity();
+
+		return e;
+	}
+
+	static int InsertMany(
+		Database& db,
+		const std::vector<BotStartingItems> &entries
+	)
+	{
+		std::vector<std::string> insert_chunks;
+
+		for (auto &e: entries) {
+			std::vector<std::string> v;
+
+			v.push_back(std::to_string(e.id));
+			v.push_back(std::to_string(e.races));
+			v.push_back(std::to_string(e.classes));
+			v.push_back(std::to_string(e.item_id));
+			v.push_back(std::to_string(e.item_charges));
+			v.push_back(std::to_string(e.slot_id));
+			v.push_back(std::to_string(e.min_expansion));
+			v.push_back(std::to_string(e.max_expansion));
+			v.push_back("'" + Strings::Escape(e.content_flags) + "'");
+			v.push_back("'" + Strings::Escape(e.content_flags_disabled) + "'");
+
+			insert_chunks.push_back("(" + Strings::Implode(",", v) + ")");
+		}
+
+		std::vector<std::string> v;
+
+		auto results = db.QueryDatabase(
+			fmt::format(
+				"{} VALUES {}",
+				BaseInsert(),
+				Strings::Implode(",", insert_chunks)
+			)
+		);
+
+		return (results.Success() ? results.RowsAffected() : 0);
+	}
+
+	static std::vector<BotStartingItems> All(Database& db)
+	{
+		std::vector<BotStartingItems> all_entries;
+
+		auto results = db.QueryDatabase(
+			fmt::format(
+				"{}",
+				BaseSelect()
+			)
+		);
+
+		all_entries.reserve(results.RowCount());
+
+		for (auto row = results.begin(); row != results.end(); ++row) {
+			BotStartingItems e{};
+
+			e.id                     = static_cast<uint32_t>(strtoul(row[0], nullptr, 10));
+			e.races                  = static_cast<uint32_t>(strtoul(row[1], nullptr, 10));
+			e.classes                = static_cast<uint32_t>(strtoul(row[2], nullptr, 10));
+			e.item_id                = static_cast<uint32_t>(strtoul(row[3], nullptr, 10));
+			e.item_charges           = static_cast<uint8_t>(strtoul(row[4], nullptr, 10));
+			e.slot_id                = static_cast<int32_t>(atoi(row[5]));
+			e.min_expansion          = static_cast<int8_t>(atoi(row[6]));
+			e.max_expansion          = static_cast<int8_t>(atoi(row[7]));
+			e.content_flags          = row[8] ? row[8] : "";
+			e.content_flags_disabled = row[9] ? row[9] : "";
+
+			all_entries.push_back(e);
+		}
+
+		return all_entries;
+	}
+
+	static std::vector<BotStartingItems> GetWhere(Database& db, const std::string &where_filter)
+	{
+		std::vector<BotStartingItems> all_entries;
+
+		auto results = db.QueryDatabase(
+			fmt::format(
+				"{} WHERE {}",
+				BaseSelect(),
+				where_filter
+			)
+		);
+
+		all_entries.reserve(results.RowCount());
+
+		for (auto row = results.begin(); row != results.end(); ++row) {
+			BotStartingItems e{};
+
+			e.id                     = static_cast<uint32_t>(strtoul(row[0], nullptr, 10));
+			e.races                  = static_cast<uint32_t>(strtoul(row[1], nullptr, 10));
+			e.classes                = static_cast<uint32_t>(strtoul(row[2], nullptr, 10));
+			e.item_id                = static_cast<uint32_t>(strtoul(row[3], nullptr, 10));
+			e.item_charges           = static_cast<uint8_t>(strtoul(row[4], nullptr, 10));
+			e.slot_id                = static_cast<int32_t>(atoi(row[5]));
+			e.min_expansion          = static_cast<int8_t>(atoi(row[6]));
+			e.max_expansion          = static_cast<int8_t>(atoi(row[7]));
+			e.content_flags          = row[8] ? row[8] : "";
+			e.content_flags_disabled = row[9] ? row[9] : "";
+
+			all_entries.push_back(e);
+		}
+
+		return all_entries;
+	}
+
+	static int DeleteWhere(Database& db, const std::string &where_filter)
+	{
+		auto results = db.QueryDatabase(
+			fmt::format(
+				"DELETE FROM {} WHERE {}",
+				TableName(),
+				where_filter
+			)
+		);
+
+		return (results.Success() ? results.RowsAffected() : 0);
+	}
+
+	static int Truncate(Database& db)
+	{
+		auto results = db.QueryDatabase(
+			fmt::format(
+				"TRUNCATE TABLE {}",
+				TableName()
+			)
+		);
+
+		return (results.Success() ? results.RowsAffected() : 0);
+	}
+
+	static int64 GetMaxId(Database& db)
+	{
+		auto results = db.QueryDatabase(
+			fmt::format(
+				"SELECT COALESCE(MAX({}), 0) FROM {}",
+				PrimaryKey(),
+				TableName()
+			)
+		);
+
+		return (results.Success() && results.begin()[0] ? strtoll(results.begin()[0], nullptr, 10) : 0);
+	}
+
+	static int64 Count(Database& db, const std::string &where_filter = "")
+	{
+		auto results = db.QueryDatabase(
+			fmt::format(
+				"SELECT COUNT(*) FROM {} {}",
+				TableName(),
+				(where_filter.empty() ? "" : "WHERE " + where_filter)
+			)
+		);
+
+		return (results.Success() && results.begin()[0] ? strtoll(results.begin()[0], nullptr, 10) : 0);
+	}
+
+};
+
+#endif //EQEMU_BASE_BOT_STARTING_ITEMS_REPOSITORY_H

common/repositories/base/base_data_buckets_repository.h

@@ -15,7 +15,7 @@
 #include "../../database.h"
 #include "../../strings.h"
 #include <ctime>
-
+#include <cereal/cereal.hpp>
 
 class BaseDataBucketsRepository {
 public:
@@ -27,6 +27,21 @@ public:
 		int64_t     character_id;
 		int64_t     npc_id;
 		int64_t     bot_id;
+
+		// cereal
+		template<class Archive>
+		void serialize(Archive &ar)
+		{
+			ar(
+				CEREAL_NVP(id),
+				CEREAL_NVP(key_),
+				CEREAL_NVP(value),
+				CEREAL_NVP(expires),
+				CEREAL_NVP(character_id),
+				CEREAL_NVP(npc_id),
+				CEREAL_NVP(bot_id)
+			);
+		}
 	};
 
 	static std::string PrimaryKey()

common/repositories/base/base_raid_details_repository.h

@@ -24,9 +24,15 @@ public:
 		int32_t     loottype;
 		int8_t      locked;
 		std::string motd;
-		uint16_t    marked_npc_1;
-		uint16_t    marked_npc_2;
-		uint16_t    marked_npc_3;
+		uint32_t    marked_npc_1_entity_id;
+		uint32_t    marked_npc_1_zone_id;
+		uint32_t    marked_npc_1_instance_id;
+		uint32_t    marked_npc_2_entity_id;
+		uint32_t    marked_npc_2_zone_id;
+		uint32_t    marked_npc_2_instance_id;
+		uint32_t    marked_npc_3_entity_id;
+		uint32_t    marked_npc_3_zone_id;
+		uint32_t    marked_npc_3_instance_id;
 	};
 
 	static std::string PrimaryKey()
@@ -41,9 +47,15 @@ public:
 			"loottype",
 			"locked",
 			"motd",
-			"marked_npc_1",
-			"marked_npc_2",
-			"marked_npc_3",
+			"marked_npc_1_entity_id",
+			"marked_npc_1_zone_id",
+			"marked_npc_1_instance_id",
+			"marked_npc_2_entity_id",
+			"marked_npc_2_zone_id",
+			"marked_npc_2_instance_id",
+			"marked_npc_3_entity_id",
+			"marked_npc_3_zone_id",
+			"marked_npc_3_instance_id",
 		};
 	}
 
@@ -54,9 +66,15 @@ public:
 			"loottype",
 			"locked",
 			"motd",
-			"marked_npc_1",
-			"marked_npc_2",
-			"marked_npc_3",
+			"marked_npc_1_entity_id",
+			"marked_npc_1_zone_id",
+			"marked_npc_1_instance_id",
+			"marked_npc_2_entity_id",
+			"marked_npc_2_zone_id",
+			"marked_npc_2_instance_id",
+			"marked_npc_3_entity_id",
+			"marked_npc_3_zone_id",
+			"marked_npc_3_instance_id",
 		};
 	}
 
@@ -97,13 +115,19 @@ public:
 	{
 		RaidDetails e{};
 
-		e.raidid       = 0;
-		e.loottype     = 0;
-		e.locked       = 0;
-		e.motd         = "";
-		e.marked_npc_1 = 0;
-		e.marked_npc_2 = 0;
-		e.marked_npc_3 = 0;
+		e.raidid                   = 0;
+		e.loottype                 = 0;
+		e.locked                   = 0;
+		e.motd                     = "";
+		e.marked_npc_1_entity_id   = 0;
+		e.marked_npc_1_zone_id     = 0;
+		e.marked_npc_1_instance_id = 0;
+		e.marked_npc_2_entity_id   = 0;
+		e.marked_npc_2_zone_id     = 0;
+		e.marked_npc_2_instance_id = 0;
+		e.marked_npc_3_entity_id   = 0;
+		e.marked_npc_3_zone_id     = 0;
+		e.marked_npc_3_instance_id = 0;
 
 		return e;
 	}
@@ -140,13 +164,19 @@ public:
 		if (results.RowCount() == 1) {
 			RaidDetails e{};
 
-			e.raidid       = static_cast<int32_t>(atoi(row[0]));
-			e.loottype     = static_cast<int32_t>(atoi(row[1]));
-			e.locked       = static_cast<int8_t>(atoi(row[2]));
-			e.motd         = row[3] ? row[3] : "";
-			e.marked_npc_1 = static_cast<uint16_t>(strtoul(row[4], nullptr, 10));
-			e.marked_npc_2 = static_cast<uint16_t>(strtoul(row[5], nullptr, 10));
-			e.marked_npc_3 = static_cast<uint16_t>(strtoul(row[6], nullptr, 10));
+			e.raidid                   = static_cast<int32_t>(atoi(row[0]));
+			e.loottype                 = static_cast<int32_t>(atoi(row[1]));
+			e.locked                   = static_cast<int8_t>(atoi(row[2]));
+			e.motd                     = row[3] ? row[3] : "";
+			e.marked_npc_1_entity_id   = static_cast<uint32_t>(strtoul(row[4], nullptr, 10));
+			e.marked_npc_1_zone_id     = static_cast<uint32_t>(strtoul(row[5], nullptr, 10));
+			e.marked_npc_1_instance_id = static_cast<uint32_t>(strtoul(row[6], nullptr, 10));
+			e.marked_npc_2_entity_id   = static_cast<uint32_t>(strtoul(row[7], nullptr, 10));
+			e.marked_npc_2_zone_id     = static_cast<uint32_t>(strtoul(row[8], nullptr, 10));
+			e.marked_npc_2_instance_id = static_cast<uint32_t>(strtoul(row[9], nullptr, 10));
+			e.marked_npc_3_entity_id   = static_cast<uint32_t>(strtoul(row[10], nullptr, 10));
+			e.marked_npc_3_zone_id     = static_cast<uint32_t>(strtoul(row[11], nullptr, 10));
+			e.marked_npc_3_instance_id = static_cast<uint32_t>(strtoul(row[12], nullptr, 10));
 
 			return e;
 		}
@@ -184,9 +214,15 @@ public:
 		v.push_back(columns[1] + " = " + std::to_string(e.loottype));
 		v.push_back(columns[2] + " = " + std::to_string(e.locked));
 		v.push_back(columns[3] + " = '" + Strings::Escape(e.motd) + "'");
-		v.push_back(columns[4] + " = " + std::to_string(e.marked_npc_1));
-		v.push_back(columns[5] + " = " + std::to_string(e.marked_npc_2));
-		v.push_back(columns[6] + " = " + std::to_string(e.marked_npc_3));
+		v.push_back(columns[4] + " = " + std::to_string(e.marked_npc_1_entity_id));
+		v.push_back(columns[5] + " = " + std::to_string(e.marked_npc_1_zone_id));
+		v.push_back(columns[6] + " = " + std::to_string(e.marked_npc_1_instance_id));
+		v.push_back(columns[7] + " = " + std::to_string(e.marked_npc_2_entity_id));
+		v.push_back(columns[8] + " = " + std::to_string(e.marked_npc_2_zone_id));
+		v.push_back(columns[9] + " = " + std::to_string(e.marked_npc_2_instance_id));
+		v.push_back(columns[10] + " = " + std::to_string(e.marked_npc_3_entity_id));
+		v.push_back(columns[11] + " = " + std::to_string(e.marked_npc_3_zone_id));
+		v.push_back(columns[12] + " = " + std::to_string(e.marked_npc_3_instance_id));
 
 		auto results = db.QueryDatabase(
 			fmt::format(
@@ -212,9 +248,15 @@ public:
 		v.push_back(std::to_string(e.loottype));
 		v.push_back(std::to_string(e.locked));
 		v.push_back("'" + Strings::Escape(e.motd) + "'");
-		v.push_back(std::to_string(e.marked_npc_1));
-		v.push_back(std::to_string(e.marked_npc_2));
-		v.push_back(std::to_string(e.marked_npc_3));
+		v.push_back(std::to_string(e.marked_npc_1_entity_id));
+		v.push_back(std::to_string(e.marked_npc_1_zone_id));
+		v.push_back(std::to_string(e.marked_npc_1_instance_id));
+		v.push_back(std::to_string(e.marked_npc_2_entity_id));
+		v.push_back(std::to_string(e.marked_npc_2_zone_id));
+		v.push_back(std::to_string(e.marked_npc_2_instance_id));
+		v.push_back(std::to_string(e.marked_npc_3_entity_id));
+		v.push_back(std::to_string(e.marked_npc_3_zone_id));
+		v.push_back(std::to_string(e.marked_npc_3_instance_id));
 
 		auto results = db.QueryDatabase(
 			fmt::format(
@@ -248,9 +290,15 @@ public:
 			v.push_back(std::to_string(e.loottype));
 			v.push_back(std::to_string(e.locked));
 			v.push_back("'" + Strings::Escape(e.motd) + "'");
-			v.push_back(std::to_string(e.marked_npc_1));
-			v.push_back(std::to_string(e.marked_npc_2));
-			v.push_back(std::to_string(e.marked_npc_3));
+			v.push_back(std::to_string(e.marked_npc_1_entity_id));
+			v.push_back(std::to_string(e.marked_npc_1_zone_id));
+			v.push_back(std::to_string(e.marked_npc_1_instance_id));
+			v.push_back(std::to_string(e.marked_npc_2_entity_id));
+			v.push_back(std::to_string(e.marked_npc_2_zone_id));
+			v.push_back(std::to_string(e.marked_npc_2_instance_id));
+			v.push_back(std::to_string(e.marked_npc_3_entity_id));
+			v.push_back(std::to_string(e.marked_npc_3_zone_id));
+			v.push_back(std::to_string(e.marked_npc_3_instance_id));
 
 			insert_chunks.push_back("(" + Strings::Implode(",", v) + ")");
 		}
@@ -284,13 +332,19 @@ public:
 		for (auto row = results.begin(); row != results.end(); ++row) {
 			RaidDetails e{};
 
-			e.raidid       = static_cast<int32_t>(atoi(row[0]));
-			e.loottype     = static_cast<int32_t>(atoi(row[1]));
-			e.locked       = static_cast<int8_t>(atoi(row[2]));
-			e.motd         = row[3] ? row[3] : "";
-			e.marked_npc_1 = static_cast<uint16_t>(strtoul(row[4], nullptr, 10));
-			e.marked_npc_2 = static_cast<uint16_t>(strtoul(row[5], nullptr, 10));
-			e.marked_npc_3 = static_cast<uint16_t>(strtoul(row[6], nullptr, 10));
+			e.raidid                   = static_cast<int32_t>(atoi(row[0]));
+			e.loottype                 = static_cast<int32_t>(atoi(row[1]));
+			e.locked                   = static_cast<int8_t>(atoi(row[2]));
+			e.motd                     = row[3] ? row[3] : "";
+			e.marked_npc_1_entity_id   = static_cast<uint32_t>(strtoul(row[4], nullptr, 10));
+			e.marked_npc_1_zone_id     = static_cast<uint32_t>(strtoul(row[5], nullptr, 10));
+			e.marked_npc_1_instance_id = static_cast<uint32_t>(strtoul(row[6], nullptr, 10));
+			e.marked_npc_2_entity_id   = static_cast<uint32_t>(strtoul(row[7], nullptr, 10));
+			e.marked_npc_2_zone_id     = static_cast<uint32_t>(strtoul(row[8], nullptr, 10));
+			e.marked_npc_2_instance_id = static_cast<uint32_t>(strtoul(row[9], nullptr, 10));
+			e.marked_npc_3_entity_id   = static_cast<uint32_t>(strtoul(row[10], nullptr, 10));
+			e.marked_npc_3_zone_id     = static_cast<uint32_t>(strtoul(row[11], nullptr, 10));
+			e.marked_npc_3_instance_id = static_cast<uint32_t>(strtoul(row[12], nullptr, 10));
 
 			all_entries.push_back(e);
 		}
@@ -315,13 +369,19 @@ public:
 		for (auto row = results.begin(); row != results.end(); ++row) {
 			RaidDetails e{};
 
-			e.raidid       = static_cast<int32_t>(atoi(row[0]));
-			e.loottype     = static_cast<int32_t>(atoi(row[1]));
-			e.locked       = static_cast<int8_t>(atoi(row[2]));
-			e.motd         = row[3] ? row[3] : "";
-			e.marked_npc_1 = static_cast<uint16_t>(strtoul(row[4], nullptr, 10));
-			e.marked_npc_2 = static_cast<uint16_t>(strtoul(row[5], nullptr, 10));
-			e.marked_npc_3 = static_cast<uint16_t>(strtoul(row[6], nullptr, 10));
+			e.raidid                   = static_cast<int32_t>(atoi(row[0]));
+			e.loottype                 = static_cast<int32_t>(atoi(row[1]));
+			e.locked                   = static_cast<int8_t>(atoi(row[2]));
+			e.motd                     = row[3] ? row[3] : "";
+			e.marked_npc_1_entity_id   = static_cast<uint32_t>(strtoul(row[4], nullptr, 10));
+			e.marked_npc_1_zone_id     = static_cast<uint32_t>(strtoul(row[5], nullptr, 10));
+			e.marked_npc_1_instance_id = static_cast<uint32_t>(strtoul(row[6], nullptr, 10));
+			e.marked_npc_2_entity_id   = static_cast<uint32_t>(strtoul(row[7], nullptr, 10));
+			e.marked_npc_2_zone_id     = static_cast<uint32_t>(strtoul(row[8], nullptr, 10));
+			e.marked_npc_2_instance_id = static_cast<uint32_t>(strtoul(row[9], nullptr, 10));
+			e.marked_npc_3_entity_id   = static_cast<uint32_t>(strtoul(row[10], nullptr, 10));
+			e.marked_npc_3_zone_id     = static_cast<uint32_t>(strtoul(row[11], nullptr, 10));
+			e.marked_npc_3_instance_id = static_cast<uint32_t>(strtoul(row[12], nullptr, 10));
 
 			all_entries.push_back(e);
 		}

common/repositories/bot_starting_items_repository.h

@@ -0,0 +1,50 @@
+#ifndef EQEMU_BOT_STARTING_ITEMS_REPOSITORY_H
+#define EQEMU_BOT_STARTING_ITEMS_REPOSITORY_H
+
+#include "../database.h"
+#include "../strings.h"
+#include "base/base_bot_starting_items_repository.h"
+
+class BotStartingItemsRepository: public BaseBotStartingItemsRepository {
+public:
+
+    /**
+     * This file was auto generated and can be modified and extended upon
+     *
+     * Base repository methods are automatically
+     * generated in the "base" version of this repository. The base repository
+     * is immutable and to be left untouched, while methods in this class
+     * are used as extension methods for more specific persistence-layer
+     * accessors or mutators.
+     *
+     * Base Methods (Subject to be expanded upon in time)
+     *
+     * Note: Not all tables are designed appropriately to fit functionality with all base methods
+     *
+     * InsertOne
+     * UpdateOne
+     * DeleteOne
+     * FindOne
+     * GetWhere(std::string where_filter)
+     * DeleteWhere(std::string where_filter)
+     * InsertMany
+     * All
+     *
+     * Example custom methods in a repository
+     *
+     * BotStartingItemsRepository::GetByZoneAndVersion(int zone_id, int zone_version)
+     * BotStartingItemsRepository::GetWhereNeverExpires()
+     * BotStartingItemsRepository::GetWhereXAndY()
+     * BotStartingItemsRepository::DeleteWhereXAndY()
+     *
+     * Most of the above could be covered by base methods, but if you as a developer
+     * find yourself re-using logic for other parts of the code, its best to just make a
+     * method that can be re-used easily elsewhere especially if it can use a base repository
+     * method and encapsulate filters there
+     */
+
+	// Custom extended repository methods here
+
+};
+
+#endif //EQEMU_BOT_STARTING_ITEMS_REPOSITORY_H

common/repositories/character_stats_record_repository.h

@@ -0,0 +1,50 @@
+#ifndef EQEMU_CHARACTER_STATS_RECORD_REPOSITORY_H
+#define EQEMU_CHARACTER_STATS_RECORD_REPOSITORY_H
+
+#include "../database.h"
+#include "../strings.h"
+#include "base/base_character_stats_record_repository.h"
+
+class CharacterStatsRecordRepository: public BaseCharacterStatsRecordRepository {
+public:
+
+    /**
+     * This file was auto generated and can be modified and extended upon
+     *
+     * Base repository methods are automatically
+     * generated in the "base" version of this repository. The base repository
+     * is immutable and to be left untouched, while methods in this class
+     * are used as extension methods for more specific persistence-layer
+     * accessors or mutators.
+     *
+     * Base Methods (Subject to be expanded upon in time)
+     *
+     * Note: Not all tables are designed appropriately to fit functionality with all base methods
+     *
+     * InsertOne
+     * UpdateOne
+     * DeleteOne
+     * FindOne
+     * GetWhere(std::string where_filter)
+     * DeleteWhere(std::string where_filter)
+     * InsertMany
+     * All
+     *
+     * Example custom methods in a repository
+     *
+     * CharacterStatsRecordRepository::GetByZoneAndVersion(int zone_id, int zone_version)
+     * CharacterStatsRecordRepository::GetWhereNeverExpires()
+     * CharacterStatsRecordRepository::GetWhereXAndY()
+     * CharacterStatsRecordRepository::DeleteWhereXAndY()
+     *
+     * Most of the above could be covered by base methods, but if you as a developer
+     * find yourself re-using logic for other parts of the code, its best to just make a
+     * method that can be re-used easily elsewhere especially if it can use a base repository
+     * method and encapsulate filters there
+     */
+
+	// Custom extended repository methods here
+
+};
+
+#endif //EQEMU_CHARACTER_STATS_RECORD_REPOSITORY_H

common/repositories/items_repository.h

@@ -53,7 +53,7 @@ public:
 		auto query = fmt::format(
 			"SELECT `id` FROM {} WHERE LOWER(`name`) LIKE '%%{}%%' ORDER BY id ASC",
 			TableName(),
-			search_string
+			Strings::Escape(search_string)
 		);
 
 		if (query_limit >= 1) {

common/repositories/raid_details_repository.h

@@ -47,17 +47,21 @@ public:
 	static int UpdateRaidMarkedNPC(
 		Database& db,
 		int32_t raid_id,
-		uint8_t marked_npc_number,
-		uint8_t value
+		uint32_t marked_npc_entity_id,
+		uint32_t marked_npc_zone_id,
+		uint32_t marked_npc_instance_id,
+		uint32_t slot_number
 	) {
 		auto results = db.QueryDatabase(
 			fmt::format(
-				"UPDATE `{}` SET `marked_npc_{}` = '{}' WHERE raidid = '{}';",
+				"UPDATE `{0}` SET `marked_npc_{4}_entity_id` = '{1}',`marked_npc_{4}_zone_id` = '{2}',`marked_npc_{4}_instance_id` = '{3}' WHERE raidid = '{5}';",
 				TableName(),
-				marked_npc_number,
-				value,
+				marked_npc_entity_id,
+				marked_npc_zone_id,
+				marked_npc_instance_id,
+				slot_number,
 				raid_id
-			)
+				)
 		);
 
 		return results.Success() ? results.RowsAffected() : 0;

common/ruletypes.h

@@ -58,6 +58,7 @@ RULE_REAL(Character, ExpMultiplier, 0.5, "If greater than 0, the experience gain
 RULE_REAL(Character, AAExpMultiplier, 0.5, "If greater than 0, the AA experience gained is multiplied by this value. ")
 RULE_REAL(Character, GroupExpMultiplier, 0.5, "The experience in a group is multiplied by this value in addition to the group multiplier. The group multiplier is: 2 members=x 1.2, 3=x1.4, 4=x1.6, 5=x1.8, 6=x2.16")
 RULE_REAL(Character, RaidExpMultiplier, 0.2, "The experience gained in raids is multiplied by (1-RaidExpMultiplier) ")
+RULE_REAL(Character, FinalRaidExpMultiplier, 1.0, "Multiplies all raid experience by this value. Default: 1.0")
 RULE_BOOL(Character, UseXPConScaling, true, "When activated, the experience is modified depending on the difference between player level and NPC level. The values from the rules GreenModifier to RedModifier are used")
 RULE_INT(Character, ShowExpValues, 0, "Show experience values. 0=normal, 1=show raw experience values, 2=show raw experience values and percent")
 RULE_INT(Character, GreenModifier, 20, "The experience obtained for green con mobs is multiplied by value/100")
@@ -256,6 +257,7 @@ RULE_REAL(Pets, AttackCommandRange, 150, "Range at which a pet will respond to a
 RULE_BOOL(Pets, UnTargetableSwarmPet, false, "Setting whether swarm pets should be targetable")
 RULE_REAL(Pets, PetPowerLevelCap, 10, "Maximum number of levels a player pet can go up with pet power")
 RULE_BOOL(Pets, CanTakeNoDrop, false, "Setting whether anyone can give no-drop items to pets")
+RULE_BOOL(Pets, CanTakeQuestItems, true, "Setting whether anyone can give quest items to pets")
 RULE_BOOL(Pets, LivelikeBreakCharmOnInvis, true, "Default: true will break charm on any type of invis (hide/ivu/iva/etc) false will only break if the pet can not see you (ex. you have an undead pet and cast IVU")
 RULE_BOOL(Pets, ClientPetsUseOwnerNameInLastName, true, "Disable this to keep client pet's last names from being owner_name's pet")
 RULE_CATEGORY_END()
@@ -287,7 +289,8 @@ RULE_BOOL(World, GMAccountIPList, false, "Check IP list against GM accounts. Thi
 RULE_INT(World, MinGMAntiHackStatus, 1, "Minimum status to check against AntiHack list")
 RULE_INT(World, SoFStartZoneID, -1, "Sets the Starting Zone for SoF Clients separate from Titanium Clients (-1 is disabled)")
 RULE_INT(World, TitaniumStartZoneID, -1, "Sets the Starting Zone for Titanium Clients (-1 is disabled). Replaces the old method")
-RULE_INT(World, ExpansionSettings, 16383, "Sets the expansion settings for the server, This is sent on login to world and affects client expansion settings. Defaults to all expansions enabled up to TSS, value is bitmask")
+RULE_INT(World, ExpansionSettings, 16383, "Sets the expansion settings bitmask for the server, This is sent on login to world and affects client expansion settings. Defaults to all expansions enabled up to TSS, value is bitmask")
+RULE_INT(World, CharacterSelectExpansionSettings, -1, "Sets the expansion settings bitmask for character select if you wish to override. -1 is off")
 RULE_BOOL(World, UseClientBasedExpansionSettings, true, "If true it will overrule World, ExpansionSettings and set someone's expansion based on the client they're using")
 RULE_INT(World, PVPSettings, 0, "Sets the PVP settings for the server. 1=Rallos Zek RuleSet, 2=Tallon/Vallon Zek Ruleset, 4=Sullon Zek Ruleset, 6=Discord Ruleset, anything above 6 is the Discord Ruleset without the no-drop restrictions removed. NOTE: edit IsAttackAllowed in Zone-table to accomodate for these rules")
 RULE_INT(World, PVPMinLevel, 0, "Minimum level to pvp")
@@ -449,10 +452,14 @@ RULE_BOOL(Spells, TargetsTargetRequiresCombatRange, true, "Disable to remove com
 RULE_BOOL(Spells, NPCBuffLevelRestrictions, false, "Impose BuffLevelRestrictions on NPCs if true")
 RULE_INT(Spells, ResurrectionEffectBlock, 2, "0 = allow overwrites/rule disabled. If set to 1 = Block all buffs that would overwrite Resurrection Effects. If set to 2 = Will not overwrite Resurrection Effects, instead moves new buff to an empty slot if available. Default is 2.")
 RULE_BOOL(Spells, WaterMatchRequiredForLoS, true, "Enable/Disable the requirement of both the attacker/victim being both in or out of water for spells LoS to pass.")
+RULE_INT(Spells, WizardCritMinimumRandomRatio, 20, "The minimum value for the random range which Wizards and Caster DPS Mercs innately have for spell crit ratio. Set to 20 for vanilla values.")
+RULE_INT(Spells, WizardCritMaximumRandomRatio, 70, "The maximum value for the random range which Wizards and Caster DPS Mercs innately have for spell crit ratio. Set to 70 for vanilla values.")
+RULE_INT(Spells, DefensiveProcPenaltyLevelGap, 6, "Defensive Proc Penalty Level Gap where procs start losing their proc rate at RuleR(Spells, DefensiveProcPenaltyModifier)% per level difference")
+RULE_REAL(Spells, DefensiveProcPenaltyLevelGapModifier, 10.0f, "Defensive Proc Penalty Level Gap Modifier where procs start losing their proc rate at defined % after RuleI(Spells, DefensiveProcLevelGap) level difference")
 RULE_CATEGORY_END()
 
 RULE_CATEGORY(Combat)
-RULE_REAL(Combat, AERampageSafeZone, 0.018, "max hit ae ramp reduction range")
+RULE_REAL(Combat, AERampageMaxDistance, 70, "Max AERampage range (% of max combat distance)")
 RULE_INT(Combat, PetBaseCritChance, 0, "Pet base crit chance")
 RULE_INT(Combat, NPCBashKickLevel, 6, "The level that NPCcan KICK/BASH")
 RULE_INT(Combat, MeleeCritDifficulty, 8900, "Value against which is rolled to check if a melee crit is triggered. Lower is easier")
@@ -523,6 +530,8 @@ RULE_BOOL(Combat, BackstabIgnoresElemental, false, "Enable or disable Elemental
 RULE_BOOL(Combat, BackstabIgnoresBane, false, "Enable or disable Bane weapon damage affecting backstab damage, false by default.")
 RULE_BOOL(Combat, SummonMeleeRange, true, "Enable or disable summoning of a player when already in melee range of the summoner.")
 RULE_BOOL(Combat, WaterMatchRequiredForAutoFireLoS, true, "Enable/Disable the requirement of both the attacker/victim being both in or out of water for AutoFire LoS to pass.")
+RULE_INT(Combat, ExtraAllowedKickClassesBitmask, 0, "Bitmask for allowing extra classes beyond Warrior, Ranger, Beastlord, and Berserker to kick, No Extra Classes (0) by default")
+RULE_INT(Combat, MaxProcs, 4, "Adjustable maximum number of procs per round, the hard cap is MAX_PROCS (11). Requires mob repop or client zone when changed")
 RULE_CATEGORY_END()
 
 RULE_CATEGORY(NPC)
@@ -540,6 +549,7 @@ RULE_INT(NPC, LastFightingDelayMovingMin, 10000, "Minimum time before mob goes h
 RULE_INT(NPC, LastFightingDelayMovingMax, 20000, "Maximum time before mob goes home after all aggro loss (milliseconds)")
 RULE_BOOL(NPC, SmartLastFightingDelayMoving, true, "When true, mobs that started going home previously will do so again immediately if still on FD hate list")
 RULE_BOOL(NPC, ReturnNonQuestNoDropItems, false, "Returns NO DROP items on NPC that don't have an EVENT_TRADE sub in their script")
+RULE_BOOL(NPC, ReturnQuestItemsFromNonQuestNPCs, false, "Returns Quest items traded to NPCs that are not flagged as a Quest NPC")
 RULE_INT(NPC, StartEnrageValue, 9, " Percentage HP that an NPC will begin to enrage")
 RULE_BOOL(NPC, LiveLikeEnrage, false, "If set to true then only player controlled pets will enrage")
 RULE_BOOL(NPC, EnableMeritBasedFaction, false, "If set to true, faction will be given in the same way as experience (solo/group/raid)")
@@ -619,7 +629,7 @@ RULE_BOOL(Bots, FinishBuffing, false, "Allow for buffs to complete even if the b
 RULE_BOOL(Bots, GroupBuffing, false, "Bots will cast single target buffs as group buffs, default is false for single. Does not make single target buffs work for MGB")
 RULE_INT(Bots, HealRotationMaxMembers, 24, "Maximum number of heal rotation members")
 RULE_INT(Bots, HealRotationMaxTargets, 12, "Maximum number of heal rotation targets")
-RULE_REAL(Bots, ManaRegen, 2.0, "Adjust mana regen for bots, 1 is fast and higher numbers slow it down 3 is about the same as players")
+RULE_REAL(Bots, ManaRegen, 2.0, "Adjust mana regen. Acts as a final multiplier, stacks with Rule Character:ManaRegenMultiplier.")
 RULE_BOOL(Bots, PreferNoManaCommandSpells, true, "Give sorting priority to newer no-mana spells (i.e., 'Bind Affinity')")
 RULE_BOOL(Bots, QuestableSpawnLimit, false, "Optional quest method to manage bot spawn limits using the quest_globals name bot_spawn_limit, see: /bazaar/Aediles_Thrall.pl")
 RULE_INT(Bots, SpawnLimit, 71, "Number of bots a character can have spawned at one time, You + 71 bots is a 12 group pseudo-raid")
@@ -814,6 +824,7 @@ RULE_CATEGORY_END()
 RULE_CATEGORY(Expansion)
 RULE_INT(Expansion, CurrentExpansion, -1, "The current expansion enabled for the server [-1 = ALL, 0 = Classic, 1 = Kunark etc.]")
 RULE_BOOL(Expansion, UseCurrentExpansionAAOnly, false, "When true will only load AA ranks that match CurrentExpansion rule")
+RULE_INT(Expansion, AutoGrantAAExpansion, -1, "Expansion to auto grant AAs up to, [-1 = Disabled, 0 = Classic, 1 = Kunark etc.]")
 RULE_CATEGORY_END()
 
 RULE_CATEGORY(Instances)

common/say_link.cpp

@@ -376,7 +376,7 @@ std::string EQ::SayLinkEngine::InjectSaylinksIfNotExist(const char *message)
 
 void EQ::SayLinkEngine::LoadCachedSaylinks()
 {
-	auto saylinks = SaylinkRepository::GetWhere(database, "phrase not like '%#%'");
+	auto saylinks = SaylinkRepository::GetWhere(database, "phrase not REGEXP BINARY '[A-Z]' and phrase not REGEXP '[0-9]'");
 	LogSaylink("Loaded [{}] saylinks into cache", saylinks.size());
 	g_cached_saylinks = saylinks;
 }
@@ -399,6 +399,7 @@ SaylinkRepository::Saylink EQ::SayLinkEngine::GetOrSaveSaylink(std::string sayli
 
 	// return if found from the database
 	if (!saylinks.empty()) {
+		g_cached_saylinks.emplace_back(saylinks[0]);
 		return saylinks[0];
 	}
 

common/servertalk.h

@@ -10,6 +10,7 @@
 #include <cereal/types/chrono.hpp>
 #include <cereal/types/string.hpp>
 #include <cereal/types/vector.hpp>
+#include <glm/vec4.hpp>
 
 #define SERVER_TIMEOUT	45000	// how often keepalive gets sent
 #define INTERSERVER_TIMER					10000
@@ -249,6 +250,7 @@
 #define ServerOP_ReloadZonePoints 0x4122
 #define ServerOP_ReloadDzTemplates 0x4123
 #define ServerOP_ReloadZoneData 0x4124
+#define ServerOP_ReloadDataBucketsCache 0x4125
 
 #define ServerOP_CZDialogueWindow 0x4500
 #define ServerOP_CZLDoNUpdate 0x4501
@@ -286,6 +288,8 @@
 // player events
 #define ServerOP_PlayerEvent 0x5100
 
+#define ServerOP_DataBucketCacheUpdate 0x5200
+
 enum {
 	CZUpdateType_Character,
 	CZUpdateType_Group,
@@ -1073,7 +1077,7 @@ struct ServerRaidMessage_Struct {
 
 struct ServerRaidMOTD_Struct {
 	uint32 rid;
-	char motd[0];
+	char motd[1024];
 };
 
 struct ServerRaidNote_Struct {
@@ -1483,95 +1487,96 @@ struct CZClientMessageString_Struct {
 
 struct CZDialogueWindow_Struct {
 	uint8 update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name
-	int update_identifier; // Character ID, Group ID, Raid ID, Guild ID, or Expedition ID based on update type, 0 for Character Name
-	char message[4096];
-	char client_name[64]; // Only used by Character Name Type, else empty
+	int   update_identifier; // Character ID, Group ID, Raid ID, Guild ID, or Expedition ID based on update type, 0 for Character Name
+	char  message[4096];
+	char  client_name[64]; // Only used by Character Name Type, else empty
 };
 
 struct CZLDoNUpdate_Struct {
-	uint8 update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name
-	uint8 update_subtype; // 0 - Loss, 1 - Points, 2 - Win
-	int update_identifier; // Character ID, Group ID, Raid ID, Guild ID, or Expedition ID based on update type, 0 for Character Name
+	uint8  update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name
+	uint8  update_subtype; // 0 - Loss, 1 - Points, 2 - Win
+	int    update_identifier; // Character ID, Group ID, Raid ID, Guild ID, or Expedition ID based on update type, 0 for Character Name
 	uint32 theme_id;
-	int points; // Only used in Points Subtype, else 1
-	char client_name[64]; // Only used by Character Name Type, else empty
+	int    points; // Only used in Points Subtype, else 1
+	char   client_name[64]; // Only used by Character Name Type, else empty
 };
 
 struct CZMarquee_Struct {
-	uint8 update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name
-	int update_identifier; // Character ID, Group ID, Raid ID, Guild ID, or Expedition ID based on update type, 0 for Character Name
+	uint8  update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name
+	int    update_identifier; // Character ID, Group ID, Raid ID, Guild ID, or Expedition ID based on update type, 0 for Character Name
 	uint32 type;
 	uint32 priority;
 	uint32 fade_in;
 	uint32 fade_out;
 	uint32 duration;
-	char message[512];
-	char client_name[64]; // Only used by Character Name Type, else empty
+	char   message[512];
+	char   client_name[64]; // Only used by Character Name Type, else empty
 };
 
 struct CZMessage_Struct {
-	uint8 update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name
-	int update_identifier; // Group ID, Raid ID, Guild ID, or Expedition ID based on update type, 0 for Character Name
+	uint8  update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name
+	int    update_identifier; // Group ID, Raid ID, Guild ID, or Expedition ID based on update type, 0 for Character Name
 	uint32 type;
-	char message[512];
-	char client_name[64]; // Only used by Character Name Type, else empty
+	char   message[512];
+	char   client_name[64]; // Only used by Character Name Type, else empty
 };
 
 struct CZMove_Struct {
-	uint8 update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name
-	uint8 update_subtype; // 0 - Move Zone, 1 - Move Zone Instance
-	int update_identifier; // Character ID, Group ID, Raid ID, Guild ID, or Expedition ID based on update type, 0 for Character Name
-	uint16 instance_id; // Only used by Move Zone Instance, else 0
-	char zone_short_name[32]; // Only by with Move Zone, else empty
-	char client_name[64]; // Only used by Character Name Type, else empty
+	std::string client_name       = std::string(); // Only used by Character Name Type, else empty
+	glm::vec4   coordinates       = glm::vec4(0.f); // XYZ or XYZH, heading is optional, defaults to 0.
+	uint16      instance_id       = 0; // Only used by Move Zone Instance, else 0
+	uint32      update_identifier = 0; // Character ID, Group ID, Raid ID, Guild ID, or Expedition ID based on update type, 0 for Character Name
+	uint8       update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name
+	uint8       update_subtype; // 0 - Move Zone, 1 - Move Zone Instance
+	std::string zone_short_name   = std::string(); // Only used by Move Zone, else empty
 };
 
 struct CZSetEntityVariable_Struct {
 	uint8 update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name, 6 - NPC
-	int update_identifier; // Group ID, Raid ID, Guild ID, Expedition ID, or NPC ID based on update type, 0 for Character Name
-	char variable_name[256];
-	char variable_value[256];
-	char client_name[64]; // Only used by Character Type, else empty
+	int   update_identifier; // Group ID, Raid ID, Guild ID, Expedition ID, or NPC ID based on update type, 0 for Character Name
+	char  variable_name[256];
+	char  variable_value[256];
+	char  client_name[64]; // Only used by Character Type, else empty
 };
 
 struct CZSignal_Struct {
 	uint8 update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name, 6 - NPC
-	int update_identifier; // Character ID, Group ID, Raid ID, Guild ID, Expedition ID, or NPC ID based on update type, 0 for Character Name
-	int signal_id;
-	char client_name[64]; // Only used by Character Name Type, else empty
+	int   update_identifier; // Character ID, Group ID, Raid ID, Guild ID, Expedition ID, or NPC ID based on update type, 0 for Character Name
+	int   signal_id;
+	char  client_name[64]; // Only used by Character Name Type, else empty
 };
 
 struct CZSpell_Struct {
-	uint8 update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name
-	uint8 update_subtype; // 0 - Cast Spell, 1 - Remove Spell
-	int update_identifier; // Character ID, Group ID, Raid ID, Guild ID, or Expedition ID based on update type, 0 for Character Name
+	uint8  update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name
+	uint8  update_subtype; // 0 - Cast Spell, 1 - Remove Spell
+	int    update_identifier; // Character ID, Group ID, Raid ID, Guild ID, or Expedition ID based on update type, 0 for Character Name
 	uint32 spell_id;
-	char client_name[64]; // Only used by Character Name Type, else empty
+	char   client_name[64]; // Only used by Character Name Type, else empty
 };
 
 struct CZTaskUpdate_Struct {
-	uint8 update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name
-	uint8 update_subtype; // 0 - Activity Reset, 1 - Activity Update, 2 - Assign Task, 3 - Disable Task, 4 - Enable Task, 5 - Fail Task, 6 - Remove Task
-	int update_identifier; // Character ID, Group ID, Raid ID, Guild ID, or Expedition ID based on update type, 0 for Character Name
+	uint8  update_type; // 0 - Character, 1 - Group, 2 - Raid, 3 - Guild, 4 - Expedition, 5 - Character Name
+	uint8  update_subtype; // 0 - Activity Reset, 1 - Activity Update, 2 - Assign Task, 3 - Disable Task, 4 - Enable Task, 5 - Fail Task, 6 - Remove Task
+	int    update_identifier; // Character ID, Group ID, Raid ID, Guild ID, or Expedition ID based on update type, 0 for Character Name
 	uint32 task_identifier;
-	int task_subidentifier; // Activity ID for Activity Reset and Activity Update, NPC Entity ID for Assign Task, else -1
-	int update_count; // Only used by Activity Update, else 1
-	bool enforce_level_requirement; // Only used by Assign Task
-	char client_name[64]; // Only used by Character Name Type, else empty
+	int    task_subidentifier; // Activity ID for Activity Reset and Activity Update, NPC Entity ID for Assign Task, else -1
+	int    update_count; // Only used by Activity Update, else 1
+	bool   enforce_level_requirement; // Only used by Assign Task
+	char   client_name[64]; // Only used by Character Name Type, else empty
 };
 
 struct WWDialogueWindow_Struct {
-	char message[4096];
+	char  message[4096];
 	uint8 min_status;
 	uint8 max_status;
 };
 
 struct WWLDoNUpdate_Struct {
-	uint8 update_type; // 0 - Loss, 1 - Points, 2 - Win
+	uint8  update_type; // 0 - Loss, 1 - Points, 2 - Win
 	uint32 theme_id;
-	int points; // Only used in Points Subtype, else 1
-	uint8 min_status;
-	uint8 max_status;
+	int    points; // Only used in Points Subtype, else 1
+	uint8  min_status;
+	uint8  max_status;
 };
 
 struct WWMarquee_Struct {
@@ -1580,56 +1585,57 @@ struct WWMarquee_Struct {
 	uint32 fade_in;
 	uint32 fade_out;
 	uint32 duration;
-	char message[512];
-	uint8 min_status;
-	uint8 max_status;
+	char   message[512];
+	uint8  min_status;
+	uint8  max_status;
 };
 
 struct WWMessage_Struct {
 	uint32 type;
-	char message[512];
-	uint8 min_status;
-	uint8 max_status;
+	char   message[512];
+	uint8  min_status;
+	uint8  max_status;
 };
 
 struct WWMove_Struct {
-	uint8 update_type; // 0 - Move Zone, 1 - Move Zone Instance
-	char zone_short_name[32]; // Used with Move Zone
+	uint8  update_type; // 0 - Move Zone, 1 - Move Zone Instance
+	char   zone_short_name[32]; // Used with Move Zone
 	uint16 instance_id; // Used with Move Zone Instance
-	uint8 min_status;
-	uint8 max_status;
+	uint8  min_status;
+	uint8  max_status;
 };
 
+
 struct WWSetEntityVariable_Struct {
 	uint8 update_type; // 0 - Character, 1 - NPC
-	char variable_name[256];
-	char variable_value[256];
+	char  variable_name[256];
+	char  variable_value[256];
 	uint8 min_status;
 	uint8 max_status;
 };
 
 struct WWSignal_Struct {
 	uint8 update_type; // 0 - Character, 1 - NPC
-	int signal_id;
+	int   signal_id;
 	uint8 min_status;
 	uint8 max_status;
 };
 
 struct WWSpell_Struct {
-	uint8 update_type; // 0 - Cast Spell, 1 - Remove Spell
+	uint8  update_type; // 0 - Cast Spell, 1 - Remove Spell
 	uint32 spell_id;
-	uint8 min_status;
-	uint8 max_status;
+	uint8  min_status;
+	uint8  max_status;
 };
 
 struct WWTaskUpdate_Struct {
-	uint8 update_type; // 0 - Activity Reset, 1 - Activity Update, 2 - Assign Task, 3 - Disable Task, 4 - Enable Task, 5 - Fail Task, 6 - Remove Task
+	uint8  update_type; // 0 - Activity Reset, 1 - Activity Update, 2 - Assign Task, 3 - Disable Task, 4 - Enable Task, 5 - Fail Task, 6 - Remove Task
 	uint32 task_identifier;
-	int task_subidentifier; // Activity ID for Activity Reset and Activity Update, NPC Entity ID for Assign Task, else -1
-	int update_count; // Update Count for Activity Update, else 1
-	bool enforce_level_requirement; // Only used by Assign Task, else false
-	uint8 min_status;
-	uint8 max_status;
+	int    task_subidentifier; // Activity ID for Activity Reset and Activity Update, NPC Entity ID for Assign Task, else -1
+	int    update_count; // Update Count for Activity Update, else 1
+	bool   enforce_level_requirement; // Only used by Assign Task, else false
+	uint8  min_status;
+	uint8  max_status;
 };
 
 struct ReloadWorld_Struct {
@@ -1820,6 +1826,11 @@ struct ServerSendPlayerEvent_Struct {
 	char cereal_data[0];
 };
 
+struct ServerDataBucketCacheUpdate_Struct {
+	uint32_t cereal_size;
+	char cereal_data[0];
+};
+
 struct ServerFlagUpdate_Struct {
 	uint32 account_id;
 	int16 admin;

common/strings.cpp

@@ -42,10 +42,32 @@
 #include <stdio.h>
 #include <iostream>
 
+#include <random>
+#include <string>
+
 //Const char based
 #include "strings_legacy.cpp" // legacy c functions
 #include "strings_misc.cpp" // anything non "Strings" scoped
 
+std::string Strings::Random(size_t length)
+{
+	static auto &chrs = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
+
+	thread_local static std::mt19937 rg{std::random_device{}()};
+
+	thread_local static std::uniform_int_distribution<std::string::size_type> pick(0, sizeof(chrs) - 2);
+
+	std::string s;
+
+	s.reserve(length);
+
+	while (length--) {
+		s += chrs[pick(rg)];
+	}
+
+	return s;
+}
+
 std::vector<std::string> Strings::Split(const std::string &str, const char delim)
 {
 	std::vector<std::string> ret;
@@ -64,7 +86,7 @@ std::vector<std::string> Strings::Split(const std::string &str, const char delim
 }
 
 // this one takes delimiter length into consideration
-std::vector<std::string> Strings::Split(const std::string& s, const std::string& delimiter)
+std::vector<std::string> Strings::Split(const std::string &s, const std::string &delimiter)
 {
 	size_t                   pos_start = 0, pos_end, delim_len = delimiter.length();
 	std::string              token;
@@ -376,7 +398,7 @@ std::string Strings::Money(uint64 platinum, uint64 gold, uint64 silver, uint64 c
 	std::string money_string = "Unknown";
 	if (copper && silver && gold && platinum) { // CSGP
 		money_string = fmt::format(
-			"{} Platinum, {} Gold, {} Silver, and {} Copper",
+			"{} platinum, {} gold, {} silver, and {} copper",
 			Strings::Commify(std::to_string(platinum)),
 			Strings::Commify(std::to_string(gold)),
 			Strings::Commify(std::to_string(silver)),
@@ -385,7 +407,7 @@ std::string Strings::Money(uint64 platinum, uint64 gold, uint64 silver, uint64 c
 	}
 	else if (copper && silver && !gold && platinum) { // CSP
 		money_string = fmt::format(
-			"{} Platinum, {} Silver, and {} Copper",
+			"{} platinum, {} silver, and {} copper",
 			Strings::Commify(std::to_string(platinum)),
 			Strings::Commify(std::to_string(silver)),
 			Strings::Commify(std::to_string(copper))
@@ -393,7 +415,7 @@ std::string Strings::Money(uint64 platinum, uint64 gold, uint64 silver, uint64 c
 	}
 	else if (copper && silver && gold && !platinum) { // CSG
 		money_string = fmt::format(
-			"{} Gold, {} Silver, and {} Copper",
+			"{} gold, {} silver, and {} copper",
 			Strings::Commify(std::to_string(gold)),
 			Strings::Commify(std::to_string(silver)),
 			Strings::Commify(std::to_string(copper))
@@ -401,21 +423,21 @@ std::string Strings::Money(uint64 platinum, uint64 gold, uint64 silver, uint64 c
 	}
 	else if (copper && !silver && !gold && platinum) { // CP
 		money_string = fmt::format(
-			"{} Platinum and {} Copper",
+			"{} platinum and {} copper",
 			Strings::Commify(std::to_string(platinum)),
 			Strings::Commify(std::to_string(copper))
 		);
 	}
 	else if (copper && silver && !gold && !platinum) { // CS
 		money_string = fmt::format(
-			"{} Silver and {} Copper",
+			"{} silver and {} copper",
 			Strings::Commify(std::to_string(silver)),
 			Strings::Commify(std::to_string(copper))
 		);
 	}
 	else if (!copper && silver && gold && platinum) { // SGP
 		money_string = fmt::format(
-			"{} Platinum, {} Gold, and {} Silver",
+			"{} platinum, {} gold, and {} silver",
 			Strings::Commify(std::to_string(platinum)),
 			Strings::Commify(std::to_string(gold)),
 			Strings::Commify(std::to_string(silver))
@@ -423,21 +445,21 @@ std::string Strings::Money(uint64 platinum, uint64 gold, uint64 silver, uint64 c
 	}
 	else if (!copper && silver && !gold && platinum) { // SP
 		money_string = fmt::format(
-			"{} Platinum and {} Silver",
+			"{} platinum and {} silver",
 			Strings::Commify(std::to_string(platinum)),
 			Strings::Commify(std::to_string(silver))
 		);
 	}
 	else if (!copper && silver && gold && !platinum) { // SG
 		money_string = fmt::format(
-			"{} Gold and {} Silver",
+			"{} gold and {} silver",
 			Strings::Commify(std::to_string(gold)),
 			Strings::Commify(std::to_string(silver))
 		);
 	}
 	else if (copper && !silver && gold && platinum) { // CGP
 		money_string = fmt::format(
-			"{} Platinum, {} Gold, and {} Copper",
+			"{} platinum, {} gold, and {} copper",
 			Strings::Commify(std::to_string(platinum)),
 			Strings::Commify(std::to_string(gold)),
 			Strings::Commify(std::to_string(copper))
@@ -445,39 +467,39 @@ std::string Strings::Money(uint64 platinum, uint64 gold, uint64 silver, uint64 c
 	}
 	else if (copper && !silver && gold && !platinum) { // CG
 		money_string = fmt::format(
-			"{} Gold and {} Copper",
+			"{} gold and {} copper",
 			Strings::Commify(std::to_string(gold)),
 			Strings::Commify(std::to_string(copper))
 		);
 	}
 	else if (!copper && !silver && gold && platinum) { // GP
 		money_string = fmt::format(
-			"{} Platinum and {} Gold",
+			"{} platinum and {} gold",
 			Strings::Commify(std::to_string(platinum)),
 			Strings::Commify(std::to_string(gold))
 		);
 	}
 	else if (!copper && !silver && !gold && platinum) { // P
 		money_string = fmt::format(
-			"{} Platinum",
+			"{} platinum",
 			Strings::Commify(std::to_string(platinum))
 		);
 	}
 	else if (!copper && !silver && gold && !platinum) { // G
 		money_string = fmt::format(
-			"{} Gold",
+			"{} gold",
 			Strings::Commify(std::to_string(gold))
 		);
 	}
 	else if (!copper && silver && !gold && !platinum) { // S
 		money_string = fmt::format(
-			"{} Silver",
+			"{} silver",
 			Strings::Commify(std::to_string(silver))
 		);
 	}
 	else if (copper && !silver && !gold && !platinum) { // C
 		money_string = fmt::format(
-			"{} Copper",
+			"{} copper",
 			Strings::Commify(std::to_string(copper))
 		);
 	}
@@ -783,21 +805,6 @@ bool Strings::ToBool(const std::string& bool_string)
 	return false;
 }
 
-// returns a random string of specified length
-std::string Strings::Random(size_t length)
-{
-	auto        randchar = []() -> char {
-		const char   charset[] = "0123456789"
-								 "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
-								 "abcdefghijklmnopqrstuvwxyz";
-		const size_t max_index = (sizeof(charset) - 1);
-		return charset[static_cast<size_t>(std::rand()) % max_index];
-	};
-	std::string str(length, 0);
-	std::generate_n(str.begin(), length, randchar);
-	return str;
-}
-
 // a wrapper for stoi which will return a fallback if the string
 // fails to cast to a number
 int Strings::ToInt(const std::string &s, int fallback)

common/strings.h

@@ -185,7 +185,6 @@ public:
 		value = strtod(tmp_str.data(), nullptr);
 		return res;
 	}
-
 };
 
 const std::string StringFormat(const char *format, ...);

common/version.h

@@ -25,7 +25,7 @@
 
 // Build variables
 // these get injected during the build pipeline
-#define CURRENT_VERSION "22.21.0-dev" // always append -dev to the current version for custom-builds
+#define CURRENT_VERSION "22.31.1-dev" // always append -dev to the current version for custom-builds
 #define LOGIN_VERSION "0.8.0"
 #define COMPILE_DATE    __DATE__
 #define COMPILE_TIME    __TIME__
@@ -42,9 +42,9 @@
  * Manifest: https://github.com/EQEmu/Server/blob/master/utils/sql/db_update_manifest.txt
  */
 
-#define CURRENT_BINARY_DATABASE_VERSION 9233
+#define CURRENT_BINARY_DATABASE_VERSION 9240
 
-#define CURRENT_BINARY_BOTS_DATABASE_VERSION 9039
+#define CURRENT_BINARY_BOTS_DATABASE_VERSION 9040
 
 #endif
 

package.json

@@ -1,6 +1,6 @@
 {
   "name": "eqemu-server",
-  "version": "22.21.0",
+  "version": "22.31.1",
   "repository": {
     "type": "git",
     "url": "https://github.com/EQEmu/Server.git"

ucs/database.cpp

@@ -156,6 +156,11 @@ bool UCSDatabase::VerifyMailKey(const std::string& characterName, int IPAddress,
 		return false;
 	}
 
+	if (results.RowCount() == 0) {
+		LogInfo("No mailkeys found for [{}].", characterName.c_str());
+		return false;
+	}
+	
 	auto row = results.begin();
 
 	// The key is the client's IP address (expressed as 8 hex digits) and an 8 hex digit random string generated

utils/patches/patch_RoF2.conf

@@ -370,6 +370,7 @@ OP_AggroMeterTargetInfo=0x16bc
 OP_AggroMeterUpdate=0x1781
 OP_UnderWorld=0x2eb3			# clients sends up when they detect an underworld issue, might be useful for cheat detection
 OP_KickPlayers=0x6770
+OP_BookButton=0x6146
 
 # Expeditions
 OP_DzQuit=0xb2e3

utils/patches/patch_SoD.conf

@@ -360,6 +360,7 @@ OP_Marquee=0x7dc9
 OP_Fling=0x2b88
 OP_CancelSneakHide=0x7705
 OP_UnderWorld=0x51ae			# clients sends up when they detect an underworld issue, might be useful for cheat detection
+OP_BookButton=0x4348
 
 # Expedition
 OP_DzQuit=0x054e

utils/patches/patch_SoF.conf

@@ -341,6 +341,7 @@ OP_Marquee=0x2f75
 OP_Untargetable=0x3e36
 OP_CancelSneakHide=0x5335
 OP_UnderWorld=0x7580		# clients sends up when they detect an underworld issue, might be useful for cheat detection
+OP_BookButton=0x4eee
 
 #expedition
 OP_DzQuit=0x20d6

utils/patches/patch_Titanium.conf

@@ -423,6 +423,8 @@ OP_CancelInvite=0x0000
 OP_RaidJoin=0x1f21			# ShowEQ 10/27/05
 OP_RaidInvite=0x5891			# ShowEQ 10/27/05
 OP_RaidUpdate=0x1f21			# EQEmu 06/29/05
+OP_RaidDelegateAbility=0x56eb
+OP_RaidClearNPCMarks=0x1794
 
 OP_InspectBuffs=0x4FB6
 

utils/patches/patch_UF.conf

@@ -371,6 +371,7 @@ OP_Marquee=0x3675
 OP_Fling=0x51b1
 OP_CancelSneakHide=0x7686
 OP_UnderWorld=0x2d9d			# clients sends up when they detect an underworld issue, might be useful for cheat detection
+OP_BookButton=0x018e
 
 OP_DzQuit=0x1539
 OP_DzListTimers=0x21e9
@@ -533,6 +534,8 @@ OP_LFGResponse=0x0000				#
 OP_RaidInvite=0x60b5				# C
 OP_RaidUpdate=0x4d8b				# C
 OP_RaidJoin=0x0000					#
+OP_RaidDelegateAbility=0x0297
+OP_RaidClearNPCMarks=0x2af4
 
 # Button-push commands
 OP_Taunt=0x30e2						# C

utils/scripts/build/linux-build.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-set -x
+set -ex
 
 sudo chown eqemu:eqemu /drone/src/ * -R
 sudo chown eqemu:eqemu /home/eqemu/.ccache/ * -R
@@ -9,7 +9,17 @@ git submodule init && git submodule update
 
 perl utils/scripts/build/tag-version.pl
 
-mkdir -p build && cd build && cmake -DEQEMU_BUILD_TESTS=ON -DEQEMU_BUILD_STATIC=ON -DEQEMU_BUILD_LOGIN=ON -DEQEMU_BUILD_LUA=ON -DCMAKE_CXX_FLAGS_RELWITHDEBINFO:STRING="-Os" -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -G 'Unix Makefiles' .. && make -j$((`nproc`-4))
+mkdir -p build && cd build && \
+  cmake -DEQEMU_BUILD_TESTS=ON \
+      -DEQEMU_BUILD_STATIC=ON \
+      -DEQEMU_BUILD_LOGIN=ON \
+      -DEQEMU_BUILD_LUA=ON \
+      -DEQEMU_BUILD_PERL=ON \
+      -DCMAKE_CXX_FLAGS:STRING="-O1 -g" \
+      -DCMAKE_CXX_FLAGS_RELWITHDEBINFO:STRING="-O1 -g" \
+      -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
+      -G 'Unix Makefiles' \
+      .. && make -j$((`nproc`-4))
 
 curl https://raw.githubusercontent.com/Akkadius/eqemu-install-v2/master/eqemu_config.json --output eqemu_config.json
 ./bin/tests

utils/scripts/build/should-release/go.mod

@@ -10,7 +10,7 @@ require (
 require (
 	github.com/golang/protobuf v1.3.2 // indirect
 	github.com/google/go-querystring v1.1.0 // indirect
-	golang.org/x/crypto v0.1.0 // indirect
-	golang.org/x/net v0.7.0 // indirect
+	golang.org/x/crypto v0.14.0 // indirect
+	golang.org/x/net v0.17.0 // indirect
 	google.golang.org/appengine v1.6.7 // indirect
 )

utils/scripts/build/should-release/go.sum

@@ -10,12 +10,12 @@ github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD
 github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
 golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
-golang.org/x/crypto v0.1.0 h1:MDRAIl0xIo9Io2xV565hzXHw3zVseKrJKodhohM5CjU=
-golang.org/x/crypto v0.1.0/go.mod h1:RecgLatLF4+eUMCP1PoPZQb+cVrJcOPbHkTkbkB9sbw=
+golang.org/x/crypto v0.14.0 h1:wBqGXzWJW6m1XrIKlAH0Hs1JJ7+9KBwnIO8v66Q9cHc=
+golang.org/x/crypto v0.14.0/go.mod h1:MVFd36DqK4CsrnJYDkBA3VC4m2GkXAM0PvzMCn4JQf4=
 golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
 golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
-golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g=
-golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
+golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
+golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be h1:vEDujvNQGv4jgYKudGeI/+DAX4Jffq6hpD55MmoEvKs=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=

utils/scripts/generators/repository-generator.pl

@@ -112,6 +112,7 @@ if ($requested_table_to_generate ne "all") {
 }
 
 my @cereal_enabled_tables = (
+    "data_buckets",
     "player_event_logs"
 );
 

utils/scripts/import_13th_floor.pl

@@ -1,4 +1,4 @@
-#! /usr/bin/perl
+#!/usr/bin/perl
 
 ########################################################################
 #::: 13th floor import script
@@ -9,247 +9,267 @@
 use DBI;
 use DBD::mysql;
 
-my $database_name = "";
+my $db_host = "";
+my $db_port = "";
+my $db_name = "";
+my $db_user = "";
+my $db_pass = "";
 my $total_items = 0;
 my $read_items_file = "items.txt"; #default
-my $dbh = LoadMysql();
+
+read_eqemu_config_json();
+
+my $dbh = DBI->connect("DBI:mysql:database=$db_name;host=$db_host;port=$db_port", $db_user, $db_pass) or die "Cannot connect to MySql.";
 
 read_items_file_from_13th_floor_text();
 update_items_table();
 
-sub LoadMysql{	
-	#::: Config Variables
-	my $confile = "eqemu_config.xml";
-	open(F, "<$confile") or die "Unable to open config: $confile\n";
-	my $indb = 0;
-	while(<F>) {
-		s/\r//g;
-		if(/<database>/i) { $indb = 1; }
-		next unless($indb == 1);
-		if(/<\/database>/i) { $indb = 0; last; }
-		if(/<host>(.*)<\/host>/i) { $host = $1; } 
-		elsif(/<username>(.*)<\/username>/i) { $user = $1; } 
-		elsif(/<password>(.*)<\/password>/i) { $pass = $1; } 
-		elsif(/<db>(.*)<\/db>/i) { $db = $1; }
-	}
-	$database_name = $db;
-	#::: DATA SOURCE NAME
-	$dsn = "dbi:mysql:$db:localhost:3306";
-	#::: PERL DBI CONNECT
-	$connect = DBI->connect($dsn, $user, $pass);
-	return $connect;
+print "\n\nImport complete!\n\n";
+
+sub read_eqemu_config_json {
+    use JSON;
+    my $json = new JSON();
+    my $config;
+    my $config_file = "eqemu_config.json";
+
+    my $content;
+    open(my $fh, '<', $config_file) or die "cannot open file $config_file"; {
+      local $/;
+      $content = <$fh>;
+    }
+    close($fh);
+
+    $config = $json->decode($content);
+
+    $db_host = $config->{"server"}{"database"}{"host"};
+    $db_port = $config->{"server"}{"database"}{"port"};
+    $db_name = $config->{"server"}{"database"}{"db"};
+    $db_user = $config->{"server"}{"database"}{"username"};
+    $db_pass = $config->{"server"}{"database"}{"password"};
 }
 
 sub read_items_file_from_13th_floor_text {
 
-	#::: Read from file and place into array
-	open(F, "<" . $read_items_file) or die "Unable to open itemfile: " . $read_items_file . "\n";
-	my @item_file_lines = <F>;
-	close(F);
+        #::: Read from file and place into array
+        open(F, "<" . $read_items_file) or die "Unable to open itemfile: " . $read_items_file . "\n";
+        my @item_file_lines = <F>;
+        close(F);
 
-	#::: Chomp this array...
-	my @newitem_file_lines;
-	chomp($item_file_lines[0]); 
-	@fields = split("(?<!\\\\)\\|", $item_file_lines[0]);
-	
-	my $sth = $dbh->prepare("SHOW TABLES LIKE 'items_floor'");
-	$sth->execute();
-	my $has_items_floor = $sth->fetchrow_array();
-	
-	#::: If we have items_floor
-	if ($has_items_floor eq '') {
-		$dbh->do("CREATE TABLE `items_floor` (`" . join("` VARCHAR(64) NOT NULL DEFAULT '', `", @fields). "` VARCHAR(64) NOT NULL DEFAULT '', UNIQUE INDEX `ID` (`id`)) COLLATE='latin1_swedish_ci' ENGINE=MyISAM");
-		$dbh->do("ALTER TABLE `items_floor` CHANGE `id` `id` INT(11) NOT NULL DEFAULT '0'");
-		printf "Database items_floor created\n";
-	}
-	
-	#::: Create REPLACE INTO header and define worker variables...
-	$master_insert = "REPLACE INTO `items_floor` (" . join(",", @fields) . ") VALUES ";
-	$query_insert_ph = ""; #::: Used for building placeholder values in query Ex: (?, ?, ?)
-	@field_values = (); #::: Used for stuffing mysql field values
-	$query_count = 0; #::: Used for chunking query updates
-	$print_cycle = 0; #::: Counter for console updates
-	$start_time = time(); #::: Start time for import
-	$total_items_file = scalar(grep $_, @item_file_lines) - 1; #::: Total items in text file
+        #::: Chomp this array...
+        my @newitem_file_lines;
+        chomp($item_file_lines[0]);
+        @fields = split("(?<!\\\\)\\|", $item_file_lines[0]);
 
-	#::: Iterate through each item in items.txt
-	for (1 .. $#item_file_lines) {
-		@f = split("(?<!\\\\)\\|", $item_file_lines[$_]);
+        my $sth = $dbh->prepare("SHOW TABLES LIKE 'items_floor'");
+        $sth->execute();
+        my $has_items_floor = $sth->fetchrow_array();
 
-		#::: Build our individual prepared statement (?, ?) values in the insert_ph
-		#::: ?, ? placeholders will be resolved via @field_values in the execute
-		$query_insert_ph .= " (";
-		foreach (@f) {
-			push (@field_values, trim($_));
-			$query_insert_ph .= "?, ";
-		}
-		$query_insert_ph = substr($query_insert_ph, 0, -2);
-		$query_insert_ph .= "), ";
+        #::: If we don't have items_floor table
+        if ($has_items_floor eq '') {
+                $dbh->do("CREATE TABLE `items_floor` (`" . join("` VARCHAR(64) NOT NULL DEFAULT '', `", @fields). "` VARCHAR(64) NOT NULL DEFAULT '', UNIQUE INDEX `ID` (`id`)) COLLATE='latin1_swedish_ci' ENGINE=MyISAM");
+                $dbh->do("ALTER TABLE `items_floor` CHANGE `id` `id` INT(11) NOT NULL DEFAULT '0'");
+                printf "Database items_floor created\n";
+        }
 
-		#::: Let's chunk our updates so we can break up the amount of individual queries
-		if($query_count > 500){
-			$query_insert_ph = substr($query_insert_ph, 0, -2);
-			$dbh->prepare($master_insert . " " . $query_insert_ph)->execute(@field_values);
-			$query_count = 0;
-			$query_insert_ph = "";
-			@field_values = ();
-		}
-		
-		#::: Print updates to console
-		if($print_cycle > 25){
-			print "Processing (" . $read_items_file . ") :: (Items: " . $total_items . "/" . $total_items_file . ") 		\r";
-			$print_cycle = 0;
-		}
+        #::: Create REPLACE INTO header and define worker variables...
+        $master_insert = "REPLACE INTO `items_floor` (" . join(",", @fields) . ") VALUES ";
+        $query_insert_ph = ""; #::: Used for building placeholder values in query Ex: (?, ?, ?)
+        @field_values = (); #::: Used for stuffing mysql field values
+        $query_count = 0; #::: Used for chunking query updates
+        $print_cycle = 0; #::: Counter for console updates
+        $start_time = time(); #::: Start time for import
+        $total_items_file = scalar(grep $_, @item_file_lines) - 1; #::: Total items in text file
 
-		#::: Counters
-		$total_items++;
-		$query_count++;
-		$print_cycle++;
-	}
-	
-	#::: One last processing print
-	print "Processing (" . $read_items_file . ") :: (Items: " . $total_items . "/" . $total_items_file . ") 		\r";
-	
-	printf "\n" . $total_items . " items added to database... Took " . (time() - $start_time) . " second(s)... \n";
-	
-	print "Flipping slots 21 and 22...";
-	$rows_affected = $dbh->prepare("
-		UPDATE `items_floor` 
-		SET `slots` = (`slots` ^ 6291456) 
-		WHERE (`slots` & 6291456) 
-		IN (2097152, 4194304)")->execute();
-	print " Rows affected (" . $rows_affected . ")\n";
+        #::: Iterate through each item in items.txt
+        for (1 .. $#item_file_lines) {
+                @f = split("(?<!\\\\)\\|", $item_file_lines[$_]);
+
+                #::: Build our individual prepared statement (?, ?) values in the insert_ph
+                #::: ?, ? placeholders will be resolved via @field_values in the execute
+                $query_insert_ph .= " (";
+                foreach (@f) {
+                        push (@field_values, trim($_));
+                        $query_insert_ph .= "?, ";
+                }
+                $query_insert_ph = substr($query_insert_ph, 0, -2);
+                $query_insert_ph .= "), ";
+
+                #::: Let's chunk our updates so we can break up the amount of individual queries
+                if($query_count > 500){
+                        $query_insert_ph = substr($query_insert_ph, 0, -2);
+                        $dbh->prepare($master_insert . " " . $query_insert_ph)->execute(@field_values);
+                        $query_count = 0;
+                        $query_insert_ph = "";
+                        @field_values = ();
+                }
+
+                #::: Print updates to console
+                if($print_cycle > 25){
+                        print "Processing (" . $read_items_file . ") :: (Items: " . $total_items . "/" . $total_items_file . ")                 \r";
+                        $print_cycle = 0;
+                }
+
+                #::: Counters
+                $total_items++;
+                $query_count++;
+                $print_cycle++;
+        }
+
+        #::: One last processing print
+        print "Processing (" . $read_items_file . ") :: (Items: " . $total_items . "/" . $total_items_file . ")                 \r";
+
+        printf "\n" . $total_items . " items imported... Took " . (time() - $start_time) . " second(s)... \n";
+
+        #::: Process slots 21, 22
+        print "Flipping slots 21 and 22...";
+        $rows_affected = $dbh->prepare("
+                UPDATE `items_floor`
+                SET `slots` = (`slots` ^ 6291456)
+                WHERE (`slots` & 6291456)
+                IN (2097152, 4194304)")->execute();
+        print " :: Rows affected (" . $rows_affected . ")\n";
+
+        #::: Update idfile entries
+        print "Updating idfile entries...";
+        $rows_affected = $dbh->prepare("
+                UPDATE `items_floor`
+                SET `idfile` = CONCAT('IT', `idfile`)")->execute();
+        print " :: Rows affected(" . $rows_affected . ")\n";
 }
 
 sub update_items_table {
 
-	#::: Keep Items table sane
-	$query_handle = $dbh->prepare("
-		ALTER TABLE `items`
-		MODIFY COLUMN `UNK132`  text CHARACTER SET utf8 COLLATE utf8_general_ci NULL;
-	");
-	$query_handle->execute();
-	
-	my @matching_table;
-	my @missing_items_table;
-	my @missing_items_floor_table;
-	
-	#::: Get columns from `items`
-	my $sth = $dbh->prepare("SHOW COLUMNS FROM `items`;");
-	$sth->execute();
-	my @items_table;
-	while (my @row = $sth->fetchrow_array()) {
-		push(@items_table, $row[0]);
-	}
-	
-	#::: Get columns from `items_floor`
-	$sth2 = $dbh->prepare("SHOW COLUMNS FROM `items_floor`");
-	$sth2->execute();
-	my @items_floor_table;
-	while (my @row = $sth2->fetchrow_array()) {
-		push(@items_floor_table, $row[0]);
-	}
-	
-	#::: Go through the original items table columns and line them up with what columns match on 13th floor
-	#::: This is so we can use the matching columns to update and insert item data into `items` table
-	foreach $value (@items_table) {
-		if ( grep( /^$value$/i, @items_floor_table ) ) {
-			push(@matching_table, $value);
-		} else {
-			#::: What values are we missing from EMU items table..
-			push(@missing_items_table, $value);
-		}
-	}
-	
-	#::: What values are we missing from.. 13thFloor
-	foreach $value (@items_floor_table) {
-		if ( grep( /^$value$/i, @items_table ) ) {
-			#DO NOTHING...
-		} else {
-			push(@missing_items_floor_table, $value);
-		}
-	}
+        print "Updating items table...\n";
 
-	#::: Go through the matched columns and build our query strings...
-	
-	my $items_field_list = ""; #::: Build the field list for the INSERT (field1, field2)
-	my $items_floor_field_list = ""; #::: What fields we will select from items_floor table to insert into items (matched columns)
-	my $update_fields = ""; #::: To update an existing item entry if it exists...
+        my @matching_table;
+        my @missing_items_table;
+        my @missing_items_floor_table;
 
-	foreach $match (@matching_table) {
-		$match = lc($match);
-		$update_fields .= "`" . $match . "` = fi.`" . $match . "`, ";
-		$items_field_list .= "`" . $match . "`, ";
-		$items_floor_field_list .= "fi.`" . $match . "`, ";
-	}
-	#::: Trim ', ' off the ends
-	$update_fields = substr($update_fields, 0, -2);
-	$items_field_list = substr($items_field_list, 0, -2);
-	$items_floor_field_list = substr($items_floor_field_list, 0, -2);
-	
-	#::: Mixed up fields...
-	$items_floor_field_list =~ s/booktype/booklang/g; #our booktype is mixed with theirs...
-	$update_fields =~ s/`booktype` = fi.`booktype`/`booktype` = fi.`booklang`/g;
+        print "Comparing table structure...\n";
+        #::: Get columns from `items`
+        my $sth = $dbh->prepare("SHOW COLUMNS FROM `items`;");
+        $sth->execute();
+        my @items_table;
+        while (my @row = $sth->fetchrow_array()) {
+                push(@items_table, $row[0]);
+        }
 
-	#::: FIELDS THAT DO NOT MATCH GO HERE
-	my @items_add = (
-		"casttime_", "endur", "range", "attuneable", "evolvinglevel", "herosforgemodel", "scrolltype",
-		"scriptfileid", "powersourcecapacity", "augslot1unk2", "augslot2unk2", "augslot3unk2", "augslot4unk2", 
-		"augslot5unk2", "augslot6unk2", "recskill", "book"
-	);
-	my @items_floor_add = (
-		"foodduration", "endurance", "therange", "attunable", "evolvl", "heroforge1", "scrolleffecttype", 
-		"rightclickscriptid", "powersourcecap",	   "augslot1unk", "augslot2unk", "augslot3unk", "augslot4unk", 
-		"augslot5unk", "augslot6unk", "reqskill", "booktype"
-	);
-	
-	#::: Match the mis-matched fields...
-	my $spot = 0;
-	foreach $value (@items_add) {
-		$items_field_list .= ", `" . $value . "`";
-		$update_fields .= ", `" . $value . "` = fi.`" . $items_floor_add[$spot] . "`";
-		$spot++;
-		@missing_items_table = grep {$_ ne $value} @missing_items_table;
-	}
-	foreach $value (@items_floor_add) {
-		$items_floor_field_list .= ", fi.`" . $value . "`";
-		@missing_items_floor_table = grep {$_ ne $value} @missing_items_floor_table;
-	}
-	
-	my $update_query = "
-		INSERT INTO items (" . $items_field_list . ") 
-		SELECT " . $items_floor_field_list . "
-		FROM items_floor fi 
-		ON DUPLICATE KEY UPDATE " . $update_fields;
-	
-	#::: Print missing fields to file
-	my $write_file = "missing_item_fields.txt";
-	
-	open(F, ">$write_file") or die "Unable to open questfile: $write_file\n";
-	print F "$update_query \n\n";
-	print F "EQEMU items Table missing fields\n";
-	foreach $value (@missing_items_table) {
-		print F "$value\n";
-	}
-	print F "\n\n13thFloor items Table missing fields\n";
-	foreach $value (@missing_items_floor_table) {
-		print F "$value\n";
-	}
-	close(F);
-	
-	#::: Number of rows affected by query
-	$rows = $dbh->do($update_query);
-	
-	#::: Update stackables
-	$dbh->do("UPDATE items i SET i.stackable = 1 WHERE i.stacksize > 1");
-	
-	print "Added all new items to Items table (" . $rows . ")!\n";
-	
+        #::: Get columns from `items_floor`
+        $sth2 = $dbh->prepare("SHOW COLUMNS FROM `items_floor`");
+        $sth2->execute();
+        my @items_floor_table;
+        while (my @row = $sth2->fetchrow_array()) {
+                push(@items_floor_table, $row[0]);
+        }
+
+        #::: Go through the original items table columns and line them up with what columns match on 13th floor
+        #::: This is so we can use the matching columns to update and insert item data into `items` table
+        foreach $value (@items_table) {
+                if ( grep( /^$value$/i, @items_floor_table ) ) {
+                        push(@matching_table, $value);
+                } else {
+                        #::: What values are we missing from EMU items table..
+                        push(@missing_items_table, $value);
+                }
+        }
+
+        #::: What values are we missing from.. 13thFloor
+        foreach $value (@items_floor_table) {
+                if ( grep( /^$value$/i, @items_table ) ) {
+                        #DO NOTHING...
+                } else {
+                        push(@missing_items_floor_table, $value);
+                }
+        }
+
+        #::: Go through the matched columns and build our query strings...
+
+        my $items_field_list = ""; #::: Build the field list for the INSERT (field1, field2)
+        my $items_floor_field_list = ""; #::: What fields we will select from items_floor table to insert into items (matched columns)
+        my $update_fields = ""; #::: To update an existing item entry if it exists...
+
+        foreach $match (@matching_table) {
+                $match = lc($match);
+                $update_fields .= "`" . $match . "` = fi.`" . $match . "`, ";
+                $items_field_list .= "`" . $match . "`, ";
+                $items_floor_field_list .= "fi.`" . $match . "`, ";
+        }
+        #::: Trim ', ' off the ends
+        $update_fields = substr($update_fields, 0, -2);
+        $items_field_list = substr($items_field_list, 0, -2);
+        $items_floor_field_list = substr($items_floor_field_list, 0, -2);
+
+        #::: Mixed up fields...
+        $items_floor_field_list =~ s/booktype/booklang/g; #our booktype is mixed with theirs...
+        $update_fields =~ s/`booktype` = fi.`booktype`/`booktype` = fi.`booklang`/g;
+
+        #::: FIELDS THAT DO NOT MATCH GO HERE
+        my @items_add = (
+                "casttime_", "endur", "range", "attuneable", "evolvinglevel", "herosforgemodel", "scrolltype",
+                "scriptfileid", "powersourcecapacity", "augslot1unk2", "augslot2unk2", "augslot3unk2", "augslot4unk2",
+                "augslot5unk2", "augslot6unk2", "recskill", "book", "procunk1"
+        );
+        my @items_floor_add = (
+                "foodduration", "endurance", "therange", "attunable", "evolvl", "heroforge1", "scrolleffecttype",
+                "rightclickscriptid", "powersourcecap",    "augslot1unk", "augslot2unk", "augslot3unk", "augslot4unk",
+                "augslot5unk", "augslot6unk", "reqskill", "booktype", "prockunk1"
+        );
+
+        #::: Match the mis-matched fields...
+        print "Matching fields...\n";
+        my $spot = 0;
+        foreach $value (@items_add) {
+                $items_field_list .= ", `" . $value . "`";
+                $update_fields .= ", `" . $value . "` = fi.`" . $items_floor_add[$spot] . "`";
+                $spot++;
+                @missing_items_table = grep {$_ ne $value} @missing_items_table;
+        }
+        foreach $value (@items_floor_add) {
+                $items_floor_field_list .= ", fi.`" . $value . "`";
+                @missing_items_floor_table = grep {$_ ne $value} @missing_items_floor_table;
+        }
+
+        my $update_query = "
+                INSERT INTO items (" . $items_field_list . ")
+                SELECT " . $items_floor_field_list . "
+                FROM items_floor fi
+                ON DUPLICATE KEY UPDATE " . $update_fields;
+
+        #::: Print missing fields to file
+        print "Writing query and discrepencies to file...\n";
+        my $write_file = "missing_item_fields.txt";
+
+        open(F, ">$write_file") or die "Unable to open file: $write_file\n";
+        print F "$update_query \n\n";
+        print F "EQEMU items table extra fields:\n";
+        foreach $value (@missing_items_table) {
+                print F "$value\n";
+        }
+        print F "\n\n13thFloor items table extra fields:\n";
+        foreach $value (@missing_items_floor_table) {
+                print F "$value\n";
+        }
+        close(F);
+
+        #::: Number of rows affected by query
+        $rows = $dbh->do($update_query);
+        print "Added or updated " . $rows . " entries.\n";
+
+        #::: Update stackables
+        print "Updating stackable field...\n";
+        $dbh->do("UPDATE items i SET i.stackable = 1 WHERE i.stacksize > 1");
+
+        #::: Update legacy research tome bagtypes
+        print "Updating legacy research tomes...\n";
+        $dbh->do("UPDATE items i SET i.bagtype = 24 WHERE i.id IN (17655, 17903)"); #RESEARCHWIZ
+        $dbh->do("UPDATE items i SET i.bagtype = 25 WHERE i.id IN (17502, 17653)"); #RESEARCHMAG
+        $dbh->do("UPDATE items i SET i.bagtype = 26 WHERE i.id IN (17501, 17654)"); #RESEARCHNEC
+        $dbh->do("UPDATE items i SET i.bagtype = 27 WHERE i.id IN (17500, 17652)"); #RESEARCHENC
 }
 
 sub trim($) {
-	my $string = shift;
-	$string =~ s/^\s+//;
-	$string =~ s/\s+$//;
-	return $string;
-}
+        my $string = shift;
+        $string =~ s/^\s+//;
+        $string =~ s/\s+$//;
+        return $string;
+}

utils/sql/peq-dump/peq-dump.sh

@@ -44,7 +44,7 @@ echo "Generating [create_*] table exports..."
 bash -c "${world_bin} database:dump --login-tables --table-structure-only --dump-output-to-console | sed 's/ AUTO_INCREMENT=[0-9]*\b//g' > ${dump_path}create_tables_login.sql"
 bash -c "${world_bin} database:dump --player-tables --table-structure-only --dump-output-to-console | sed 's/ AUTO_INCREMENT=[0-9]*\b//g' > ${dump_path}create_tables_player.sql"
 bash -c "${world_bin} database:dump --state-tables --table-structure-only --dump-output-to-console | sed 's/ AUTO_INCREMENT=[0-9]*\b//g' > ${dump_path}create_tables_state.sql"
-echo 'REPLACE INTO `instance_list` VALUES (1,25,1,1,0,0,1),(2,25,2,1,0,0,1),(3,151,1,1,0,0,1),(4,114,1,1,0,0,1),(5,344,1,1,0,0,1),(6,202,0,1,0,0,1);' >> "${dump_path}create_tables_state.sql"
+bash -c "${world_bin} database:dump --static-instance-data --dump-output-to-console >> ${dump_path}create_tables_state.sql"
 bash -c "${world_bin} database:dump --query-serv-tables --table-structure-only --dump-output-to-console | sed 's/ AUTO_INCREMENT=[0-9]*\b//g' > ${dump_path}create_tables_queryserv.sql"
 
 # with content

world/cli/database_dump.cpp

@@ -15,6 +15,7 @@ void WorldserverCLI::DatabaseDump(int argc, char **argv, argh::parser &cmd, std:
 		"--state-tables",
 		"--system-tables",
 		"--query-serv-tables",
+		"--static-instance-data",
 		"--table-structure-only",
 		"--table-lock",
 		"--dump-path=",
@@ -51,6 +52,7 @@ void WorldserverCLI::DatabaseDump(int argc, char **argv, argh::parser &cmd, std:
 	s->SetDumpWithCompression(cmd[{"--compress"}]);
 	s->SetDumpOutputToConsole(cmd[{"--dump-output-to-console"}]);
 	s->SetDumpDropTableSyntaxOnly(cmd[{"--drop-table-syntax-only"}]);
+	s->SetDumpStaticInstanceData(cmd[{"--static-instance-data"}]);
 
 	s->DatabaseDump();
 }

world/client.cpp

@@ -208,11 +208,14 @@ void Client::SendExpansionInfo() {
 	auto outapp = new EQApplicationPacket(OP_ExpansionInfo, sizeof(ExpansionInfo_Struct));
 	ExpansionInfo_Struct *eis = (ExpansionInfo_Struct*)outapp->pBuffer;
 
-	if (RuleB(World, UseClientBasedExpansionSettings)) {
+	if (RuleI(World, CharacterSelectExpansionSettings) != -1) {
+		eis->Expansions = RuleI(World, CharacterSelectExpansionSettings);
+	}
+	else if (RuleB(World, UseClientBasedExpansionSettings)) {
 		eis->Expansions = EQ::expansions::ConvertClientVersionToExpansionsMask(eqs->ClientVersion());
 	}
 	else {
-		eis->Expansions = (RuleI(World, ExpansionSettings) & EQ::expansions::ConvertClientVersionToExpansionsMask(eqs->ClientVersion()));
+		eis->Expansions = RuleI(World, ExpansionSettings);
 	}
 
 	QueuePacket(outapp);

world/console.cpp

@@ -1226,15 +1226,20 @@ void ConsoleCrossZoneMove(
 	}
 
 	auto pack = new ServerPacket(ServerOP_CZMove, sizeof(CZMove_Struct));
-	auto* CZM = (CZMove_Struct*) pack->pBuffer;
+	auto m = (CZMove_Struct*) pack->pBuffer;
 
-	CZM->update_type       = update_type;
-	CZM->update_subtype    = !instance_id ? CZMoveUpdateSubtype_MoveZone : CZMoveUpdateSubtype_MoveZoneInstance;
-	CZM->update_identifier = update_identifier;
-	CZM->instance_id       = instance_id;
+	if (!name.empty()) {
+		m->client_name = name;
+	}
 
-	strn0cpy(CZM->zone_short_name, zone_short_name.c_str(), sizeof(CZM->zone_short_name));
-	strn0cpy(CZM->client_name, name.c_str(), sizeof(CZM->client_name));
+	m->instance_id       = instance_id;
+	m->update_identifier = update_identifier;
+	m->update_type       = update_type;
+	m->update_subtype    = !instance_id ? CZMoveUpdateSubtype_MoveZone : CZMoveUpdateSubtype_MoveZoneInstance;
+
+	if (!zone_short_name.empty()) {
+		m->zone_short_name = zone_short_name;
+	}
 
 	zoneserver_list.SendPacket(pack);
 	safe_delete(pack);

world/eqemu_api_world_data_service.cpp

@@ -138,6 +138,7 @@ std::vector<Reload> reload_types = {
 	Reload{.command = "alternate_currencies", .opcode = ServerOP_ReloadAlternateCurrencies, .desc = "Alternate Currencies"},
 	Reload{.command = "blocked_spells", .opcode = ServerOP_ReloadBlockedSpells, .desc = "Blocked Spells"},
 	Reload{.command = "commands", .opcode = ServerOP_ReloadCommands, .desc = "Commands"},
+	Reload{.command = "data_buckets_cache", .opcode = ServerOP_ReloadDataBucketsCache, .desc = "Data Buckets Cache"},
 	Reload{.command = "doors", .opcode = ServerOP_ReloadDoors, .desc = "Doors"},
 	Reload{.command = "dztemplates", .opcode = ServerOP_ReloadDzTemplates, .desc = "Dynamic Zone Templates"},
 	Reload{.command = "ground_spawns", .opcode = ServerOP_ReloadGroundSpawns, .desc = "Ground Spawns"},

world/zoneserver.cpp

@@ -46,6 +46,7 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 #include "../common/repositories/player_event_logs_repository.h"
 #include "../common/events/player_event_logs.h"
 #include "../common/patches/patches.h"
+#include "../zone/data_bucket.h"
 
 extern ClientList client_list;
 extern GroupLFPList LFPGroupList;
@@ -794,11 +795,29 @@ void ZoneServer::HandleMessage(uint16 opcode, const EQ::Net::Packet &p) {
 				client = client_list.FindCharacter(ztz->name);
 			}
 
-			LogInfo("ZoneToZone request for [{}] current zone [{}] req zone [{}]", ztz->name, ztz->current_zone_id, ztz->requested_zone_id);
+			LogZoning(
+				"ZoneToZone request for client [{}] guild_id [{}] requested_zone [{}] requested_zone_id [{}] requested_instance_id [{}] current_zone [{}] current_zone_id [{}] current_instance_id [{}] response [{}] admin [{}] ignorerestrictions [{}]",
+				ztz->name,
+				ztz->guild_id,
+				ZoneName(ztz->requested_zone_id),
+				ztz->requested_zone_id,
+				ztz->requested_instance_id,
+				ZoneName(ztz->current_zone_id),
+				ztz->current_zone_id,
+				ztz->current_instance_id,
+				ztz->response,
+				ztz->admin,
+				ztz->ignorerestrictions
+			);
 
 			/* This is a request from the egress zone */
 			if (GetZoneID() == ztz->current_zone_id && GetInstanceID() == ztz->current_instance_id) {
-				LogInfo("Processing ZTZ for egress from zone for client [{}]", ztz->name);
+				LogZoning(
+					"ZoneToZone request for client [{}] for egress from zone [{}]",
+					ztz->name,
+					ZoneName(ztz->current_zone_id),
+					ztz->current_zone_id
+				);
 
 				if (
 					ztz->admin < AccountStatus::QuestTroupe &&
@@ -806,6 +825,14 @@ void ZoneServer::HandleMessage(uint16 opcode, const EQ::Net::Packet &p) {
 					zoneserver_list.IsZoneLocked(ztz->requested_zone_id)
 				) {
 					ztz->response = 0;
+
+					LogZoning(
+						"ZoneToZone request for client [{}] for egress from zone [{}] denied, zone is locked",
+						ztz->name,
+						ZoneName(ztz->current_zone_id),
+						ztz->current_zone_id
+					);
+
 					SendPacket(pack);
 					break;
 				}
@@ -817,12 +844,25 @@ void ZoneServer::HandleMessage(uint16 opcode, const EQ::Net::Packet &p) {
 				);
 
 				if (ingress_server) {
-					LogInfo("Found a zone already booted for [{}]", ztz->name);
+					LogZoning(
+						"Found a zone already booted for ZoneToZone for client [{}] for ingress_server from zone [{}] found booted zone",
+						ztz->name,
+						ZoneName(ztz->current_zone_id),
+						ztz->current_zone_id
+					);
+
 					ztz->response = 1;
 				} else {
 					int server_id;
 					if ((server_id = zoneserver_list.TriggerBootup(ztz->requested_zone_id, ztz->requested_instance_id))) {
-						LogInfo("Successfully booted a zone for [{}]", ztz->name);
+						LogZoning(
+							"ZoneToZone successfully booted a zone for character [{}] zone [{}] ({}) instance [{}] ({})",
+							ztz->name,
+							ZoneName(ztz->requested_zone_id),
+							ztz->requested_zone_id,
+							ztz->requested_instance_id,
+							server_id
+						);
 						ztz->response = 1;
 						ingress_server = zoneserver_list.FindByID(server_id);
 					} else {
@@ -840,8 +880,8 @@ void ZoneServer::HandleMessage(uint16 opcode, const EQ::Net::Packet &p) {
 					ingress_server->SendPacket(pack);	// inform target server
 				}
 			} else {
-				LogInfo(
-					"Processing ZTZ for ingress to zone for client [{}] instance_id [{}] zone_id [{}]",
+				LogZoning(
+					"Processing ZTZ for egress to zone for client [{}] instance_id [{}] zone_id [{}]",
 					ztz->name,
 					ztz->current_instance_id,
 					ztz->current_zone_id
@@ -853,7 +893,13 @@ void ZoneServer::HandleMessage(uint16 opcode, const EQ::Net::Packet &p) {
 				);
 
 				if (egress_server) {
-					LogInfo("Found egress server, forwarding client");
+					LogZoning(
+						"Found egress server_id [{}] zone_id [{}] zone_name [{}] instance_id [{}], forwarding client",
+						egress_server->GetID(),
+						egress_server->GetZoneID(),
+						egress_server->GetZoneName(),
+						egress_server->GetInstanceID()
+					);
 					egress_server->SendPacket(pack);
 				}
 			}
@@ -1342,13 +1388,13 @@ void ZoneServer::HandleMessage(uint16 opcode, const EQ::Net::Packet &p) {
 		case ServerOP_ReloadBlockedSpells:
 		case ServerOP_ReloadCommands:
 		case ServerOP_ReloadDoors:
+		case ServerOP_ReloadDataBucketsCache:
 		case ServerOP_ReloadGroundSpawns:
 		case ServerOP_ReloadLevelEXPMods:
 		case ServerOP_ReloadMerchants:
 		case ServerOP_ReloadNPCEmotes:
 		case ServerOP_ReloadObjects:
 		case ServerOP_ReloadPerlExportSettings:
-		case ServerOP_ReloadRules:
 		case ServerOP_ReloadStaticZoneData:
 		case ServerOP_ReloadTitles:
 		case ServerOP_ReloadTraps:
@@ -1373,6 +1419,11 @@ void ZoneServer::HandleMessage(uint16 opcode, const EQ::Net::Packet &p) {
 			zoneserver_list.SendPacket(pack);
 			break;
 		}
+		case ServerOP_ReloadRules: {
+			zoneserver_list.SendPacket(pack);
+			RuleManager::Instance()->LoadRules(&database, "default", true);
+			break;
+		}
 		case ServerOP_ReloadContentFlags: {
 			zoneserver_list.SendPacket(pack);
 			content_service.SetExpansionContext()->ReloadContentFlags();
@@ -1468,6 +1519,11 @@ void ZoneServer::HandleMessage(uint16 opcode, const EQ::Net::Packet &p) {
 			DynamicZone::HandleZoneMessage(pack);
 			break;
 		}
+		case ServerOP_DataBucketCacheUpdate: {
+			zoneserver_list.SendPacket(pack);
+
+			break;
+		}
 		default: {
 			LogInfo("Unknown ServerOPcode from zone {:#04x}, size [{}]", pack->opcode, pack->size);
 			DumpPacket(pack->pBuffer, pack->size);

zone/CMakeLists.txt

@@ -133,6 +133,8 @@ SET(zone_sources
     quest_parser_collection.cpp
     raids.cpp
     raycast_mesh.cpp
+    sidecar_api/sidecar_api.cpp
+    sidecar_api/loot_simulator_controller.cpp
     shared_task_zone_messaging.cpp
     spawn2.cpp
     spawn2.h
@@ -253,6 +255,7 @@ SET(zone_headers
     quest_parser_collection.h
     raids.h
     raycast_mesh.h
+    sidecar_api/sidecar_api.h
     shared_task_zone_messaging.h
     spawn2.cpp
     spawn2.h
@@ -273,8 +276,9 @@ SET(zone_headers
     zone_config.h
     zonedb.h
     zonedump.h
+    zone_cli.h
     zone_reload.h
-)
+    zone_cli.cpp)
 
 ADD_EXECUTABLE(zone ${zone_sources} ${zone_headers})
 

zone/aa.cpp

@@ -1111,7 +1111,7 @@ void Client::PurchaseAlternateAdvancementRank(int rank_id) {
 		return;
 	}
 
-	FinishAlternateAdvancementPurchase(rank, false);
+	FinishAlternateAdvancementPurchase(rank, false, true);
 }
 
 bool Client::GrantAlternateAdvancementAbility(int aa_id, int points, bool ignore_cost) {
@@ -1134,13 +1134,13 @@ bool Client::GrantAlternateAdvancementAbility(int aa_id, int points, bool ignore
 		}
 
 		ret = true;
-		FinishAlternateAdvancementPurchase(rank, ignore_cost);
+		FinishAlternateAdvancementPurchase(rank, ignore_cost, true);
 	}
 
 	return ret;
 }
 
-void Client::FinishAlternateAdvancementPurchase(AA::Rank *rank, bool ignore_cost) {
+void Client::FinishAlternateAdvancementPurchase(AA::Rank *rank, bool ignore_cost, bool send_message_and_save) {
 	auto rank_id = rank->base_ability->first_rank_id;
 
 	if (rank->base_ability->charges) {
@@ -1156,7 +1156,7 @@ void Client::FinishAlternateAdvancementPurchase(AA::Rank *rank, bool ignore_cost
 		SetAA(rank_id, rank->current_value, 0);
 
 		//if not max then send next aa
-		if (rank->next) {
+		if (rank->next && send_message_and_save) {
 			SendAlternateAdvancementRank(rank->base_ability->id, rank->next->current_value);
 		}
 	}
@@ -1164,10 +1164,12 @@ void Client::FinishAlternateAdvancementPurchase(AA::Rank *rank, bool ignore_cost
 	auto cost = !ignore_cost ? rank->cost : 0;
 
 	m_pp.aapoints -= static_cast<uint32>(cost);
-	SaveAA();
 
-	SendAlternateAdvancementPoints();
-	SendAlternateAdvancementStats();
+	if (send_message_and_save) {
+		SaveAA();
+		SendAlternateAdvancementPoints();
+		SendAlternateAdvancementStats();
+	}
 
 	if (player_event_logs.IsEventEnabled(PlayerEvent::AA_PURCHASE)) {
 		auto e = PlayerEvent::AAPurchasedEvent{
@@ -1181,14 +1183,16 @@ void Client::FinishAlternateAdvancementPurchase(AA::Rank *rank, bool ignore_cost
 	}
 
 	if (rank->prev) {
-		MessageString(
-			Chat::Yellow,
-			AA_IMPROVE,
-			std::to_string(rank->title_sid).c_str(),
-			std::to_string(rank->prev->current_value).c_str(),
-			std::to_string(cost).c_str(),
-			cost == 1 ? std::to_string(AA_POINT).c_str() : std::to_string(AA_POINTS).c_str()
-		);
+		if (send_message_and_save) {
+			MessageString(
+				Chat::Yellow,
+				AA_IMPROVE,
+				std::to_string(rank->title_sid).c_str(),
+				std::to_string(rank->prev->current_value).c_str(),
+				std::to_string(cost).c_str(),
+				cost == 1 ? std::to_string(AA_POINT).c_str() : std::to_string(AA_POINTS).c_str()
+			);
+		}
 
 		/* QS: Player_Log_AA_Purchases */
 		if (RuleB(QueryServ, PlayerLogAAPurchases)) {
@@ -1203,13 +1207,15 @@ void Client::FinishAlternateAdvancementPurchase(AA::Rank *rank, bool ignore_cost
 			QServ->PlayerLogEvent(Player_Log_AA_Purchases, CharacterID(), event_desc);
 		}
 	} else {
-		MessageString(
-			Chat::Yellow,
-			AA_GAIN_ABILITY,
-			std::to_string(rank->title_sid).c_str(),
-			std::to_string(cost).c_str(),
-			cost == 1 ? std::to_string(AA_POINT).c_str() : std::to_string(AA_POINTS).c_str()
-		);
+		if (send_message_and_save) {
+			MessageString(
+				Chat::Yellow,
+				AA_GAIN_ABILITY,
+				std::to_string(rank->title_sid).c_str(),
+				std::to_string(cost).c_str(),
+				cost == 1 ? std::to_string(AA_POINT).c_str() : std::to_string(AA_POINTS).c_str()
+			);
+		}
 
 		/* QS: Player_Log_AA_Purchases */
 		if (RuleB(QueryServ, PlayerLogAAPurchases)) {
@@ -1570,6 +1576,10 @@ bool Mob::SetAA(uint32 rank_id, uint32 new_value, uint32 charges) {
 
 
 bool Mob::CanUseAlternateAdvancementRank(AA::Rank *rank) {
+	if (!rank) {
+		return false;
+	}
+
 	AA::Ability *ability = rank->base_ability;
 
 	if(!ability)
@@ -1594,6 +1604,15 @@ bool Mob::CanUseAlternateAdvancementRank(AA::Rank *rank) {
 		}
 	}
 
+	int expansion = RuleI(Expansion, CurrentExpansion);
+	bool use_expansion_aa = RuleB(Expansion, UseCurrentExpansionAAOnly);
+	if (use_expansion_aa && expansion >= 0) {
+		if (rank->expansion > expansion) {
+			return false;
+		}
+	}
+
+
 	if (IsClient()) {
 		if (rank->expansion && !(CastToClient()->GetPP().expansions & (1 << (rank->expansion - 1)))) {
 			return false;
@@ -1645,6 +1664,10 @@ bool Mob::CanPurchaseAlternateAdvancementRank(AA::Rank *rank, bool check_price,
 		return false;
 	}
 
+	if (IsClient() && CastToClient()->HasAlreadyPurchasedRank(rank)) {
+		return false;
+	}
+
 	//You can't purchase grant only AAs they can only be assigned
 	if(check_grant && ability->grant_only) {
 		return false;
@@ -1701,9 +1724,7 @@ bool Mob::CanPurchaseAlternateAdvancementRank(AA::Rank *rank, bool check_price,
 }
 
 void Zone::LoadAlternateAdvancement() {
-	if(!content_db.LoadAlternateAdvancementAbilities(aa_abilities,
-		aa_ranks))
-	{
+	if (!content_db.LoadAlternateAdvancementAbilities(aa_abilities, aa_ranks)) {
 		aa_abilities.clear();
 		aa_ranks.clear();
 		LogInfo("Failed to load Alternate Advancement Data");
@@ -1766,7 +1787,7 @@ bool ZoneDatabase::LoadAlternateAdvancementAbilities(std::unordered_map<int, std
 {
 	abilities.clear();
 	std::string query = "SELECT id, name, category, classes, races, deities, drakkin_heritage, status, type, charges, "
-		"grant_only, reset_on_death, first_rank_id FROM aa_ability WHERE enabled = 1";
+		"grant_only, reset_on_death, auto_grant_enabled, first_rank_id FROM aa_ability WHERE enabled = 1";
 	auto results = QueryDatabase(query);
 	if(results.Success()) {
 		for(auto row = results.begin(); row != results.end(); ++row) {
@@ -1784,7 +1805,8 @@ bool ZoneDatabase::LoadAlternateAdvancementAbilities(std::unordered_map<int, std
 			ability->charges = Strings::ToInt(row[9]);
 			ability->grant_only = Strings::ToBool(row[10]);
 			ability->reset_on_death = Strings::ToBool(row[11]);
-			ability->first_rank_id = Strings::ToInt(row[12]);
+			ability->auto_grant_enabled = Strings::ToBool(row[12]);
+			ability->first_rank_id = Strings::ToInt(row[13]);
 			ability->first = nullptr;
 
 			abilities[ability->id] = std::unique_ptr<AA::Ability>(ability);
@@ -1795,17 +1817,11 @@ bool ZoneDatabase::LoadAlternateAdvancementAbilities(std::unordered_map<int, std
 	}
 
 	LogInfo("Loaded [{}] Alternate Advancement Abilities", Strings::Commify((int)abilities.size()));
-	int expansion = RuleI(Expansion, CurrentExpansion);
-	bool use_expansion_aa = RuleB(Expansion, UseCurrentExpansionAAOnly);
-
 	ranks.clear();
-	if (use_expansion_aa && expansion >= 0) {
-		query = fmt::format("SELECT id, upper_hotkey_sid, lower_hotkey_sid, title_sid, desc_sid, cost, level_req, spell, spell_type, recast_time, "
-		"next_id, expansion FROM aa_ranks WHERE expansion <= {}", expansion);
-	} else {
-		query = "SELECT id, upper_hotkey_sid, lower_hotkey_sid, title_sid, desc_sid, cost, level_req, spell, spell_type, recast_time, "
+
+	query = "SELECT id, upper_hotkey_sid, lower_hotkey_sid, title_sid, desc_sid, cost, level_req, spell, spell_type, recast_time, "
 		"next_id, expansion FROM aa_ranks";
-	}
+
 	results = QueryDatabase(query);
 	if(results.Success()) {
 		for(auto row = results.begin(); row != results.end(); ++row) {
@@ -2070,3 +2086,124 @@ void Client::TogglePurchaseAlternativeAdvancementRank(int rank_id){
 	CalcBonuses();
 }
 
+void Client::AutoGrantAAPoints() {
+	int auto_grant_expansion = RuleI(Expansion, AutoGrantAAExpansion);
+
+	if (auto_grant_expansion == -1) {
+		return;
+	}
+
+	//iterate through every AA
+	for (auto& iter : zone->aa_abilities) {
+		auto ability = iter.second.get();
+
+		if (ability->grant_only) {
+			continue;
+		}
+
+		if (ability->charges > 0) {
+			continue;
+		}
+
+		if (!ability->auto_grant_enabled) {
+			continue;
+		}
+
+		auto level = GetLevel();
+		auto p = 1;
+		auto rank = ability->first;
+		while (rank != nullptr) {
+			if (CanUseAlternateAdvancementRank(rank)) {
+				if (rank->expansion <= auto_grant_expansion && rank->level_req <= level && !HasAlreadyPurchasedRank(rank)) {
+					FinishAlternateAdvancementPurchase(rank, true, false);
+
+					if (rank->prev) {
+						MessageString(
+							Chat::Yellow,
+							AA_IMPROVE,
+							std::to_string(rank->title_sid).c_str(),
+							std::to_string(rank->prev->current_value).c_str(),
+							"0",
+							std::to_string(AA_POINTS).c_str()
+						);
+					}
+					else {
+						MessageString(
+							Chat::Yellow,
+							AA_GAIN_ABILITY,
+							std::to_string(rank->title_sid).c_str(),
+							"0",
+							std::to_string(AA_POINTS).c_str()
+						);
+					}
+				}
+			}
+			else {
+				break;
+			}
+
+			p++;
+			rank = rank->next;
+		}
+	}
+
+	SendClearAA();
+	SendAlternateAdvancementTable();
+	SendAlternateAdvancementPoints();
+	SendAlternateAdvancementStats();
+}
+
+void Client::GrantAllAAPoints(uint8 unlock_level)
+{
+	//iterate through every AA
+	for (auto& aa : zone->aa_abilities) {
+		AA::Ability* ability = aa.second.get();
+
+		if (ability->charges > 0) {
+			continue;
+		}
+
+		const uint8 level = unlock_level ? unlock_level : GetLevel();
+
+		AA::Rank* rank = ability->first;
+		while (rank) {
+			if (!CanUseAlternateAdvancementRank(rank)) {
+				break;
+			}
+
+			if (rank->level_req <= level && !HasAlreadyPurchasedRank(rank)) {
+				FinishAlternateAdvancementPurchase(rank, true, false);
+			}
+
+			rank = rank->next;
+		}
+	}
+
+	SaveAA();
+	SendClearAA();
+	SendAlternateAdvancementTable();
+	SendAlternateAdvancementPoints();
+	SendAlternateAdvancementStats();
+}
+
+bool Client::HasAlreadyPurchasedRank(AA::Rank* rank) {
+	const auto& aa = aa_ranks.find(rank->base_ability->id);
+	if (aa == aa_ranks.end()) {
+		return false;
+	}
+
+	const auto& ability_rank = zone->GetAlternateAdvancementAbilityAndRank(aa->first, aa->second.first);
+
+	AA::Ability* ability = ability_rank.first;
+	AA::Rank*    current = ability_rank.second;
+
+	while (current) {
+		if (current == rank) {
+			return true;
+		}
+
+		current = current->prev;
+	}
+
+	return false;
+}

zone/aa_ability.h

@@ -50,6 +50,7 @@ public:
 	int status;
 	bool grant_only;
 	bool reset_on_death;
+	bool auto_grant_enabled;
 	int type;
 	int charges;
 	int first_rank_id;

zone/aggro.cpp

@@ -975,8 +975,7 @@ bool Mob::IsBeneficialAllowed(Mob *target)
 	return false;
 }
 
-bool Mob::CombatRange(Mob* other, float fixed_size_mod, bool aeRampage)
-{
+bool Mob::CombatRange(Mob* other, float fixed_size_mod, bool aeRampage, ExtraAttackOptions *opts) {
 	if (!other) {
 		return(false);
 	}
@@ -1071,18 +1070,40 @@ bool Mob::CombatRange(Mob* other, float fixed_size_mod, bool aeRampage)
 			SetPseudoRoot(false);
 		}
 	}
+	
 	if (aeRampage) {
-		float multiplyer = GetSize() * RuleR(Combat, AERampageSafeZone);
-		float ramp_range = (size_mod * multiplyer);
+		float aeramp_size = RuleR(Combat, AERampageMaxDistance);
+		
+		LogCombatDetail("AERampage: Default - aeramp_size = [{}] ", aeramp_size);
+
+		if (opts) {
+			if (opts->range_percent > 0) {
+				aeramp_size = opts->range_percent;
+				LogCombatDetail("AE Rampage: range_percent = [{}] -- aeramp_size [{}]", opts->range_percent, aeramp_size);
+			}
+		}
+
+		if (aeramp_size <= 0) {
+			aeramp_size = 0.90;
+		} else {
+			aeramp_size /= 100;
+		}
+
+		float ramp_range = size_mod * aeramp_size;
+
+		LogCombatDetail("AE Rampage: ramp_range = [{}] -- (size_mod [{}] * aeramp_size [{}])", ramp_range, size_mod, aeramp_size);
+		LogCombatDetail("AE Rampage: _DistNoRoot [{}] <= ramp_range [{}]", _DistNoRoot, ramp_range);
+		
 		if (_DistNoRoot <= ramp_range) {
+			LogCombatDetail("AE Rampage: Combat Distance returned [true]");
 			return true;
 		} else {
+			LogCombatDetail("AE Rampage: Combat Distance returned [false]");
 			return false;
 		}
 	}
 
-	if (_DistNoRoot <= size_mod)
-	{
+	if (_DistNoRoot <= size_mod) {
 		//A hack to kill an exploit till we get something better.
 		if (flymode != GravityBehavior::Flying && _zDist > 500 && !CheckLastLosState()) {
 			return false;

zone/attack.cpp

@@ -1741,7 +1741,8 @@ bool Client::Death(Mob* killerMob, int64 damage, uint16 spell, EQ::skills::Skill
 			static_cast<int>(attack_skill)
 		);
 
-		if (parse->EventPlayer(EVENT_DEATH, this, export_string, 0) != 0) {
+		std::vector<std::any> args = { CastToMob() };
+		if (parse->EventPlayer(EVENT_DEATH, this, export_string, 0, &args) != 0) {
 			if (GetHP() < 0) {
 				SetHP(0);
 			}
@@ -2388,7 +2389,8 @@ bool NPC::Death(Mob* killer_mob, int64 damage, uint16 spell, EQ::skills::SkillTy
 				static_cast<int>(attack_skill)
 			);
 
-			if (parse->EventNPC(EVENT_DEATH, this, oos, export_string, 0) != 0) {
+			std::vector<std::any> args = { CastToMob() };
+			if (parse->EventNPC(EVENT_DEATH, this, oos, export_string, 0, &args) != 0) {
 				if (GetHP() < 0) {
 					SetHP(0);
 				}
@@ -2405,7 +2407,9 @@ bool NPC::Death(Mob* killer_mob, int64 damage, uint16 spell, EQ::skills::SkillTy
 				spell,
 				static_cast<int>(attack_skill)
 			);
-			if (parse->EventBot(EVENT_DEATH, CastToBot(), oos, export_string, 0) != 0) {
+
+			std::vector<std::any> args = { CastToMob() };
+			if (parse->EventBot(EVENT_DEATH, CastToBot(), oos, export_string, 0, &args) != 0) {
 				if (GetHP() < 0) {
 					SetHP(0);
 				}
@@ -3624,7 +3628,7 @@ int64 Mob::ReduceAllDamage(int64 damage)
 
 bool Mob::HasProcs() const
 {
-	for (int i = 0; i < MAX_PROCS; i++) {
+	for (int i = 0; i < m_max_procs; i++) {
 		if (IsValidSpell(PermaProcs[i].spellID) || IsValidSpell(SpellProcs[i].spellID)) {
 			return true;
 		}
@@ -3642,7 +3646,7 @@ bool Mob::HasProcs() const
 
 bool Mob::HasDefensiveProcs() const
 {
-	for (int i = 0; i < MAX_PROCS; i++) {
+	for (int i = 0; i < m_max_procs; i++) {
 		if (IsValidSpell(DefensiveProcs[i].spellID)) {
 			return true;
 		}
@@ -3678,7 +3682,7 @@ bool Mob::HasSkillProcSuccess() const
 
 bool Mob::HasRangedProcs() const
 {
-	for (int i = 0; i < MAX_PROCS; i++){
+	for (int i = 0; i < m_max_procs; i++){
 		if (IsValidSpell(RangedProcs[i].spellID)) {
 			return true;
 		}
@@ -4226,11 +4230,11 @@ void Mob::CommonDamage(Mob* attacker, int64 &damage, const uint16 spell_id, cons
 		//this was done to simplify the code here (since we can only effectively skip one mob on queue)
 		eqFilterType filter;
 		Mob* skip = attacker;
-		if (attacker && attacker->GetOwnerID()) {
+		if (attacker && attacker->GetOwner()) {
 			//attacker is a pet, let pet owners see their pet's damage
 			Mob* owner = attacker->GetOwner();
 			if (owner && owner->IsClient()) {
-				if ((IsValidSpell(spell_id) || (FromDamageShield)) && damage > 0) {
+				if (FromDamageShield && damage > 0) {
 					//special crap for spell damage, looks hackish to me
 					char val1[20] = { 0 };
 					owner->MessageString(Chat::NonMelee, OTHER_HIT_NONMELEE, GetCleanName(), ConvertArray(damage, val1));
@@ -4249,7 +4253,15 @@ void Mob::CommonDamage(Mob* attacker, int64 &damage, const uint16 spell_id, cons
 						filter = FilterPetMisses;
 
 					if (!FromDamageShield)
-						owner->CastToClient()->QueuePacket(outapp, true, CLIENT_CONNECTED, filter);
+						entity_list.QueueCloseClients(
+							this, /* Sender */
+							outapp, /* packet */
+							false, /* Skip Sender */
+							((IsValidSpell(spell_id)) ? RuleI(Range, SpellMessages) : RuleI(Range, DamageMessages)),
+							0, /* don't skip anyone on spell */
+							true, /* Packet ACK */
+							filter /* eqFilterType filter */
+							);
 				}
 			}
 
@@ -4351,16 +4363,21 @@ void Mob::CommonDamage(Mob* attacker, int64 &damage, const uint16 spell_id, cons
 					CastToClient()->QueuePacket(outapp);
 				}
 
-				// Otherwise, send normal spell or melee message to observers.
-				entity_list.QueueCloseClients(
-					this, /* Sender */
-					outapp, /* packet */
-					true, /* Skip Sender */
-					range, /* distance packet travels at the speed of sound */
-					(IsValidSpell(spell_id) && skill_used != EQ::skills::SkillTigerClaw) ? 0 : skip,
-					true, /* Packet ACK */
-					filter /* eqFilterType filter */
-					);
+				// Send normal message to observers
+				// Exclude damage done by client pets as that's handled
+				// elsewhere using proper "my pet damage filter"
+				Mob *owner = attacker->GetOwner();
+				if (!owner || (owner && !owner->IsClient())) {
+					entity_list.QueueCloseClients(
+						this, /* Sender */
+						outapp, /* packet */
+						true, /* Skip Sender */
+						range, /* distance packet travels at the speed of sound */
+						(IsValidSpell(spell_id) && skill_used != EQ::skills::SkillTigerClaw) ? 0 : skip,
+						true, /* Packet ACK */
+						filter /* eqFilterType filter */
+						);
+				}
 			}
 		}
 
@@ -4526,8 +4543,8 @@ float Mob::GetDefensiveProcChances(float &ProcBonus, float &ProcChance, uint16 h
 }
 
 // argument 'weapon' not used
-void Mob::TryDefensiveProc(Mob *on, uint16 hand) {
-
+void Mob::TryDefensiveProc(Mob *on, uint16 hand)
+{
 	if (!on) {
 		SetTarget(nullptr);
 		LogError("A null Mob object was passed to Mob::TryDefensiveProc for evaluation!");
@@ -4539,31 +4556,34 @@ void Mob::TryDefensiveProc(Mob *on, uint16 hand) {
 	}
 
 	if (!on->HasDied() && on->GetHP() > 0) {
-
-		float ProcChance, ProcBonus;
-		on->GetDefensiveProcChances(ProcBonus, ProcChance, hand, this);
+		float proc_chance, proc_bonus;
+		on->GetDefensiveProcChances(proc_bonus, proc_chance, hand, this);
 
 		if (hand == EQ::invslot::slotSecondary) {
-			ProcChance /= 2;
+			proc_chance /= 2;
 		}
 
-		int level_penalty = 0;
-		int level_diff = GetLevel() - on->GetLevel();
-		if (level_diff > 6) {//10% penalty per level if > 6 levels over target.
-			level_penalty = (level_diff - 6) * 10;
+		int level_penalty     = 0;
+		int level_diff        = GetLevel() - on->GetLevel();
+		int penalty_level_gap = RuleI(Spells, DefensiveProcPenaltyLevelGap);
+		if (
+			penalty_level_gap >= 0 &&
+			level_diff > penalty_level_gap
+		) {//10% penalty per level if > penalty_level_gap levels over target.
+			level_penalty = (level_diff - penalty_level_gap) * RuleR(Spells, DefensiveProcPenaltyLevelGapModifier);
 		}
 
-		ProcChance -= ProcChance*level_penalty / 100;
+		proc_chance -= proc_chance * level_penalty / 100;
 
-		if (ProcChance < 0) {
+		if (proc_chance < 0) {
 			return;
 		}
 
 		//Spell Procs and Quest added procs
-		for (int i = 0; i < MAX_PROCS; i++) {
+		for (int i = 0; i < m_max_procs; i++) {
 			if (IsValidSpell(DefensiveProcs[i].spellID)) {
 				if (!IsProcLimitTimerActive(DefensiveProcs[i].base_spellID, DefensiveProcs[i].proc_reuse_time, ProcType::DEFENSIVE_PROC)) {
-					float chance = ProcChance * (static_cast<float>(DefensiveProcs[i].chance) / 100.0f);
+					float chance = proc_chance * (static_cast<float>(DefensiveProcs[i].chance) / 100.0f);
 					if (zone->random.Roll(chance)) {
 						ExecWeaponProc(nullptr, DefensiveProcs[i].spellID, on);
 						CheckNumHitsRemaining(NumHit::DefensiveSpellProcs, 0, DefensiveProcs[i].base_spellID);
@@ -4576,14 +4596,14 @@ void Mob::TryDefensiveProc(Mob *on, uint16 hand) {
 		//AA Procs
 		if (IsOfClientBot()) {
 			for (int i = 0; i < MAX_AA_PROCS; i += 4) {
-				int32 aa_rank_id = aabonuses.DefensiveProc[i + +SBIndex::COMBAT_PROC_ORIGIN_ID];
-				int32 aa_spell_id = aabonuses.DefensiveProc[i + SBIndex::COMBAT_PROC_SPELL_ID];
-				int32 aa_proc_chance = 100 + aabonuses.DefensiveProc[i + SBIndex::COMBAT_PROC_RATE_MOD];
+				int32  aa_rank_id          = aabonuses.DefensiveProc[i + +SBIndex::COMBAT_PROC_ORIGIN_ID];
+				int32  aa_spell_id         = aabonuses.DefensiveProc[i + SBIndex::COMBAT_PROC_SPELL_ID];
+				int32  aa_proc_chance      = 100 + aabonuses.DefensiveProc[i + SBIndex::COMBAT_PROC_RATE_MOD];
 				uint32 aa_proc_reuse_timer = aabonuses.DefensiveProc[i + SBIndex::COMBAT_PROC_REUSE_TIMER];
 
 				if (aa_rank_id) {
 					if (!IsProcLimitTimerActive(-aa_rank_id, aa_proc_reuse_timer, ProcType::DEFENSIVE_PROC)) {
-						float chance = ProcChance * (static_cast<float>(aa_proc_chance) / 100.0f);
+						float chance = proc_chance * (static_cast<float>(aa_proc_chance) / 100.0f);
 						if (zone->random.Roll(chance) && IsValidSpell(aa_spell_id)) {
 							ExecWeaponProc(nullptr, aa_spell_id, on);
 							SetProcLimitTimer(-aa_rank_id, aa_proc_reuse_timer, ProcType::DEFENSIVE_PROC);
@@ -4763,7 +4783,7 @@ void Mob::TrySpellProc(const EQ::ItemInstance *inst, const EQ::ItemData *weapon,
 
 	int16 poison_slot=-1;
 
-	for (uint32 i = 0; i < MAX_PROCS; i++) {
+	for (uint32 i = 0; i < m_max_procs; i++) {
 		if (IsPet() && hand != EQ::invslot::slotPrimary) //Pets can only proc spell procs from their primay hand (ie; beastlord pets)
 			continue; // If pets ever can proc from off hand, this will need to change
 
@@ -5192,8 +5212,11 @@ bool Mob::TryFinishingBlow(Mob *defender, int64 &damage)
 			FB_Level = itembonuses.FinishingBlowLvl[SBIndex::FINISHING_EFFECT_LEVEL_MAX];
 
 		// modern AA description says rank 1 (500) is 50% chance
-		int ProcChance =
-				aabonuses.FinishingBlow[SBIndex::FINISHING_EFFECT_PROC_CHANCE] + spellbonuses.FinishingBlow[SBIndex::FINISHING_EFFECT_PROC_CHANCE] + spellbonuses.FinishingBlow[SBIndex::FINISHING_EFFECT_PROC_CHANCE];
+		int ProcChance = (
+			aabonuses.FinishingBlow[SBIndex::FINISHING_EFFECT_PROC_CHANCE] +
+			itembonuses.FinishingBlow[SBIndex::FINISHING_EFFECT_PROC_CHANCE] +
+			spellbonuses.FinishingBlow[SBIndex::FINISHING_EFFECT_PROC_CHANCE]
+		);
 
 		if (FB_Level && FB_Dmg && (defender->GetLevel() <= FB_Level) &&
 			(ProcChance >= zone->random.Int(1, 1000))) {
@@ -6458,3 +6481,8 @@ int64 Mob::GetManaRegen() const
 {
 	return mana_regen;
 }
+
+int64 Mob::GetEnduranceRegen() const
+{
+	return 0; // not implemented
+}

zone/bonuses.cpp

@@ -6036,7 +6036,7 @@ float Mob::CheckHeroicBonusesDataBuckets(std::string bucket_name)
 		DataBucketKey k = GetScopedBucketKeys();
 		k.key = bucket_name;
 		if (IsOfClientBot()) {
-			bucket_value = DataBucket::CheckBucketKey(this, k);
+			bucket_value = DataBucket::GetData(k).value;
 		}
 
 		if (bucket_value.empty() || !Strings::IsNumber(bucket_value)) {

zone/bot.cpp

@@ -22,8 +22,11 @@
 #include "doors.h"
 #include "quest_parser_collection.h"
 #include "lua_parser.h"
+#include "../common/repositories/bot_inventories_repository.h"
 #include "../common/repositories/bot_spell_settings_repository.h"
+#include "../common/repositories/bot_starting_items_repository.h"
 #include "../common/data_verification.h"
+#include "../common/repositories/criteria/content_filter_criteria.h"
 
 // This constructor is used during the bot create command
 Bot::Bot(NPCType *npcTypeData, Client* botOwner) : NPC(npcTypeData, nullptr, glm::vec4(), Ground, false), rest_timer(1), ping_timer(1) {
@@ -119,7 +122,15 @@ Bot::Bot(NPCType *npcTypeData, Client* botOwner) : NPC(npcTypeData, nullptr, glm
 }
 
 // This constructor is used when the bot is loaded out of the database
-Bot::Bot(uint32 botID, uint32 botOwnerCharacterID, uint32 botSpellsID, double totalPlayTime, uint32 lastZoneId, NPCType *npcTypeData)
+Bot::Bot(
+	uint32 botID,
+	uint32 botOwnerCharacterID,
+	uint32 botSpellsID,
+	double totalPlayTime,
+	uint32 lastZoneId,
+	NPCType *npcTypeData,
+	int32 expansion_bitmask
+)
 	: NPC(npcTypeData, nullptr, glm::vec4(), Ground, false), rest_timer(1), ping_timer(1)
 {
 	GiveNPCTypeData(npcTypeData);
@@ -161,6 +172,7 @@ Bot::Bot(uint32 botID, uint32 botOwnerCharacterID, uint32 botSpellsID, double to
 	RestRegenHP = 0;
 	RestRegenMana = 0;
 	RestRegenEndurance = 0;
+	m_expansion_bitmask = expansion_bitmask;
 	SetBotID(botID);
 	SetBotSpellID(botSpellsID);
 	SetSpawnStatus(false);
@@ -268,26 +280,48 @@ Bot::Bot(uint32 botID, uint32 botOwnerCharacterID, uint32 botSpellsID, double to
 				case SE_IllusionCopy:
 				case SE_Illusion: {
 					if (spell.base_value[x1] == -1) {
-						if (gender == 1)
-							gender = 0;
-						else if (gender == 0)
-							gender = 1;
-						SendIllusionPacket(GetRace(), gender, 0xFF, 0xFF);
-					}
-					else if (spell.base_value[x1] == -2) // WTF IS THIS
+						if (gender == FEMALE) {
+							gender = MALE;
+						} else if (gender == MALE) {
+							gender = FEMALE;
+						}
+
+						SendIllusionPacket(
+							AppearanceStruct{
+								.gender_id = gender,
+								.race_id = GetRace(),
+							}
+						);
+					} else if (spell.base_value[x1] == -2) // WTF IS THIS
 					{
 						if (GetRace() == IKSAR || GetRace() == VAHSHIR || GetRace() <= GNOME) {
-							SendIllusionPacket(GetRace(), GetGender(), spell.limit_value[x1], spell.max_value[x1]);
+							SendIllusionPacket(
+								AppearanceStruct{
+									.gender_id = GetGender(),
+									.helmet_texture = static_cast<uint8>(spell.max_value[x1]),
+									.race_id = GetRace(),
+									.texture = static_cast<uint8>(spell.limit_value[x1]),
+								}
+							);
 						}
+					} else if (spell.max_value[x1] > 0) {
+						SendIllusionPacket(
+							AppearanceStruct{
+								.helmet_texture = static_cast<uint8>(spell.max_value[x1]),
+								.race_id = static_cast<uint16>(spell.base_value[x1]),
+								.texture = static_cast<uint8>(spell.limit_value[x1]),
+							}
+						);
+					} else {
+						SendIllusionPacket(
+							AppearanceStruct{
+								.helmet_texture = static_cast<uint8>(spell.max_value[x1]),
+								.race_id = static_cast<uint16>(spell.base_value[x1]),
+								.texture = static_cast<uint8>(spell.limit_value[x1]),
+							}
+						);
 					}
-					else if (spell.max_value[x1] > 0)
-					{
-						SendIllusionPacket(spell.base_value[x1], 0xFF, spell.limit_value[x1], spell.max_value[x1]);
-					}
-					else
-					{
-						SendIllusionPacket(spell.base_value[x1], 0xFF, 0xFF, 0xFF);
-					}
+
 					switch (spell.base_value[x1]) {
 					case OGRE:
 						SendAppearancePacket(AT_Size, 9);
@@ -427,12 +461,21 @@ Bot::Bot(uint32 botID, uint32 botOwnerCharacterID, uint32 botSpellsID, double to
 Bot::~Bot() {
 	AI_Stop();
 	LeaveHealRotationMemberPool();
+	DataBucket::DeleteCachedBuckets(DataBucketLoadType::Bot, GetBotID());
 
 	if (HasPet()) {
 		GetPet()->Depop();
 	}
 
 	entity_list.RemoveBot(GetID());
+
+	if (GetGroup()) {
+		GetGroup()->MemberZoned(this);
+	}
+
+	if (GetRaid()) {
+		GetRaid()->MemberZoned(CastToClient());
+	}
 }
 
 void Bot::SetBotID(uint32 botID) {
@@ -611,7 +654,7 @@ NPCType *Bot::FillNPCTypeStruct(
 	n->current_hp = hp;
 	n->max_hp = hp;
 	n->size = size;
-	n->runspeed = 0.7f;
+	n->runspeed = 1.25f;
 	n->gender = gender;
 	n->race = botRace;
 	n->class_ = botClass;
@@ -2187,10 +2230,10 @@ void Bot::AI_Process()
 // OK TO IDLE
 
 		// Ok to idle
-		if (TryIdleChecks(fm_distance)) {
+		if (TryNonCombatMovementChecks(bot_owner, follow_mob, Goal)) {
 			return;
 		}
-		if (TryNonCombatMovementChecks(bot_owner, follow_mob, Goal)) {
+		if (TryIdleChecks(fm_distance)) {
 			return;
 		}
 		if (TryBardMovementCasts()) {
@@ -2223,23 +2266,11 @@ bool Bot::TryNonCombatMovementChecks(Client* bot_owner, const Mob* follow_mob, g
 		if ((!bot_owner->GetBotPulling() || PULLING_BOT) && (destination_distance > GetFollowDistance())) {
 
 			if (!IsRooted()) {
-
 				if (rest_timer.Enabled()) {
 					rest_timer.Disable();
 				}
 
-				bool running = true;
-
-				if (destination_distance < GetFollowDistance() + BOT_FOLLOW_DISTANCE_WALK) {
-					running = false;
-				}
-
-				if (running) {
-					RunTo(Goal.x, Goal.y, Goal.z);
-				}
-				else {
-					WalkTo(Goal.x, Goal.y, Goal.z);
-				}
+				RunTo(Goal.x, Goal.y, Goal.z);
 
 				return true;
 			}
@@ -3305,16 +3336,25 @@ bool Bot::Spawn(Client* botCharacterOwner) {
 
 		if (auto raid = entity_list.GetRaidByBotName(GetName())) {
 			// Safety Check to confirm we have a valid raid
-			if (raid->IsRaidMember(GetBotOwner()->CastToClient())) {
+			auto owner = GetBotOwner();
+			if (owner && !raid->IsRaidMember(owner->GetCleanName())) {
 				Bot::RemoveBotFromRaid(this);
 			} else {
-				raid->LearnMembers();
 				SetRaidGrouped(true);
+				raid->LearnMembers();
+				raid->VerifyRaid();
 			}
 		}
 		else if (auto group = entity_list.GetGroupByMobName(GetName())) {
-			group->LearnMembers();
-			SetGrouped(true);
+			// Safety Check to confirm we have a valid group
+			auto owner = GetBotOwner();
+			if (owner && !group->IsGroupMember(owner->GetCleanName())) {
+				Bot::RemoveBotFromGroup(this, group);
+			} else {
+				SetGrouped(true);
+				group->LearnMembers();
+				group->VerifyGroup();
+			}
 		}
 
 		return true;
@@ -4511,10 +4551,9 @@ void Bot::PerformTradeWithClient(int16 begin_slot_id, int16 end_slot_id, Client*
 }
 
 bool Bot::Death(Mob *killerMob, int64 damage, uint16 spell_id, EQ::skills::SkillType attack_skill) {
-	if (!NPC::Death(killerMob, damage, spell_id, attack_skill))
+	if (!NPC::Death(killerMob, damage, spell_id, attack_skill)) {
 		return false;
-
-	Save();
+	}
 
 	Mob *my_owner = GetBotOwner();
 	if (my_owner && my_owner->IsClient() && my_owner->CastToClient()->GetBotOption(Client::booDeathMarquee)) {
@@ -4524,81 +4563,11 @@ bool Bot::Death(Mob *killerMob, int64 damage, uint16 spell_id, EQ::skills::Skill
 			my_owner->CastToClient()->SendMarqueeMessage(Chat::White, 510, 0, 1000, 3000, StringFormat("%s has been slain", GetCleanName()));
 	}
 
-	Mob *give_exp = hate_list.GetDamageTopOnHateList(this);
-	Client *give_exp_client = nullptr;
-	if (give_exp && give_exp->IsClient())
-		give_exp_client = give_exp->CastToClient();
-
-	bool IsLdonTreasure = (GetClass() == LDON_TREASURE);
 	const auto c = entity_list.GetCorpseByID(GetID());
 	if (c) {
 		c->Depop();
 	}
 
-	if (HasRaid()) {
-		if (auto raid = entity_list.GetRaidByBotName(GetName()); raid) {
-			for (auto& m: raid->members) {
-				if (strcmp(m.member_name, GetName()) == 0) {
-					m.member = nullptr;
-				}
-			}
-		}
-	}
-	else if (HasGroup()) {
-		if (auto g = GetGroup()) {
-			for (int i = 0; i < MAX_GROUP_MEMBERS; i++) {
-				if (g->members[i]) {
-					if (g->members[i] == this) {
-						// If the leader dies, make the next bot the leader
-						// and reset all bots followid
-						if (g->IsLeader(g->members[i])) {
-							if (g->members[i + 1]) {
-								g->SetLeader(g->members[i + 1]);
-								g->members[i + 1]->SetFollowID(g->members[i]->GetFollowID());
-								for (int j = 0; j < MAX_GROUP_MEMBERS; j++) {
-									if (g->members[j] && (g->members[j] != g->members[i + 1]))
-										g->members[j]->SetFollowID(g->members[i + 1]->GetID());
-								}
-							}
-						}
-
-						// delete from group data
-						RemoveBotFromGroup(this, g);
-						//Make sure group still exists if it doesnt they were already updated in RemoveBotFromGroup
-						g = GetGroup();
-						if (!g)
-							break;
-
-						// if group members exist below this one, move
-						// them all up one slot in the group list
-						int j = (i + 1);
-						for (; j < MAX_GROUP_MEMBERS; j++) {
-							if (g->members[j]) {
-								g->members[j - 1] = g->members[j];
-								strcpy(g->membername[j - 1], g->members[j]->GetCleanName());
-								g->membername[j][0] = '\0';
-								memset(g->membername[j], 0, 64);
-								g->members[j] = nullptr;
-							}
-						}
-
-						// update the client group
-						EQApplicationPacket* outapp = new EQApplicationPacket(OP_GroupUpdate, sizeof(GroupJoin_Struct));
-						GroupJoin_Struct* gu = (GroupJoin_Struct*) outapp->pBuffer;
-						gu->action = groupActLeave;
-						strcpy(gu->membername, GetCleanName());
-						for (int k = 0; k < MAX_GROUP_MEMBERS; k++) {
-							if (g->members[k] && g->members[k]->IsClient()) {
-								g->members[k]->CastToClient()->QueuePacket(outapp);
-							}
-						}
-						safe_delete(outapp);
-					}
-				}
-			}
-		}
-	}
-
 	LeaveHealRotationMemberPool();
 
 	if ((GetPullingFlag() || GetReturningFlag()) && my_owner && my_owner->IsClient()) {
@@ -4617,6 +4586,7 @@ bool Bot::Death(Mob *killerMob, int64 damage, uint16 spell_id, EQ::skills::Skill
 		parse->EventBot(EVENT_DEATH_COMPLETE, this, killerMob, export_string, 0);
 	}
 
+	Zone();
 	entity_list.RemoveBot(GetID());
 
 return true;
@@ -5431,10 +5401,10 @@ void Bot::ProcessBotOwnerRefDelete(Mob* botOwner) {
 int64 Bot::CalcMaxMana() {
 	switch(GetCasterClass()) {
 		case 'I':
-			max_mana = (GenerateBaseManaPoints() + itembonuses.Mana + spellbonuses.Mana + GroupLeadershipAAManaEnhancement());
+			max_mana = (GenerateBaseManaPoints() + itembonuses.Mana + spellbonuses.Mana + aabonuses.Mana + GroupLeadershipAAManaEnhancement());
 			max_mana += itembonuses.heroic_max_mana;
 		case 'W': {
-			max_mana = (GenerateBaseManaPoints() + itembonuses.Mana + spellbonuses.Mana + GroupLeadershipAAManaEnhancement());
+			max_mana = (GenerateBaseManaPoints() + itembonuses.Mana + spellbonuses.Mana + aabonuses.Mana + GroupLeadershipAAManaEnhancement());
 			max_mana += itembonuses.heroic_max_mana;
 			break;
 		}
@@ -8197,18 +8167,19 @@ bool Bot::CheckDataBucket(std::string bucket_name, const std::string& bucket_val
 		DataBucketKey k = GetScopedBucketKeys();
 		k.key = bucket_name;
 
-		auto player_value = DataBucket::CheckBucketKey(this, k);
-		if (player_value.empty() && GetBotOwner()) {
+		auto b = DataBucket::GetData(k);
+		if (b.value.empty() && GetBotOwner()) {
 			// fetch from owner
 			k = GetBotOwner()->GetScopedBucketKeys();
+			k.key = bucket_name;
 
-			player_value = DataBucket::CheckBucketKey(GetBotOwner(), k);
-			if (player_value.empty()) {
+			b = DataBucket::GetData(k);
+			if (b.value.empty()) {
 				return false;
 			}
 		}
 
-		if (zone->CompareDataBucket(bucket_comparison, bucket_value, player_value)) {
+		if (zone->CompareDataBucket(bucket_comparison, bucket_value, b.value)) {
 			return true;
 		}
 	}
@@ -8598,6 +8569,7 @@ std::vector<Mob*> Bot::GetApplySpellList(
 void Bot::ApplySpell(
 	int spell_id,
 	int duration,
+	int level,
 	ApplySpellType apply_type,
 	bool allow_pets,
 	bool is_raid_group_only
@@ -8605,13 +8577,14 @@ void Bot::ApplySpell(
 	const auto& l = GetApplySpellList(apply_type, allow_pets, is_raid_group_only);
 
 	for (const auto& m : l) {
-		m->ApplySpellBuff(spell_id, duration);
+		m->ApplySpellBuff(spell_id, duration, level);
 	}
 }
 
 void Bot::SetSpellDuration(
 	int spell_id,
 	int duration,
+	int level,
 	ApplySpellType apply_type,
 	bool allow_pets,
 	bool is_raid_group_only
@@ -8619,7 +8592,7 @@ void Bot::SetSpellDuration(
 	const auto& l = GetApplySpellList(apply_type, allow_pets, is_raid_group_only);
 
 	for (const auto& m : l) {
-		m->SetBuffDuration(spell_id, duration);
+		m->SetBuffDuration(spell_id, duration, level);
 	}
 }
 
@@ -8759,4 +8732,50 @@ bool Bot::CheckSpawnConditions(Client* c) {
 	return true;
 }
 
+void Bot::AddBotStartingItems(uint16 race_id, uint8 class_id)
+{
+	if (!IsPlayerRace(race_id) || !IsPlayerClass(class_id)) {
+		return;
+	}
+
+	const uint16 race_bitmask  = GetPlayerRaceBit(race_id);
+	const uint16 class_bitmask = GetPlayerClassBit(class_id);
+
+	const auto& l = BotStartingItemsRepository::GetWhere(
+		content_db,
+		fmt::format(
+			"(races & {} OR races = 0) AND "
+			"(classes & {} OR classes = 0) {}",
+			race_bitmask,
+			class_bitmask,
+			ContentFilterCriteria::apply()
+		)
+	);
+
+	if (l.empty()) {
+		return;
+	}
+
+	std::vector<BotInventoriesRepository::BotInventories> v;
+
+	for (const auto& e : l) {
+		if (
+			CanClassEquipItem(e.item_id) &&
+			(CanRaceEquipItem(e.item_id) || RuleB(Bots, AllowBotEquipAnyRaceGear))
+		) {
+			auto i = BotInventoriesRepository::NewEntity();
+			i.bot_id       = GetBotID();
+			i.slot_id      = e.slot_id;
+			i.item_id      = e.item_id;
+			i.inst_charges = e.item_charges;
+
+			v.emplace_back(i);
+		}
+	}
+
+	if (!v.empty()) {
+		BotInventoriesRepository::InsertMany(content_db, v);
+	}
+}
+
 uint8 Bot::spell_casting_chances[SPELL_TYPE_COUNT][PLAYER_CLASS_COUNT][EQ::constants::STANCE_TYPE_COUNT][cntHSND] = { 0 };

zone/bot.h

@@ -39,7 +39,6 @@
 
 constexpr uint32 BOT_FOLLOW_DISTANCE_DEFAULT = 184; // as DSq value (~13.565 units)
 constexpr uint32 BOT_FOLLOW_DISTANCE_DEFAULT_MAX = 2500; // as DSq value (50 units)
-constexpr uint32 BOT_FOLLOW_DISTANCE_WALK = 1000; // as DSq value (~31.623 units)
 
 constexpr uint32 BOT_KEEP_ALIVE_INTERVAL = 5000; // 5 seconds
 
@@ -130,7 +129,7 @@ public:
 
 	// Class Constructors
 	Bot(NPCType *npcTypeData, Client* botOwner);
-	Bot(uint32 botID, uint32 botOwnerCharacterID, uint32 botSpellsID, double totalPlayTime, uint32 lastZoneId, NPCType *npcTypeData);
+	Bot(uint32 botID, uint32 botOwnerCharacterID, uint32 botSpellsID, double totalPlayTime, uint32 lastZoneId, NPCType *npcTypeData, int32 expansion_bitmask);
 
 	//abstract virtual override function implementations requird by base abstract class
 	bool Death(Mob* killerMob, int64 damage, uint16 spell_id, EQ::skills::SkillType attack_skill) override;
@@ -461,6 +460,8 @@ public:
 		uint8 gender
 	);
 
+	void AddBotStartingItems(uint16 race_id, uint8 class_id);
+
 	// Static Bot Group Methods
 	static bool AddBotToGroup(Bot* bot, Group* group);
 	static bool RemoveBotFromGroup(Bot* bot, Group* group);
@@ -584,7 +585,7 @@ public:
 
 	// "Quest API" Methods
 	bool HasBotSpellEntry(uint16 spellid);
-	void ApplySpell(int spell_id, int duration = 0, ApplySpellType apply_type = ApplySpellType::Solo, bool allow_pets = false, bool is_raid_group_only = true);
+	void ApplySpell(int spell_id, int duration = 0, int level = -1, ApplySpellType apply_type = ApplySpellType::Solo, bool allow_pets = false, bool is_raid_group_only = true);
 	void BreakInvis();
 	void Escape();
 	void Fling(float value, float target_x, float target_y, float target_z, bool ignore_los = false, bool clip_through_walls = false, bool calculate_speed = false);
@@ -593,7 +594,7 @@ public:
 	int32 GetAugmentIDAt(int16 slot_id, uint8 augslot);
 	int32 GetRawItemAC();
 	void SendSpellAnim(uint16 targetid, uint16 spell_id);
-	void SetSpellDuration(int spell_id, int duration = 0, ApplySpellType apply_type = ApplySpellType::Solo, bool allow_pets = false, bool is_raid_group_only = true);
+	void SetSpellDuration(int spell_id, int duration = 0, int level = -1, ApplySpellType apply_type = ApplySpellType::Solo, bool allow_pets = false, bool is_raid_group_only = true);
 
 	// "SET" Class Methods
 	void SetBotSpellID(uint32 newSpellID);

zone/bot_command.cpp

@@ -63,6 +63,7 @@
 #include "water_map.h"
 #include "worldserver.h"
 #include "dialogue_window.h"
+#include "mob.h"
 
 #include <fmt/format.h>
 
@@ -2098,7 +2099,6 @@ namespace ActionableBots
 		ABT_Target,
 		ABT_ByName,
 		ABT_OwnerGroup,
-		ABT_BotGroup,
 		ABT_TargetGroup,
 		ABT_NamesGroup,
 		ABT_HealRotation,
@@ -2439,7 +2439,6 @@ void bot_command_actionable(Client *c, const Seperator *sep)
 	c->Message(Chat::White, "target - selects target as single bot .. use ^command [target] or imply by empty actionable argument");
 	c->Message(Chat::White, "byname [name] - selects single bot by name");
 	c->Message(Chat::White, "ownergroup - selects all bots in the owner's group");
-	c->Message(Chat::White, "botgroup [name] - selects members of a bot-group by its name");
 	c->Message(Chat::White, "targetgroup - selects all bots in target's group");
 	c->Message(Chat::White, "namesgroup [name] - selects all bots in name's group");
 	c->Message(Chat::White, "healrotation [name] - selects all member and target bots of a heal rotation where name is a member");
@@ -2457,7 +2456,7 @@ void bot_command_aggressive(Client *c, const Seperator *sep)
 	if (helper_spell_list_fail(c, local_list, BCEnum::SpT_Stance) || helper_command_alias_fail(c, "bot_command_aggressive", sep->arg[0], "aggressive"))
 		return;
 	if (helper_is_help_or_usage(sep->arg[1])) {
-		c->Message(Chat::White, "usage: %s ([actionable: target | byname | ownergroup | botgroup | targetgroup | namesgroup | healrotationtargets | spawned] ([actionable_name]))", sep->arg[0]);
+		c->Message(Chat::White, "usage: %s ([actionable: target | byname | ownergroup | targetgroup | namesgroup | healrotationtargets | spawned] ([actionable_name]))", sep->arg[0]);
 		helper_send_usage_required_bots(c, BCEnum::SpT_Stance);
 		return;
 	}
@@ -2687,7 +2686,7 @@ void bot_command_attack(Client *c, const Seperator *sep)
 	}
 	if (helper_is_help_or_usage(sep->arg[1])) {
 
-		c->Message(Chat::White, "usage: <enemy_target> %s [actionable: byname | ownergroup | botgroup | namesgroup | healrotation | default: spawned] ([actionable_name])", sep->arg[0]);
+		c->Message(Chat::White, "usage: <enemy_target> %s [actionable: byname | ownergroup | namesgroup | healrotation | default: spawned] ([actionable_name])", sep->arg[0]);
 		return;
 	}
 	const int ab_mask = ActionableBots::ABM_Type2;
@@ -2944,7 +2943,7 @@ void bot_command_defensive(Client *c, const Seperator *sep)
 	if (helper_spell_list_fail(c, local_list, BCEnum::SpT_Stance) || helper_command_alias_fail(c, "bot_command_defensive", sep->arg[0], "defensive"))
 		return;
 	if (helper_is_help_or_usage(sep->arg[1])) {
-		c->Message(Chat::White, "usage: %s ([actionable: target | byname | ownergroup | botgroup | targetgroup | namesgroup | healrotationtargets | spawned] ([actionable_name]))", sep->arg[0]);
+		c->Message(Chat::White, "usage: %s ([actionable: target | byname | ownergroup | targetgroup | namesgroup | healrotationtargets | spawned] ([actionable_name]))", sep->arg[0]);
 		helper_send_usage_required_bots(c, BCEnum::SpT_Stance);
 		return;
 	}
@@ -3146,7 +3145,7 @@ void bot_command_follow(Client *c, const Seperator *sep)
 	if (helper_command_alias_fail(c, "bot_command_follow", sep->arg[0], "follow"))
 		return;
 	if (helper_is_help_or_usage(sep->arg[1])) {
-		c->Message(Chat::White, "usage: (<friendly_target>) %s ([option: reset]) [actionable: byname | ownergroup | botgroup | namesgroup | healrotation | spawned] ([actionable_name])", sep->arg[0]);
+		c->Message(Chat::White, "usage: (<friendly_target>) %s ([option: reset]) [actionable: byname | ownergroup | namesgroup | healrotation | spawned] ([actionable_name])", sep->arg[0]);
 		c->Message(Chat::White, "usage: %s chain", sep->arg[0]);
 		return;
 	}
@@ -3253,7 +3252,7 @@ void bot_command_guard(Client *c, const Seperator *sep)
 	}
 	if (helper_is_help_or_usage(sep->arg[1])) {
 
-		c->Message(Chat::White, "usage: %s ([option: clear]) [actionable: target | byname | ownergroup | botgroup | namesgroup | healrotation | spawned] ([actionable_name])", sep->arg[0]);
+		c->Message(Chat::White, "usage: %s ([option: clear]) [actionable: target | byname | ownergroup | namesgroup | healrotation | spawned] ([actionable_name])", sep->arg[0]);
 		return;
 	}
 	const int ab_mask = (ActionableBots::ABM_Target | ActionableBots::ABM_Type2);
@@ -3387,7 +3386,7 @@ void bot_command_hold(Client *c, const Seperator *sep)
 	}
 	if (helper_is_help_or_usage(sep->arg[1])) {
 
-		c->Message(Chat::White, "usage: %s ([option: clear]) [actionable: target | byname | ownergroup | botgroup | namesgroup | healrotation | spawned] ([actionable_name])", sep->arg[0]);
+		c->Message(Chat::White, "usage: %s ([option: clear]) [actionable: target | byname | ownergroup | namesgroup | healrotation | spawned] ([actionable_name])", sep->arg[0]);
 		return;
 	}
 	const int ab_mask = (ActionableBots::ABM_Target | ActionableBots::ABM_Type2);
@@ -4727,7 +4726,7 @@ void bot_command_taunt(Client *c, const Seperator *sep)
 	if (helper_command_alias_fail(c, "bot_command_taunt", sep->arg[0], "taunt"))
 		return;
 	if (helper_is_help_or_usage(sep->arg[1])) {
-		c->Message(Chat::White, "usage: %s ([option: on | off]) ([actionable: target | byname | ownergroup | botgroup | targetgroup | namesgroup | healrotationtargets | spawned] ([actionable_name]))", sep->arg[0]);
+		c->Message(Chat::White, "usage: %s ([option: on | off]) ([actionable: target | byname | ownergroup | targetgroup | namesgroup | healrotationtargets | spawned] ([actionable_name]))", sep->arg[0]);
 		return;
 	}
 	const int ab_mask = ActionableBots::ABM_Type1;
@@ -5044,7 +5043,7 @@ void bot_subcommand_bot_camp(Client *c, const Seperator *sep)
 	if (helper_command_alias_fail(c, "bot_subcommand_bot_camp", sep->arg[0], "botcamp"))
 		return;
 	if (helper_is_help_or_usage(sep->arg[1])) {
-		c->Message(Chat::White, "usage: %s ([actionable: target | byname | ownergroup | botgroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))", sep->arg[0]);
+		c->Message(Chat::White, "usage: %s ([actionable: target | byname | ownergroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))", sep->arg[0]);
 		return;
 	}
 	const int ab_mask = ActionableBots::ABM_NoFilter;
@@ -5628,7 +5627,7 @@ void bot_subcommand_bot_dye_armor(Client *c, const Seperator *sep)
 		c->Message(
 			Chat::White,
 			fmt::format(
-				"Usage: {} [Material Slot] [Red: 0-255] [Green: 0-255] [Blue: 0-255] ([actionable: target | byname | ownergroup | botgroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))",
+				"Usage: {} [Material Slot] [Red: 0-255] [Green: 0-255] [Blue: 0-255] ([actionable: target | byname | ownergroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))",
 				sep->arg[0]
 			).c_str()
 		);
@@ -5835,8 +5834,8 @@ void bot_subcommand_bot_follow_distance(Client *c, const Seperator *sep)
 	if (helper_command_alias_fail(c, "bot_subcommand_bot_follow_distance", sep->arg[0], "botfollowdistance"))
 		return;
 	if (helper_is_help_or_usage(sep->arg[1])) {
-		c->Message(Chat::White, "usage: %s [set] [distance] ([actionable: target | byname | ownergroup | botgroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))", sep->arg[0]);
-		c->Message(Chat::White, "usage: %s [clear] ([actionable: target | byname | ownergroup | botgroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))", sep->arg[0]);
+		c->Message(Chat::White, "usage: %s [set] [distance] ([actionable: target | byname | ownergroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))", sep->arg[0]);
+		c->Message(Chat::White, "usage: %s [clear] ([actionable: target | byname | ownergroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))", sep->arg[0]);
 		return;
 	}
 	const int ab_mask = ActionableBots::ABM_NoFilter;
@@ -6010,7 +6009,7 @@ void bot_subcommand_bot_inspect_message(Client *c, const Seperator *sep)
 	if (helper_command_alias_fail(c, "bot_subcommand_bot_inspect_message", sep->arg[0], "botinspectmessage"))
 		return;
 	if (helper_is_help_or_usage(sep->arg[1])) {
-		c->Message(Chat::White, "usage: %s [set | clear] ([actionable: target | byname | ownergroup | botgroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))", sep->arg[0]);
+		c->Message(Chat::White, "usage: %s [set | clear] ([actionable: target | byname | ownergroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))", sep->arg[0]);
 		c->Message(Chat::White, "Notes:");
 		if (c->ClientVersion() >= EQ::versions::ClientVersion::SoF) {
 			c->Message(Chat::White, "- Self-inspect and type your bot's inspect message");
@@ -6392,7 +6391,7 @@ void bot_subcommand_bot_report(Client *c, const Seperator *sep)
 	if (helper_command_alias_fail(c, "bot_subcommand_bot_report", sep->arg[0], "botreport"))
 		return;
 	if (helper_is_help_or_usage(sep->arg[1])) {
-		c->Message(Chat::White, "usage: %s ([actionable: target | byname | ownergroup | botgroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))", sep->arg[0]);
+		c->Message(Chat::White, "usage: %s ([actionable: target | byname | ownergroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))", sep->arg[0]);
 		return;
 	}
 	const int ab_mask = ActionableBots::ABM_NoFilter;
@@ -6751,7 +6750,7 @@ void bot_subcommand_bot_summon(Client *c, const Seperator *sep)
 		c->Message(
 			Chat::White,
 			fmt::format(
-				"Usage: {} ([actionable: target | byname | ownergroup | botgroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))",
+				"Usage: {} ([actionable: target | byname | ownergroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))",
 				sep->arg[0]
 			).c_str()
 		);
@@ -6896,7 +6895,7 @@ void bot_subcommand_bot_toggle_helm(Client *c, const Seperator *sep)
 	if (helper_command_alias_fail(c, "bot_subcommand_bot_toggle_helm", sep->arg[0], "bottogglehelm"))
 		return;
 	if (helper_is_help_or_usage(sep->arg[1])) {
-		c->Message(Chat::White, "usage: %s ([option: on | off]) ([actionable: target | byname | ownergroup | botgroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))", sep->arg[0]);
+		c->Message(Chat::White, "usage: %s ([option: on | off]) ([actionable: target | byname | ownergroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))", sep->arg[0]);
 		return;
 	}
 	const int ab_mask = ActionableBots::ABM_NoFilter;
@@ -8576,7 +8575,7 @@ void bot_subcommand_pet_get_lost(Client *c, const Seperator *sep)
 	if (helper_command_alias_fail(c, "bot_subcommand_pet_get_lost", sep->arg[0], "petgetlost"))
 		return;
 	if (helper_is_help_or_usage(sep->arg[1])) {
-		c->Message(Chat::White, "usage: %s ([actionable: target | byname | ownergroup | botgroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))", sep->arg[0]);
+		c->Message(Chat::White, "usage: %s ([actionable: target | byname | ownergroup | targetgroup | namesgroup | healrotation | spawned] ([actionable_name]))", sep->arg[0]);
 		return;
 	}
 	int ab_mask = ActionableBots::ABM_NoFilter;
@@ -8819,26 +8818,28 @@ void helper_bot_appearance_form_final(Client *bot_owner, Bot *my_bot)
 
 void helper_bot_appearance_form_update(Bot *my_bot)
 {
-	if (!my_bot)
+	if (!my_bot) {
 		return;
+	}
 
 	my_bot->SendIllusionPacket(
-		my_bot->GetRace(),
-		my_bot->GetGender(),
-		0xFF,	//my_bot->GetTexture(),		// 0xFF - change back if issues arise
-		0xFF,	//my_bot->GetHelmTexture(),	// 0xFF - change back if issues arise
-		my_bot->GetHairColor(),
-		my_bot->GetBeardColor(),
-		my_bot->GetEyeColor1(),
-		my_bot->GetEyeColor2(),
-		my_bot->GetHairStyle(),
-		my_bot->GetLuclinFace(),
-		my_bot->GetBeard(),
-		0xFF,					// aa_title (0xFF)
-		my_bot->GetDrakkinHeritage(),
-		my_bot->GetDrakkinTattoo(),
-		my_bot->GetDrakkinDetails(),
-		my_bot->GetSize()
+		AppearanceStruct{
+			.beard = my_bot->GetBeard(),
+			.beard_color = my_bot->GetBeardColor(),
+			.drakkin_details = my_bot->GetDrakkinDetails(),
+			.drakkin_heritage = my_bot->GetDrakkinHeritage(),
+			.drakkin_tattoo = my_bot->GetDrakkinTattoo(),
+			.eye_color_one = my_bot->GetEyeColor1(),
+			.eye_color_two = my_bot->GetEyeColor2(),
+			.face = my_bot->GetLuclinFace(),
+			.gender_id = my_bot->GetGender(),
+			.hair = my_bot->GetHairStyle(),
+			.hair_color = my_bot->GetHairColor(),
+			.helmet_texture = my_bot->GetHelmTexture(),
+			.race_id = my_bot->GetRace(),
+			.size = my_bot->GetSize(),
+			.texture = my_bot->GetTexture(),
+		}
 	);
 }
 
@@ -9040,6 +9041,8 @@ uint32 helper_bot_create(Client *bot_owner, std::string bot_name, uint8 bot_clas
 		parse->EventPlayer(EVENT_BOT_CREATE, bot_owner, export_string, 0);
 	}
 
+	my_bot->AddBotStartingItems(bot_race, bot_class);
+
 	safe_delete(my_bot);
 
 	return bot_id;
@@ -10002,7 +10005,7 @@ void bot_command_pickpocket(Client *c, const Seperator *sep)
 	if (helper_command_disabled(c, RuleB(Bots, AllowPickpocketCommand), "pickpocket")) {
 		return;
 	}
-	
+
 	if (helper_command_alias_fail(c, "bot_command_pickpocket", sep->arg[0], "pickpocket")) {
 		return;
 	}
@@ -10037,7 +10040,7 @@ void bot_command_pickpocket(Client *c, const Seperator *sep)
 	float     mob_xy_distance = ((mob_distance.x * mob_distance.x) + (mob_distance.y * mob_distance.y));
 	float     mob_z_distance  = (mob_distance.z * mob_distance.z);
 	float     z_offset_diff   = target_mob->GetZOffset() - c->GetZOffset();
-	
+
 	if (mob_z_distance >= (35-z_offset_diff) || mob_xy_distance > 250) {
 		c->Message(Chat::White, "You must be closer to an enemy to use this command");
 		return;

zone/bot_database.cpp

@@ -464,7 +464,8 @@ bool BotDatabase::LoadBot(const uint32 bot_id, Bot*& loaded_bot)
 		l.spells_id,
 		l.time_spawned,
 		l.zone_id,
-		t
+		t,
+		l.expansion_bitmask
 	);
 
 	if (loaded_bot) {
@@ -479,8 +480,6 @@ bool BotDatabase::LoadBot(const uint32 bot_id, Bot*& loaded_bot)
 
 		loaded_bot->SetStopMeleeLevel(l.stop_melee_level);
 
-		loaded_bot->SetExpansionBitmask(l.expansion_bitmask, false);
-
 		loaded_bot->SetBotEnforceSpellSetting((l.enforce_spell_settings ? true : false));
 
 		loaded_bot->SetBotArcherySetting((l.archery_setting ? true : false));

zone/cli/sidecar_serve_http.cpp

@@ -0,0 +1,24 @@
+#include "../../common/http/httplib.h"
+#include "../../common/eqemu_logsys.h"
+#include "../sidecar_api/sidecar_api.h"
+#include "../../common/platform.h"
+
+void ZoneCLI::SidecarServeHttp(int argc, char **argv, argh::parser &cmd, std::string &description)
+{
+	if (cmd[{"-h", "--help"}]) {
+		return;
+	}
+
+	RegisterExecutablePlatform(EQEmuExePlatform::ExePlatformZoneSidecar);
+
+	int port = 0;
+	std::string key;
+	if (!cmd("--port").str().empty()) {
+		port = strtoll(cmd("--port").str().c_str(), nullptr, 10);
+	}
+	if (!cmd("--key").str().empty()) {
+		key = cmd("--key").str();
+	}
+
+	SidecarApi::BootWebserver(port, key);
+}

zone/client.cpp

@@ -381,6 +381,8 @@ Client::Client(EQStreamInterface *ieqs) : Mob(
 Client::~Client() {
 	mMovementManager->RemoveClient(this);
 
+	DataBucket::DeleteCachedBuckets(DataBucketLoadType::Client, CharacterID());
+
 	if (RuleB(Bots, Enabled)) {
 		Bot::ProcessBotOwnerRefDelete(this);
 	}
@@ -2209,25 +2211,13 @@ void Client::ReadBook(BookRequest_Struct *book) {
 
 
 		if (ClientVersion() >= EQ::versions::ClientVersion::SoF) {
-			// Find out what slot the book was read from.
 			// SoF+ need to look up book type for the output message.
-			int16	read_from_slot;
-
-			if (book->subslot >= 0) {
-				uint16 offset;
-				offset = (book->invslot-23) * 10;	// How many packs to skip.
-				read_from_slot = 251 + offset + book->subslot;
-			}
-			else {
-				read_from_slot = book->invslot -1;
-			}
-
 			const EQ::ItemInstance *inst = nullptr;
 
-			if (read_from_slot <= EQ::invbag::GENERAL_BAGS_END)
-				{
-				inst = m_inv[read_from_slot];
-				}
+			if (book->invslot <= EQ::invbag::GENERAL_BAGS_END)
+			{
+				inst = m_inv[book->invslot];
+			}
 
 			if(inst)
 				out->type = inst->GetItem()->Book;
@@ -2238,6 +2228,9 @@ void Client::ReadBook(BookRequest_Struct *book) {
 			out->type = book->type;
 		}
 		out->invslot = book->invslot;
+		out->target_id = book->target_id;
+		out->can_cast = 0; // todo: implement
+		out->can_scribe = 0; // todo: implement
 
 		memcpy(out->booktext, booktxt2.c_str(), length);
 
@@ -8881,7 +8874,8 @@ void Client::ShowDevToolsMenu()
 	menu_reload_two += Saylink::Silent("#reload commands", "Commands");
 	menu_reload_two += " | " + Saylink::Silent("#reload content_flags", "Content Flags");
 
-	menu_reload_three += Saylink::Silent("#reload doors", "Doors");
+	menu_reload_three += Saylink::Silent("#reload data_buckets_cache", "Databuckets");
+	menu_reload_three += " | " + Saylink::Silent("#reload doors", "Doors");
 	menu_reload_three += " | " + Saylink::Silent("#reload ground_spawns", "Ground Spawns");
 
 	menu_reload_four += Saylink::Silent("#reload logs", "Level Based Experience Modifiers");
@@ -10835,6 +10829,16 @@ void Client::SendReloadCommandMessages() {
 		).c_str()
 	);
 
+	auto data_buckets_link = Saylink::Silent("#reload data_buckets_cache");
+
+	Message(
+		Chat::White,
+		fmt::format(
+			"Usage: {} - Reloads data buckets cache globally",
+			data_buckets_link
+		).c_str()
+	);
+
 	auto dztemplates_link = Saylink::Silent("#reload dztemplates");
 	Message(Chat::White, fmt::format("Usage: {} - Reloads Dynamic Zone Templates globally", dztemplates_link).c_str());
 
@@ -11077,8 +11081,6 @@ void Client::SetTrackingID(uint32 entity_id)
 	}
 
 	TrackingID = entity_id;
-
-	MessageString(Chat::Skills, TRACKING_BEGIN, m->GetCleanName());
 }
 
 int Client::GetRecipeMadeCount(uint32 recipe_id)
@@ -11277,6 +11279,7 @@ std::vector<Mob*> Client::GetApplySpellList(
 void Client::ApplySpell(
 	int spell_id,
 	int duration,
+	int level,
 	ApplySpellType apply_type,
 	bool allow_pets,
 	bool is_raid_group_only,
@@ -11285,13 +11288,14 @@ void Client::ApplySpell(
 	const auto& l = GetApplySpellList(apply_type, allow_pets, is_raid_group_only, allow_bots);
 
 	for (const auto& m : l) {
-		m->ApplySpellBuff(spell_id, duration);
+		m->ApplySpellBuff(spell_id, duration, level);
 	}
 }
 
 void Client::SetSpellDuration(
 	int spell_id,
 	int duration,
+	int level,
 	ApplySpellType apply_type,
 	bool allow_pets,
 	bool is_raid_group_only,
@@ -11300,7 +11304,7 @@ void Client::SetSpellDuration(
 	const auto& l = GetApplySpellList(apply_type, allow_pets, is_raid_group_only, allow_bots);
 
 	for (const auto& m : l) {
-		m->SetBuffDuration(spell_id, duration);
+		m->SetBuffDuration(spell_id, duration, level);
 	}
 }
 
@@ -11737,7 +11741,7 @@ void Client::ShowSpells(Client* c, ShowSpellType show_spell_type)
 					"{}. {} ({})",
 					index,
 					GetSpellName(spell_id),
-					Strings::Commify(spell_id)
+					spell_id
 				).c_str()
 			);
 		}
@@ -11755,3 +11759,29 @@ void Client::ShowSpells(Client* c, ShowSpellType show_spell_type)
 		spell_table.c_str()
 	);
 }
+
+std::string GetZoneModeString(ZoneMode mode)
+{
+	switch (mode) {
+		case ZoneToSafeCoords:
+			return "ZoneToSafeCoords";
+		case GMSummon:
+			return "GMSummon";
+		case ZoneToBindPoint:
+			return "ZoneToBindPoint";
+		case ZoneSolicited:
+			return "ZoneSolicited";
+		case ZoneUnsolicited:
+			return "ZoneUnsolicited";
+		case GateToBindPoint:
+			return "GateToBindPoint";
+		case SummonPC:
+			return "SummonPC";
+		case Rewind:
+			return "Rewind";
+		case EvacToSafeCoords:
+			return "EvacToSafeCoords";
+		default:
+			return "Unknown";
+	}
+}

zone/client.h

@@ -123,6 +123,9 @@ typedef enum {
 	EvacToSafeCoords
 } ZoneMode;
 
+// translate above enum to a string
+std::string GetZoneModeString(ZoneMode mode);
+
 enum {
 	HideCorpseNone = 0,
 	HideCorpseAll = 1,
@@ -233,6 +236,7 @@ public:
 	~Client();
 
 	void ReconnectUCS();
+	void RecordStats();
 
 	void SetDisplayMobInfoWindow(bool display_mob_info_window);
 	bool GetDisplayMobInfoWindow() const;
@@ -381,6 +385,7 @@ public:
 	inline PetInfo* GetPetInfo(uint16 pet) { return (pet==1)?&m_suspendedminion:&m_petinfo; }
 	inline InspectMessage_Struct& GetInspectMessage() { return m_inspect_message; }
 	inline const InspectMessage_Struct& GetInspectMessage() const { return m_inspect_message; }
+	void ReloadExpansionProfileSetting();
 
 	void SetPetCommandState(int button, int state);
 
@@ -900,6 +905,9 @@ public:
 	int GetAAPoints() { return m_pp.aapoints; }
 	int GetSpentAA() { return m_pp.aapoints_spent; }
 	uint32 GetRequiredAAExperience();
+	void AutoGrantAAPoints();
+	void GrantAllAAPoints(uint8 unlock_level = 0);
+	bool HasAlreadyPurchasedRank(AA::Rank* rank);
 
 	bool SendGMCommand(std::string message, bool ignore_status = false);
 
@@ -915,6 +923,7 @@ public:
 	void ApplySpell(
 		int spell_id,
 		int duration = 0,
+		int level = -1,
 		ApplySpellType apply_type = ApplySpellType::Solo,
 		bool allow_pets = false,
 		bool is_raid_group_only = true,
@@ -924,6 +933,7 @@ public:
 	void SetSpellDuration(
 		int spell_id,
 		int duration = 0,
+		int level = -1,
 		ApplySpellType apply_type = ApplySpellType::Solo,
 		bool allow_pets = false,
 		bool is_raid_group_only = true,
@@ -1657,7 +1667,7 @@ protected:
 	bool client_data_loaded;
 
 
-	void FinishAlternateAdvancementPurchase(AA::Rank *rank, bool ignore_cost);
+	void FinishAlternateAdvancementPurchase(AA::Rank *rank, bool ignore_cost, bool send_message_and_save);
 
 	Mob* bind_sight_target;
 

zone/client_packet.cpp

@@ -18,6 +18,8 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 #include "../common/global_define.h"
 #include "../common/eqemu_logsys.h"
 #include "../common/opcodemgr.h"
+#include "../common/raid.h"
+
 #include <iomanip>
 #include <iostream>
 #include <math.h>
@@ -62,6 +64,7 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 #include "../common/repositories/account_repository.h"
 
 #include "../common/events/player_event_logs.h"
+#include "../common/repositories/character_stats_record_repository.h"
 
 extern QueryServ* QServ;
 extern Zone* zone;
@@ -144,6 +147,7 @@ void MapOpcodes()
 	ConnectedOpcodes[OP_Bind_Wound] = &Client::Handle_OP_Bind_Wound;
 	ConnectedOpcodes[OP_BlockedBuffs] = &Client::Handle_OP_BlockedBuffs;
 	ConnectedOpcodes[OP_BoardBoat] = &Client::Handle_OP_BoardBoat;
+	ConnectedOpcodes[OP_BookButton] = &Client::Handle_OP_BookButton;
 	ConnectedOpcodes[OP_Buff] = &Client::Handle_OP_Buff;
 	ConnectedOpcodes[OP_BuffRemoveRequest] = &Client::Handle_OP_BuffRemoveRequest;
 	ConnectedOpcodes[OP_Bug] = &Client::Handle_OP_Bug;
@@ -915,6 +919,10 @@ void Client::CompleteConnect()
 
 	heroforge_wearchange_timer.Start(250);
 
+	RecordStats();
+
+	AutoGrantAAPoints();
+
 	// enforce some rules..
 	if (!CanEnterZone()) {
 		LogInfo("Kicking character [{}] from zone, not allowed here (missing requirements)", GetCleanName());
@@ -1442,12 +1450,7 @@ void Client::Handle_Connect_OP_ZoneEntry(const EQApplicationPacket *app)
 	if (m_pp.ldon_points_tak < 0 || m_pp.ldon_points_tak > 2000000000) { m_pp.ldon_points_tak = 0; }
 	if (m_pp.ldon_points_available < 0 || m_pp.ldon_points_available > 2000000000) { m_pp.ldon_points_available = 0; }
 
-	if (RuleB(World, UseClientBasedExpansionSettings)) {
-		m_pp.expansions = EQ::expansions::ConvertClientVersionToExpansionsMask(ClientVersion());
-	}
-	else {
-		m_pp.expansions = (RuleI(World, ExpansionSettings) & EQ::expansions::ConvertClientVersionToExpansionsMask(ClientVersion()));
-	}
+	ReloadExpansionProfileSetting();
 
 	if (!database.LoadAlternateAdvancement(this)) {
 		LogError("Error loading AA points for [{}]", GetName());
@@ -2138,7 +2141,7 @@ void Client::Handle_OP_AdventureMerchantRequest(const EQApplicationPacket *app)
 	const EQ::ItemData *item = nullptr;
 	std::list<MerchantList> merlist = zone->merchanttable[merchantid];
 	std::list<MerchantList>::const_iterator itr;
-	for (itr = merlist.begin(); itr != merlist.end() && count<255; ++itr) {
+	for (itr = merlist.begin(); itr != merlist.end(); ++itr) {
 		const MerchantList &ml = *itr;
 		if (GetLevel() < ml.level_required) {
 			continue;
@@ -4130,6 +4133,28 @@ void Client::Handle_OP_BoardBoat(const EQApplicationPacket *app)
 	return;
 }
 
+void Client::Handle_OP_BookButton(const EQApplicationPacket* app)
+{
+	if (app->size != sizeof(BookButton_Struct))
+	{
+		LogError("Size mismatch in OP_BookButton. expected [{}] got [{}]", sizeof(BookButton_Struct), app->size);
+		DumpPacket(app);
+		return;
+	}
+
+	BookButton_Struct* book = reinterpret_cast<BookButton_Struct*>(app->pBuffer);
+
+	const EQ::ItemInstance* const inst = GetInv().GetItem(book->invslot);
+	if (inst && inst->GetItem()->Book)
+	{
+		// todo: if scribe book learn recipes and delete book from inventory
+		// todo: if cast book use its spell on target and delete book from inventory (unless reusable?)
+	}
+
+	EQApplicationPacket outapp(OP_FinishWindow, 0);
+	QueuePacket(&outapp);
+}
+
 void Client::Handle_OP_Buff(const EQApplicationPacket *app)
 {
 	/*
@@ -5224,12 +5249,12 @@ void Client::Handle_OP_ConsiderCorpse(const EQApplicationPacket *app)
 
 	uint32 decay_time = t->GetDecayTime();
 	if (decay_time) {
-		auto time_string = Strings::SecondsToTime(decay_time, true);
+		const std::string& time_string = Strings::SecondsToTime(decay_time, true);
 		Message(
 			Chat::NPCQuestSay,
 			fmt::format(
 				"This corpse will decay in {}.",
-				time_string
+				Strings::ToLower(time_string)
 			).c_str()
 		);
 
@@ -12555,8 +12580,8 @@ void Client::Handle_OP_RaidCommand(const EQApplicationPacket* app)
 		if (!raid) {
 			break;
 		}
-
-		raid->SaveRaidNote(raid_command_packet->leader_name, raid_command_packet->note);
+		RaidNote_Struct* note = (RaidNote_Struct*)app->pBuffer;
+		raid->SaveRaidNote(raid_command_packet->leader_name, note->note);
 		raid->SendRaidNotesToWorld();
 		break;
 	}
@@ -14400,7 +14425,7 @@ void Client::Handle_OP_Split(const EQApplicationPacket *app)
 	if (raid)
 		raid->SplitMoney(raid->GetGroup(this), split->copper, split->silver, split->gold, split->platinum);
 	else if (group)
-		group->SplitMoney(split->copper, split->silver, split->gold, split->platinum);
+		group->SplitMoney(split->copper, split->silver, split->gold, split->platinum, this);
 
 	return;
 
@@ -15953,7 +15978,7 @@ void Client::Handle_OP_XTargetRequest(const EQApplicationPacket *app)
 			if (t.type == Type) {
 				Raid* r = GetRaid();
 				if (r) {
-					auto mm = entity_list.GetNPCByID(r->marked_npcs[t.assist_slot]);
+					auto mm = entity_list.GetNPCByID(r->marked_npcs[t.assist_slot].entity_id);
 					if (mm) {
 						UpdateXTargetType(t.type, mm->CastToMob(), mm->CastToMob()->GetName());
 					}
@@ -16392,3 +16417,101 @@ void Client::Handle_OP_RaidClearNPCMarks(const EQApplicationPacket* app)
 		r->RaidClearNPCMarks(this);
 	}
 }
+
+void Client::RecordStats()
+{
+	auto r = CharacterStatsRecordRepository::FindOne(
+		database,
+		CharacterID()
+	);
+
+	r.status                   = Admin();
+	r.name                     = GetCleanName();
+	r.aa_points                = GetAAPoints() + GetSpentAA();
+	r.level                    = GetLevel();
+	r.class_                   = GetBaseClass();
+	r.race                     = GetBaseRace();
+	r.hp                       = GetMaxHP() - GetSpellBonuses().HP;
+	r.mana                     = GetMaxMana() - GetSpellBonuses().Mana;
+	r.endurance                = GetMaxEndurance() - GetSpellBonuses().Endurance;
+	r.ac                       = GetDisplayAC() - GetSpellBonuses().AC;
+	r.strength                 = GetSTR() - GetSpellBonuses().STR;
+	r.stamina                  = GetSTA() - GetSpellBonuses().STA;
+	r.dexterity                = GetDEX() - GetSpellBonuses().DEX;
+	r.agility                  = GetAGI() - GetSpellBonuses().AGI;
+	r.intelligence             = GetINT() - GetSpellBonuses().INT;
+	r.wisdom                   = GetWIS() - GetSpellBonuses().WIS;
+	r.charisma                 = GetCHA() - GetSpellBonuses().CHA;
+	r.magic_resist             = GetMR() - GetSpellBonuses().MR;
+	r.fire_resist              = GetFR() - GetSpellBonuses().FR;
+	r.cold_resist              = GetCR() - GetSpellBonuses().CR;
+	r.poison_resist            = GetPR() - GetSpellBonuses().PR;
+	r.disease_resist           = GetDR() - GetSpellBonuses().DR;
+	r.corruption_resist        = GetCorrup() - GetSpellBonuses().Corrup;
+	r.heroic_strength          = GetHeroicSTR() - GetSpellBonuses().HeroicSTR;
+	r.heroic_stamina           = GetHeroicSTA() - GetSpellBonuses().HeroicSTA;
+	r.heroic_dexterity         = GetHeroicDEX() - GetSpellBonuses().HeroicDEX;
+	r.heroic_agility           = GetHeroicAGI() - GetSpellBonuses().HeroicAGI;
+	r.heroic_intelligence      = GetHeroicINT() - GetSpellBonuses().HeroicINT;
+	r.heroic_wisdom            = GetHeroicWIS() - GetSpellBonuses().HeroicWIS;
+	r.heroic_charisma          = GetHeroicCHA() - GetSpellBonuses().HeroicCHA;
+	r.heroic_magic_resist      = GetHeroicMR() - GetSpellBonuses().HeroicMR;
+	r.heroic_fire_resist       = GetHeroicFR() - GetSpellBonuses().HeroicFR;
+	r.heroic_cold_resist       = GetHeroicCR() - GetSpellBonuses().HeroicCR;
+	r.heroic_poison_resist     = GetHeroicPR() - GetSpellBonuses().HeroicPR;
+	r.heroic_disease_resist    = GetHeroicDR() - GetSpellBonuses().HeroicDR;
+	r.heroic_corruption_resist = GetHeroicCorrup() - GetSpellBonuses().HeroicCorrup;
+	r.haste                    = GetHaste();
+	r.accuracy                 = GetAccuracy() - GetSpellBonuses().Accuracy[EQ::skills::HIGHEST_SKILL + 1];
+	r.attack                   = GetTotalATK() - GetSpellBonuses().ATK;
+	r.avoidance                = GetAvoidance() - GetSpellBonuses().AvoidMeleeChance;
+	r.clairvoyance             = GetClair() - GetSpellBonuses().Clairvoyance;
+	r.combat_effects           = GetCombatEffects() - GetSpellBonuses().ProcChance;
+	r.damage_shield_mitigation = GetDSMit() - GetSpellBonuses().DSMitigation;
+	r.damage_shield            = GetDS() - GetSpellBonuses().DamageShield;
+	r.dot_shielding            = GetDoTShield() - GetSpellBonuses().DoTShielding;
+	r.hp_regen                 = GetHPRegen() - GetSpellBonuses().HPRegen;
+	r.mana_regen               = GetManaRegen() - GetSpellBonuses().ManaRegen;
+	r.endurance_regen          = GetEnduranceRegen() - GetSpellBonuses().EnduranceRegen;
+	r.shielding                = GetShielding() - GetSpellBonuses().MeleeMitigation;
+	r.spell_damage             = GetSpellDmg() - GetSpellBonuses().SpellDmg;
+	r.spell_shielding          = GetSpellShield() - GetSpellBonuses().SpellShield;
+	r.strikethrough            = GetStrikeThrough() - GetSpellBonuses().StrikeThrough;
+	r.stun_resist              = GetStunResist() - GetSpellBonuses().StunResist;
+	r.backstab                 = 0;
+	r.wind                     = GetWindMod();
+	r.brass                    = GetBrassMod();
+	r.string                   = GetStringMod();
+	r.percussion               = GetPercMod();
+	r.singing                  = GetSingMod();
+	r.baking                   = GetSkill(EQ::skills::SkillType::SkillBaking);
+	r.alchemy                  = GetSkill(EQ::skills::SkillType::SkillAlchemy);
+	r.jewelry                  = GetSkill(EQ::skills::SkillType::SkillJewelryMaking);
+	r.tailoring                = GetSkill(EQ::skills::SkillType::SkillTailoring);
+	r.blacksmithing            = GetSkill(EQ::skills::SkillType::SkillBlacksmithing);
+	r.fletching                = GetSkill(EQ::skills::SkillType::SkillFletching);
+	r.brewing                  = GetSkill(EQ::skills::SkillType::SkillBrewing);
+	r.fishing                  = GetSkill(EQ::skills::SkillType::SkillFishing);
+	r.pottery                  = GetSkill(EQ::skills::SkillType::SkillPottery);
+	r.alcohol                  = GetSkill(EQ::skills::SkillType::SkillAlcoholTolerance);
+	r.tinkering                = GetSkill(EQ::skills::SkillType::SkillTinkering);
+	r.updated_at               = std::time(nullptr);
+
+	if (r.character_id > 0) {
+		CharacterStatsRecordRepository::UpdateOne(database, r);
+	} else {
+		r.character_id = CharacterID();
+		r.created_at = std::time(nullptr);
+		CharacterStatsRecordRepository::InsertOne(database, r);
+	}
+}
+
+void Client::ReloadExpansionProfileSetting()
+{
+	if (RuleB(World, UseClientBasedExpansionSettings)) {
+		m_pp.expansions = EQ::expansions::ConvertClientVersionToExpansionsMask(ClientVersion());
+	}
+	else {
+		m_pp.expansions = RuleI(World, ExpansionSettings);
+	}
+}

zone/client_packet.h

@@ -56,6 +56,7 @@
 	void Handle_OP_Bind_Wound(const EQApplicationPacket *app);
 	void Handle_OP_BlockedBuffs(const EQApplicationPacket *app);
 	void Handle_OP_BoardBoat(const EQApplicationPacket *app);
+	void Handle_OP_BookButton(const EQApplicationPacket *app);
 	void Handle_OP_Buff(const EQApplicationPacket *app);
 	void Handle_OP_BuffRemoveRequest(const EQApplicationPacket *app);
 	void Handle_OP_Bug(const EQApplicationPacket *app);

zone/client_process.cpp

@@ -741,6 +741,8 @@ void Client::OnDisconnect(bool hard_disconnect) {
 		parse->EventPlayer(EVENT_DISCONNECT, this, "", 0);
 	}
 
+	RecordStats();
+
 	Disconnect();
 }
 
@@ -857,12 +859,12 @@ void Client::BulkSendMerchantInventory(int merchant_id, int npcid) {
 			DataBucketKey k = GetScopedBucketKeys();
 			k.key = bucket_name;
 
-			auto const& player_value = DataBucket::CheckBucketKey(this, k);
-			if (player_value.empty()) {
+			auto b = DataBucket::GetData(k);
+			if (b.value.empty()) {
 				continue;
 			}
 
-			if (!zone->CompareDataBucket(ml.bucket_comparison, bucket_value, player_value)) {
+			if (!zone->CompareDataBucket(ml.bucket_comparison, bucket_value, b.value)) {
 				continue;
 			}
 		}

zone/command.cpp

@@ -137,6 +137,7 @@ int command_init(void)
 		command_add("givemoney", "[Platinum] [Gold] [Silver] [Copper] - Gives specified amount of money to you or your player target", AccountStatus::GMMgmt, command_givemoney) ||
 		command_add("gmzone", "[Zone ID|Zone Short Name] [Version] [Instance Identifier] - Zones to a private GM instance (Version defaults to 0 and Instance Identifier defaults to 'gmzone' if not used)", AccountStatus::GMAdmin, command_gmzone) ||
 		command_add("goto", "[playername] or [x y z] [h] - Teleport to the provided coordinates or to your target", AccountStatus::Steward, command_goto) ||
+		command_add("grantaa", "[level] - Grants a player all available AA points up the specified level, all AAs are granted if no level is specified.", AccountStatus::GMMgmt, command_grantaa) ||
 		command_add("grid", "[add/delete] [grid_num] [wandertype] [pausetype] - Create/delete a wandering grid", AccountStatus::GMAreas, command_grid) ||
 		command_add("guild", "Guild manipulation commands. Use argument help for more info.", AccountStatus::Steward, command_guild) ||
 		command_add("help", "[Search Criteria] - List available commands and their description, specify partial command as argument to search", AccountStatus::Player, command_help) ||
@@ -436,13 +437,13 @@ int command_realdispatch(Client *c, std::string message, bool ignore_status)
 {
 	Seperator sep(message.c_str(), ' ', 10, 100, true); // "three word argument" should be considered 1 arg
 
-	std::string cstr(sep.arg[0] + 1);
+	std::string command(sep.arg[0] + 1);
 
-	if (commandlist.count(cstr) != 1) {
+	if (commandlist.count(command) != 1) {
 		return -2;
 	}
 
-	auto cur = commandlist[cstr];
+	const CommandRecord* current_command = commandlist[command];
 
 	bool is_subcommand            = false;
 	bool can_use_subcommand       = false;
@@ -450,11 +451,11 @@ int command_realdispatch(Client *c, std::string message, bool ignore_status)
 
 	const auto arguments = sep.argnum;
 
-	if (arguments >= 2) {
+	if (arguments) {
 		const std::string& sub_command = sep.arg[1];
 
 		for (const auto &e : command_subsettings) {
-			if (e.sub_command == sub_command) {
+			if (e.parent_command == command && e.sub_command == sub_command) {
 				can_use_subcommand       = c->Admin() >= static_cast<int16>(e.access_level);
 				is_subcommand            = true;
 				found_subcommand_setting = true;
@@ -464,7 +465,7 @@ int command_realdispatch(Client *c, std::string message, bool ignore_status)
 
 		if (!found_subcommand_setting) {
 			for (const auto &e: command_subsettings) {
-				if (e.sub_command == sub_command) {
+				if (e.parent_command == command && e.sub_command == sub_command) {
 					can_use_subcommand = c->Admin() >= static_cast<int16>(e.access_level);
 					is_subcommand      = true;
 					break;
@@ -474,7 +475,7 @@ int command_realdispatch(Client *c, std::string message, bool ignore_status)
 	}
 
 	if (!ignore_status) {
-		if (!is_subcommand && c->Admin() < cur->admin) {
+		if (!is_subcommand && c->Admin() < current_command->admin) {
 			c->Message(Chat::White, "Your status is not high enough to use this command.");
 			return -1;
 		}
@@ -496,7 +497,7 @@ int command_realdispatch(Client *c, std::string message, bool ignore_status)
 		QServ->PlayerLogEvent(Player_Log_Issued_Commands, c->CharacterID(), event_desc);
 	}
 
-	if (cur->admin >= COMMANDS_LOGGING_MIN_STATUS) {
+	if (current_command->admin >= COMMANDS_LOGGING_MIN_STATUS) {
 		LogCommands(
 			"[{}] ([{}]) used command: [{}] (target=[{}])",
 			c->GetName(),
@@ -506,8 +507,8 @@ int command_realdispatch(Client *c, std::string message, bool ignore_status)
 		);
 	}
 
-	if (!cur->function) {
-		LogError("Command [{}] has a null function", cstr);
+	if (!current_command->function) {
+		LogError("Command [{}] has a null function", command);
 		return -1;
 	}
 
@@ -524,7 +525,7 @@ int command_realdispatch(Client *c, std::string message, bool ignore_status)
 		RecordPlayerEventLogWithClient(c, PlayerEvent::GM_COMMAND, e);
 	}
 
-	cur->function(c, &sep);	// Dispatch C++ Command
+	current_command->function(c, &sep);	// Dispatch C++ Command
 
 	return 0;
 }
@@ -664,7 +665,7 @@ void command_hotfix(Client *c, const Seperator *sep)
 				hotfix_command = fmt::format("\"{}\" -hotfix={}", shared_memory_path, hotfix_name);
 			}
 			else {
-				hotfix_command = fmt::format("\"{}\"", shared_memory_path, hotfix_name);
+				hotfix_command = fmt::format("\"{}\"", shared_memory_path);
 			}
 
 			LogInfo("Running hotfix command [{}]", hotfix_command);
@@ -690,7 +691,7 @@ void command_hotfix(Client *c, const Seperator *sep)
 			}
 			worldserver.SendPacket(&pack);
 
-			if (c) { c->Message(Chat::White, "Hotfix applied"); }
+			worldserver.SendEmoteMessage(0, 0, AccountStatus::ApprenticeGuide, Chat::Yellow, "Hotfix applied");
 		}
 	);
 
@@ -826,6 +827,7 @@ void command_bot(Client *c, const Seperator *sep)
 #include "gm_commands/givemoney.cpp"
 #include "gm_commands/gmzone.cpp"
 #include "gm_commands/goto.cpp"
+#include "gm_commands/grantaa.cpp"
 #include "gm_commands/grid.cpp"
 #include "gm_commands/guild.cpp"
 #include "gm_commands/hp.cpp"

zone/command.h

@@ -30,6 +30,7 @@ uint8 GetCommandStatus(std::string command_name);
 void ListModifyNPCStatMap(Client *c);
 std::map<std::string, std::string> GetModifyNPCStatMap();
 std::string GetModifyNPCStatDescription(std::string stat);
+void SendFeatureSubCommands(Client *c);
 void SendNPCEditSubCommands(Client *c);
 void SendRuleSubCommands(Client *c);
 void SendGuildSubCommands(Client *c);
@@ -86,6 +87,7 @@ void command_giveitem(Client *c, const Seperator *sep);
 void command_givemoney(Client *c, const Seperator *sep);
 void command_gmzone(Client *c, const Seperator *sep);
 void command_goto(Client *c, const Seperator *sep);
+void command_grantaa(Client* c, const Seperator* sep);
 void command_grid(Client *c, const Seperator *sep);
 void command_guild(Client *c, const Seperator *sep);
 void command_help(Client *c, const Seperator *sep);
@@ -134,7 +136,7 @@ void command_petitems(Client *c, const Seperator *sep);
 void command_picklock(Client *c, const Seperator *sep);
 void command_profanity(Client *c, const Seperator *sep);
 void command_push(Client *c, const Seperator *sep);
-void command_pvp(Client *c, const Seperator *sep);;
+void command_pvp(Client *c, const Seperator *sep);
 void command_raidloot(Client* c, const Seperator* sep);
 void command_randomfeatures(Client *c, const Seperator *sep);
 void command_refreshgroup(Client *c, const Seperator *sep);

zone/common.h

@@ -207,6 +207,8 @@ enum {
 	MODIFY_AVOID_DAMAGE = 51,                    //Modify by percent the NPCs chance to riposte, block, parry or dodge individually, or for all skills
 	IMMUNE_FADING_MEMORIES = 52,
 	IMMUNE_OPEN = 53,
+	IMMUNE_ASSASSINATE = 54,
+	IMMUNE_HEADSHOT = 55,
 	MAX_SPECIAL_ATTACK
 };
 
@@ -891,7 +893,7 @@ struct ExtraAttackOptions {
 	int hit_chance;
 	int melee_damage_bonus_flat;
 	int skilldmgtaken_bonus_flat;
-
+	int range_percent;
 };
 
 struct DamageTable {

zone/corpse.cpp

@@ -1207,7 +1207,7 @@ void Corpse::MakeLootRequestPackets(Client* client, const EQApplicationPacket* a
 			client->Message(
 				Chat::Yellow,
 				fmt::format(
-					"This corpse Contains {}.",
+					"This corpse contains {}.",
 					Strings::Money(
 						GetPlatinum(),
 						GetGold(),
@@ -1217,7 +1217,7 @@ void Corpse::MakeLootRequestPackets(Client* client, const EQApplicationPacket* a
 				).c_str()
 			);
 		} else {
-			client->Message(Chat::Yellow, "This corpse Contains no money.");
+			client->Message(Chat::Yellow, "This corpse contains no money.");
 		}
 
 		auto outapp = new EQApplicationPacket(OP_MoneyOnCorpse, sizeof(moneyOnCorpseStruct));

zone/data_bucket.cpp

@@ -1,9 +1,14 @@
 #include "data_bucket.h"
+#include "entity.h"
 #include "zonedb.h"
 #include "mob.h"
+#include "worldserver.h"
 #include <ctime>
 #include <cctype>
-#include "../common/repositories/data_buckets_repository.h"
+
+extern WorldServer worldserver;
+
+std::vector<DataBucketCacheEntry> g_data_bucket_cache = {};
 
 void DataBucket::SetData(const std::string &bucket_key, const std::string &bucket_value, std::string expires_time)
 {
@@ -21,50 +26,68 @@ void DataBucket::SetData(const std::string &bucket_key, const std::string &bucke
 
 void DataBucket::SetData(const DataBucketKey &k)
 {
-	auto r = DataBucketsRepository::GetWhere(
-		database,
-		fmt::format(
-			"{} `key` = '{}' AND (`expires` > {} OR `expires` = 0) LIMIT 1",
-			DataBucket::GetScopedDbFilters(k),
-			Strings::Escape(k.key),
-			(long long) std::time(nullptr)
-		)
-	);
-
-	// if we have an entry, use it
 	auto b = DataBucketsRepository::NewEntity();
-	if (!r.empty()) {
-		b = r[0];
+	auto r = GetData(k, true);
+	// if we have an entry, use it
+	if (r.id > 0) {
+		b = r;
 	}
 
+	// add scoping to bucket
 	if (k.character_id > 0) {
 		b.character_id = k.character_id;
-	} else if (k.npc_id > 0) {
+	}
+	else if (k.npc_id > 0) {
 		b.npc_id = k.npc_id;
-	} else if (k.bot_id > 0) {
+	}
+	else if (k.bot_id > 0) {
 		b.bot_id = k.bot_id;
 	}
 
-	uint64    bucket_id         = b.id;
-	long long expires_time_unix = 0;
+	const uint64 bucket_id         = b.id;
+	int64        expires_time_unix = 0;
 
 	if (!k.expires.empty()) {
-		expires_time_unix = (long long) std::time(nullptr) + Strings::ToInt(k.expires);
+		expires_time_unix = static_cast<int64>(std::time(nullptr)) + Strings::ToInt(k.expires);
 		if (isalpha(k.expires[0]) || isalpha(k.expires[k.expires.length() - 1])) {
-			expires_time_unix = (long long) std::time(nullptr) + Strings::TimeToSeconds(k.expires);
+			expires_time_unix = static_cast<int64>(std::time(nullptr)) + Strings::TimeToSeconds(k.expires);
 		}
 	}
 
-	if (bucket_id > 0) {
-		b.expires = expires_time_unix;
-		b.value   = k.value;
+	b.expires = expires_time_unix;
+	b.value   = k.value;
+
+	if (bucket_id) {
+		// loop cache and update cache value and timestamp
+		for (auto &ce: g_data_bucket_cache) {
+			if (CheckBucketMatch(ce.e, k)) {
+				ce.e             = b;
+				ce.updated_time  = GetCurrentTimeUNIX();
+				ce.update_action = DataBucketCacheUpdateAction::Upsert;
+				SendDataBucketCacheUpdate(ce);
+				break;
+			}
+		}
+
 		DataBucketsRepository::UpdateOne(database, b);
 	}
 	else {
-		b.expires = expires_time_unix;
-		b.key_    = k.key;
-		b.value   = k.value;
-		DataBucketsRepository::InsertOne(database, b);
+		b.key_ = k.key;
+		b = DataBucketsRepository::InsertOne(database, b);
+		if (!ExistsInCache(b)) {
+			// add data bucket and timestamp to cache
+			auto ce = DataBucketCacheEntry{
+				.e = b,
+				.updated_time = DataBucket::GetCurrentTimeUNIX(),
+				.update_action = DataBucketCacheUpdateAction::Upsert
+			};
+
+			g_data_bucket_cache.emplace_back(ce);
+
+			SendDataBucketCacheUpdate(ce);
+
+			DeleteFromMissesCache(b);
+		}
 	}
 }
 
@@ -72,26 +95,114 @@ std::string DataBucket::GetData(const std::string &bucket_key)
 {
 	DataBucketKey k = {};
 	k.key = bucket_key;
-	return GetData(k);
+	return GetData(k).value;
 }
 
-std::string DataBucket::GetData(const DataBucketKey &k)
+// GetData fetches bucket data from the database or cache if it exists
+// if the bucket doesn't exist, it will be added to the cache as a miss
+// if ignore_misses_cache is true, the bucket will not be added to the cache as a miss
+// the only place we should be ignoring the misses cache is on the initial read during SetData
+DataBucketsRepository::DataBuckets DataBucket::GetData(const DataBucketKey &k, bool ignore_misses_cache)
 {
+	LogDataBuckets(
+		"Getting bucket key [{}] bot_id [{}] character_id [{}] npc_id [{}]",
+		k.key,
+		k.bot_id,
+		k.character_id,
+		k.npc_id
+	);
+
+	for (const auto &ce: g_data_bucket_cache) {
+		if (CheckBucketMatch(ce.e, k)) {
+			if (ce.e.expires > 0 && ce.e.expires < std::time(nullptr)) {
+				LogDataBuckets("Attempted to read expired key [{}] removing from cache", ce.e.key_);
+				DeleteData(k);
+				return DataBucketsRepository::NewEntity();
+			}
+
+			// this is a bucket miss, return empty entity
+			// we still cache bucket misses, so we don't have to hit the database
+			if (ce.e.id == 0) {
+				return DataBucketsRepository::NewEntity();
+			}
+
+			LogDataBuckets("Returning key [{}] value [{}] from cache", ce.e.key_, ce.e.value);
+			return ce.e;
+		}
+	}
+
 	auto r = DataBucketsRepository::GetWhere(
 		database,
 		fmt::format(
-			"{} `key` = '{}' AND (`expires` > {} OR `expires` = 0) LIMIT 1",
+			"{} `key` = '{}' LIMIT 1",
 			DataBucket::GetScopedDbFilters(k),
-			k.key,
-			(long long) std::time(nullptr)
+			k.key
 		)
 	);
 
 	if (r.empty()) {
+
+		// if we're ignoring the misses cache, don't add to the cache
+		// the only place this is ignored is during the initial read of SetData
+		if (!ignore_misses_cache) {
+			size_t size_before = g_data_bucket_cache.size();
+
+			// cache bucket misses, so we don't have to hit the database
+			// when scripts try to read a bucket that doesn't exist
+			g_data_bucket_cache.emplace_back(
+				DataBucketCacheEntry{
+					.e = DataBucketsRepository::DataBuckets{
+						.id = 0,
+						.key_ = k.key,
+						.value = "",
+						.expires = 0,
+						.character_id = k.character_id,
+						.npc_id = k.npc_id,
+						.bot_id = k.bot_id
+					},
+					.updated_time = DataBucket::GetCurrentTimeUNIX()
+				}
+			);
+
+			LogDataBuckets(
+				"Key [{}] not found in database, adding to cache as a miss character_id [{}] npc_id [{}] bot_id [{}] cache size before [{}] after [{}]",
+				k.key,
+				k.character_id,
+				k.npc_id,
+				k.bot_id,
+				size_before,
+				g_data_bucket_cache.size()
+			);
+		}
+
 		return {};
 	}
 
-	return r[0].value;
+	// if the entry has expired, delete it
+	if (r[0].expires > 0 && r[0].expires < (long long) std::time(nullptr)) {
+		DeleteData(k);
+		return {};
+	}
+
+	bool      has_cache = false;
+	for (auto &ce: g_data_bucket_cache) {
+		if (ce.e.id == r[0].id) {
+			has_cache = true;
+			break;
+		}
+	}
+
+	if (!has_cache) {
+		// add data bucket and timestamp to cache
+		g_data_bucket_cache.emplace_back(
+			DataBucketCacheEntry{
+				.e = r[0],
+				.updated_time = DataBucket::GetCurrentTimeUNIX()
+			}
+		);
+	}
+
+	return r[0];
 }
 
 std::string DataBucket::GetDataExpires(const std::string &bucket_key)
@@ -111,57 +222,68 @@ std::string DataBucket::GetDataRemaining(const std::string &bucket_key)
 
 bool DataBucket::DeleteData(const std::string &bucket_key)
 {
-	DataBucketKey r = {};
-	r.key = bucket_key;
-	return DeleteData(r);
+	DataBucketKey k = {};
+	k.key = bucket_key;
+	return DeleteData(k);
 }
 
+// GetDataBuckets bulk loads all data buckets for a mob
 bool DataBucket::GetDataBuckets(Mob *mob)
 {
-	DataBucketKey k = mob->GetScopedBucketKeys();
-	auto          l = BaseDataBucketsRepository::GetWhere(
-		database,
-		fmt::format(
-			"{} (`expires` > {} OR `expires` = 0)",
-			DataBucket::GetScopedDbFilters(k),
-			(long long) std::time(nullptr)
-		)
-	);
+	DataBucketLoadType::Type t;
+	const uint32             id = mob->GetMobTypeIdentifier();
 
-	if (l.empty()) {
+	if (!id) {
 		return false;
 	}
 
-	mob->m_data_bucket_cache.clear();
-
-	DataBucketCache d;
-
-	for (const auto &e: l) {
-		d.bucket_id      = e.id;
-		d.bucket_key     = e.key_;
-		d.bucket_value   = e.value;
-		d.bucket_expires = e.expires;
-
-		mob->m_data_bucket_cache.emplace_back(d);
+	if (mob->IsBot()) {
+		t = DataBucketLoadType::Bot;
 	}
+	else if (mob->IsClient()) {
+		t = DataBucketLoadType::Client;
+	}
+	else if (mob->IsNPC()) {
+		t = DataBucketLoadType::NPC;
+	}
+
+	BulkLoadEntities(t, {id});
 
 	return true;
 }
 
-std::string DataBucket::CheckBucketKey(const Mob *mob, const DataBucketKey &k)
-{
-	std::string     bucket_value;
-	for (const auto &d: mob->m_data_bucket_cache) {
-		if (d.bucket_key == k.key) {
-			bucket_value = d.bucket_value;
-			break;
-		}
-	}
-	return bucket_value;
-}
-
 bool DataBucket::DeleteData(const DataBucketKey &k)
 {
+	size_t size_before = g_data_bucket_cache.size();
+
+	// delete from cache where contents match
+	g_data_bucket_cache.erase(
+		std::remove_if(
+			g_data_bucket_cache.begin(),
+			g_data_bucket_cache.end(),
+			[&](DataBucketCacheEntry &ce) {
+				bool match = CheckBucketMatch(ce.e, k);
+				if (match) {
+					ce.update_action = DataBucketCacheUpdateAction::Delete;
+					SendDataBucketCacheUpdate(ce);
+				}
+
+				return match;
+			}
+		),
+		g_data_bucket_cache.end()
+	);
+
+	LogDataBuckets(
+		"Deleting bucket key [{}] bot_id [{}] character_id [{}] npc_id [{}] cache size before [{}] after [{}]",
+		k.key,
+		k.bot_id,
+		k.character_id,
+		k.npc_id,
+		size_before,
+		g_data_bucket_cache.size()
+	);
+
 	return DataBucketsRepository::DeleteWhere(
 		database,
 		fmt::format(
@@ -174,40 +296,38 @@ bool DataBucket::DeleteData(const DataBucketKey &k)
 
 std::string DataBucket::GetDataExpires(const DataBucketKey &k)
 {
-	auto r = DataBucketsRepository::GetWhere(
-		database,
-		fmt::format(
-			"{} `key` = '{}' AND (`expires` > {} OR `expires` = 0) LIMIT 1",
-			DataBucket::GetScopedDbFilters(k),
-			k.key,
-			(long long) std::time(nullptr)
-		)
+	LogDataBuckets(
+		"Getting bucket expiration key [{}] bot_id [{}] character_id [{}] npc_id [{}]",
+		k.key,
+		k.bot_id,
+		k.character_id,
+		k.npc_id
 	);
 
-	if (r.empty()) {
+	auto r = GetData(k);
+	if (r.id == 0) {
 		return {};
 	}
 
-	return fmt::format("{}", r[0].expires);
+	return std::to_string(r.expires);
 }
 
 std::string DataBucket::GetDataRemaining(const DataBucketKey &k)
 {
-	auto r = DataBucketsRepository::GetWhere(
-		database,
-		fmt::format(
-			"{} `key` = '{}' AND (`expires` > {} OR `expires` = 0) LIMIT 1",
-			DataBucket::GetScopedDbFilters(k),
-			k.key,
-			(long long) std::time(nullptr)
-		)
+	LogDataBuckets(
+		"Getting bucket remaining key [{}] bot_id [{}] character_id [{}] npc_id [{}]",
+		k.key,
+		k.bot_id,
+		k.character_id,
+		k.npc_id
 	);
 
-	if (r.empty()) {
+	auto r = GetData(k);
+	if (r.id == 0) {
 		return "0";
 	}
 
-	return fmt::format("{}", r[0].expires - (long long) std::time(nullptr));
+	return fmt::format("{}", r.expires - (long long) std::time(nullptr));
 }
 
 std::string DataBucket::GetScopedDbFilters(const DataBucketKey &k)
@@ -216,12 +336,23 @@ std::string DataBucket::GetScopedDbFilters(const DataBucketKey &k)
 	if (k.character_id > 0) {
 		query.emplace_back(fmt::format("character_id = {}", k.character_id));
 	}
-	else if (k.npc_id > 0) {
+	else {
+		query.emplace_back("character_id = 0");
+	}
+
+	if (k.npc_id > 0) {
 		query.emplace_back(fmt::format("npc_id = {}", k.npc_id));
 	}
-	else if (k.bot_id > 0) {
+	else {
+		query.emplace_back("npc_id = 0");
+	}
+
+	if (k.bot_id > 0) {
 		query.emplace_back(fmt::format("bot_id = {}", k.bot_id));
 	}
+	else {
+		query.emplace_back("bot_id = 0");
+	}
 
 	return fmt::format(
 		"{} {}",
@@ -229,3 +360,314 @@ std::string DataBucket::GetScopedDbFilters(const DataBucketKey &k)
 		!query.empty() ? "AND" : ""
 	);
 }
+
+bool DataBucket::CheckBucketMatch(const DataBucketsRepository::DataBuckets &dbe, const DataBucketKey &k)
+{
+	return (
+		dbe.key_ == k.key &&
+		dbe.bot_id == k.bot_id &&
+		dbe.character_id == k.character_id &&
+		dbe.npc_id == k.npc_id
+	);
+}
+
+void DataBucket::BulkLoadEntities(DataBucketLoadType::Type t, std::vector<uint32> ids)
+{
+	if (ids.empty()) {
+		return;
+	}
+
+	if (ids.size() == 1) {
+		bool            has_cache = false;
+		for (const auto &ce: g_data_bucket_cache) {
+			if (t == DataBucketLoadType::Bot) {
+				has_cache = ce.e.bot_id == ids[0];
+			}
+			else if (t == DataBucketLoadType::Client) {
+				has_cache = ce.e.character_id == ids[0];
+			}
+			else if (t == DataBucketLoadType::NPC) {
+				has_cache = ce.e.npc_id == ids[0];
+			}
+		}
+
+		if (has_cache) {
+			LogDataBucketsDetail("LoadType [{}] ID [{}] has cache", DataBucketLoadType::Name[t], ids[0]);
+			return;
+		}
+	}
+
+	std::string column;
+
+	switch (t) {
+		case DataBucketLoadType::Bot:
+			column = "bot_id";
+			break;
+		case DataBucketLoadType::Client:
+			column = "character_id";
+			break;
+		case DataBucketLoadType::NPC:
+			column = "npc_id";
+			break;
+		default:
+			LogError("Incorrect LoadType [{}]", t);
+			break;
+	}
+
+	const auto &l = DataBucketsRepository::GetWhere(
+		database,
+		fmt::format(
+			"{} IN ({}) AND (`expires` > {} OR `expires` = 0)",
+			column,
+			Strings::Join(ids, ", "),
+			(long long) std::time(nullptr)
+		)
+	);
+
+	if (l.empty()) {
+		return;
+	}
+
+	size_t size_before = g_data_bucket_cache.size();
+
+	LogDataBucketsDetail("cache size before [{}] l size [{}]", g_data_bucket_cache.size(), l.size());
+
+	uint32 added_count = 0;
+
+	for (const auto &e: l) {
+		if (!ExistsInCache(e)) {
+			added_count++;
+		}
+	}
+
+	g_data_bucket_cache.reserve(g_data_bucket_cache.size() + added_count);
+
+	for (const auto &e: l) {
+		if (!ExistsInCache(e)) {
+			LogDataBucketsDetail("bucket id [{}] bucket key [{}] bucket value [{}]", e.id, e.key_, e.value);
+
+			g_data_bucket_cache.emplace_back(
+				DataBucketCacheEntry{
+					.e = e,
+					.updated_time = GetCurrentTimeUNIX()
+				}
+			);
+		}
+	}
+
+	LogDataBucketsDetail("cache size after [{}]", g_data_bucket_cache.size());
+
+	LogDataBuckets(
+		"Bulk Loaded ids [{}] column [{}] new cache size is [{}]",
+		ids.size(),
+		column,
+		g_data_bucket_cache.size()
+	);
+}
+
+void DataBucket::DeleteCachedBuckets(DataBucketLoadType::Type t, uint32 id)
+{
+	size_t size_before = g_data_bucket_cache.size();
+
+	g_data_bucket_cache.erase(
+		std::remove_if(
+			g_data_bucket_cache.begin(),
+			g_data_bucket_cache.end(),
+			[&](DataBucketCacheEntry &ce) {
+				return (
+					(t == DataBucketLoadType::Bot && ce.e.bot_id == id) ||
+					(t == DataBucketLoadType::Client && ce.e.character_id == id) ||
+					(t == DataBucketLoadType::NPC && ce.e.npc_id == id)
+				);
+			}
+		),
+		g_data_bucket_cache.end()
+	);
+
+	LogDataBuckets(
+		"LoadType [{}] id [{}] cache size before [{}] after [{}]",
+		DataBucketLoadType::Name[t],
+		id,
+		size_before,
+		g_data_bucket_cache.size()
+	);
+}
+
+int64_t DataBucket::GetCurrentTimeUNIX()
+{
+	return std::chrono::duration_cast<std::chrono::nanoseconds>(
+		std::chrono::system_clock::now().time_since_epoch()
+	).count();
+}
+
+bool DataBucket::ExistsInCache(const DataBucketsRepository::DataBuckets &e)
+{
+	for (const auto &ce: g_data_bucket_cache) {
+		if (ce.e.id == e.id) {
+			return true;
+		}
+	}
+
+	return false;
+}
+
+bool DataBucket::SendDataBucketCacheUpdate(const DataBucketCacheEntry &e)
+{
+	if (!e.e.id) {
+		return false;
+	}
+
+	EQ::Net::DynamicPacket p;
+	p.PutSerialize(0, e);
+
+	auto pack_size = sizeof(ServerDataBucketCacheUpdate_Struct) + p.Length();
+	auto pack      = new ServerPacket(ServerOP_DataBucketCacheUpdate, static_cast<uint32_t>(pack_size));
+	auto buf       = reinterpret_cast<ServerDataBucketCacheUpdate_Struct *>(pack->pBuffer);
+
+	buf->cereal_size = static_cast<uint32_t>(p.Length());
+
+	memcpy(buf->cereal_data, p.Data(), p.Length());
+
+	worldserver.SendPacket(pack);
+
+	return true;
+}
+
+void DataBucket::HandleWorldMessage(ServerPacket *p)
+{
+	DataBucketCacheEntry         n;
+	auto                         s = (ServerDataBucketCacheUpdate_Struct *) p->pBuffer;
+	EQ::Util::MemoryStreamReader ss(s->cereal_data, s->cereal_size);
+	cereal::BinaryInputArchive   archive(ss);
+	archive(n);
+
+	LogDataBucketsDetail(
+		"Received cache packet for id [{}] key [{}] value [{}] action [{}]",
+		n.e.id,
+		n.e.key_,
+		n.e.value,
+		n.update_action
+	);
+
+	// delete
+	if (n.update_action == DataBucketCacheUpdateAction::Delete) {
+		DeleteFromMissesCache(n.e);
+
+		g_data_bucket_cache.erase(
+			std::remove_if(
+				g_data_bucket_cache.begin(),
+				g_data_bucket_cache.end(),
+				[&](DataBucketCacheEntry &ce) {
+					bool match = n.e.id > 0 && ce.e.id == n.e.id;
+					if (match) {
+						LogDataBuckets(
+							"[delete] cache key [{}] id [{}] cache_size before [{}] after [{}]",
+							ce.e.key_,
+							ce.e.id,
+							g_data_bucket_cache.size(),
+							g_data_bucket_cache.size() - 1
+						);
+					}
+					return match;
+				}
+			),
+			g_data_bucket_cache.end()
+		);
+
+		return;
+	}
+
+	// update
+	bool has_key = false;
+	for (auto &ce: g_data_bucket_cache) {
+		// update cache
+		if (ce.e.id == n.e.id) {
+			// reject old updates
+			int64 time_delta = ce.updated_time - n.updated_time;
+			if (ce.updated_time >= n.updated_time) {
+				LogDataBuckets(
+					"Attempted to update older cache key [{}] rejecting old time [{}] new time [{}] delta [{}] cache_size [{}]",
+					ce.e.key_,
+					ce.updated_time,
+					n.updated_time,
+					time_delta,
+					g_data_bucket_cache.size()
+				);
+				return;
+			}
+
+			DeleteFromMissesCache(n.e);
+
+			LogDataBuckets(
+				"[update] cache id [{}] key [{}] value [{}] old time [{}] new time [{}] delta [{}] cache_size [{}]",
+				ce.e.id,
+				ce.e.key_,
+				n.e.value,
+				ce.updated_time,
+				n.updated_time,
+				time_delta,
+				g_data_bucket_cache.size()
+			);
+			ce.e            = n.e;
+			ce.updated_time = n.updated_time;
+			has_key = true;
+			break;
+		}
+	}
+
+	// create
+	if (!has_key) {
+		DeleteFromMissesCache(n.e);
+
+		size_t size_before = g_data_bucket_cache.size();
+
+		g_data_bucket_cache.emplace_back(
+			DataBucketCacheEntry{
+				.e = n.e,
+				.updated_time = GetCurrentTimeUNIX()
+			}
+		);
+
+		LogDataBuckets(
+			"[create] Adding new cache id [{}] key [{}] value [{}] cache size before [{}] after [{}]",
+			n.e.id,
+			n.e.key_,
+			n.e.value,
+			size_before,
+			g_data_bucket_cache.size()
+		);
+	}
+}
+
+void DataBucket::DeleteFromMissesCache(DataBucketsRepository::DataBuckets e)
+{
+	// delete from cache where there might have been a written bucket miss to the cache
+	// this is to prevent the cache from growing too large
+	size_t size_before = g_data_bucket_cache.size();
+
+	g_data_bucket_cache.erase(
+		std::remove_if(
+			g_data_bucket_cache.begin(),
+			g_data_bucket_cache.end(),
+			[&](DataBucketCacheEntry &ce) {
+				return ce.e.id == 0 && ce.e.key_ == e.key_ &&
+					   ce.e.character_id == e.character_id &&
+					   ce.e.npc_id == e.npc_id &&
+					   ce.e.bot_id == e.bot_id;
+			}
+		),
+		g_data_bucket_cache.end()
+	);
+	LogDataBucketsDetail(
+		"Deleted bucket misses from cache where key [{}] size before [{}] after [{}]",
+		e.key_,
+		size_before,
+		g_data_bucket_cache.size()
+	);
+}
+
+void DataBucket::ClearCache()
+{
+	g_data_bucket_cache.clear();
+	LogInfo("Cleared data buckets cache");
+}

zone/data_bucket.h

@@ -9,7 +9,29 @@
 #include "../common/types.h"
 #include "../common/repositories/data_buckets_repository.h"
 #include "mob.h"
+#include "../common/json/json_archive_single_line.h"
+#include "../common/servertalk.h"
 
+enum DataBucketCacheUpdateAction : uint8 {
+	Upsert,
+	Delete
+};
+
+struct DataBucketCacheEntry {
+	DataBucketsRepository::DataBuckets e;
+	int64_t                            updated_time{};
+	DataBucketCacheUpdateAction        update_action{};
+
+	template<class Archive>
+	void serialize(Archive &ar)
+	{
+		ar(
+			CEREAL_NVP(e),
+			CEREAL_NVP(updated_time),
+			CEREAL_NVP(update_action)
+		);
+	}
+};
 
 struct DataBucketKey {
 	std::string key;
@@ -20,25 +42,53 @@ struct DataBucketKey {
 	int64_t     bot_id;
 };
 
+namespace DataBucketLoadType {
+	enum Type : uint8 {
+		Bot,
+		Client,
+		NPC,
+		MaxType
+	};
+
+	static const std::string Name[Type::MaxType] = {
+		"Bot",
+		"Client",
+		"NPC",
+	};
+}
+
 class DataBucket {
 public:
 	// non-scoped bucket methods (for global buckets)
-	static void SetData(const std::string& bucket_key, const std::string& bucket_value, std::string expires_time = "");
-	static bool DeleteData(const std::string& bucket_key);
-	static std::string GetData(const std::string& bucket_key);
-	static std::string GetDataExpires(const std::string& bucket_key);
-	static std::string GetDataRemaining(const std::string& bucket_key);
+	static void SetData(const std::string &bucket_key, const std::string &bucket_value, std::string expires_time = "");
+	static bool DeleteData(const std::string &bucket_key);
+	static std::string GetData(const std::string &bucket_key);
+	static std::string GetDataExpires(const std::string &bucket_key);
+	static std::string GetDataRemaining(const std::string &bucket_key);
 
-	static bool GetDataBuckets(Mob* mob);
+	static bool GetDataBuckets(Mob *mob);
+
+	static int64_t GetCurrentTimeUNIX();
 
 	// scoped bucket methods
-	static void SetData(const DataBucketKey& k);
-	static bool DeleteData(const DataBucketKey& k);
-	static std::string GetData(const DataBucketKey& k);
-	static std::string GetDataExpires(const DataBucketKey& k);
-	static std::string GetDataRemaining(const DataBucketKey& k);
-	static std::string CheckBucketKey(const Mob* mob, const DataBucketKey& k);
-	static std::string GetScopedDbFilters(const DataBucketKey& k);
+	static void SetData(const DataBucketKey &k);
+	static bool DeleteData(const DataBucketKey &k);
+	static DataBucketsRepository::DataBuckets GetData(const DataBucketKey &k, bool ignore_misses_cache = false);
+	static std::string GetDataExpires(const DataBucketKey &k);
+	static std::string GetDataRemaining(const DataBucketKey &k);
+	static std::string GetScopedDbFilters(const DataBucketKey &k);
+
+	// bucket repository versus key matching
+	static bool CheckBucketMatch(const DataBucketsRepository::DataBuckets &dbe, const DataBucketKey &k);
+	static bool ExistsInCache(const DataBucketsRepository::DataBuckets &e);
+
+	static void BulkLoadEntities(DataBucketLoadType::Type t, std::vector<uint32> ids);
+	static void DeleteCachedBuckets(DataBucketLoadType::Type t, uint32 id);
+
+	static bool SendDataBucketCacheUpdate(const DataBucketCacheEntry &e);
+	static void HandleWorldMessage(ServerPacket *p);
+	static void DeleteFromMissesCache(DataBucketsRepository::DataBuckets e);
+	static void ClearCache();
 };
 
 #endif //EQEMU_DATABUCKET_H

zone/doors.cpp

@@ -83,6 +83,8 @@ Doors::Doors(const DoorsRepository::Doors &door) :
 	m_close_timer.Disable();
 
 	m_disable_timer = (door.disable_timer == 1 ? true : false);
+
+	m_is_blacklisted_to_open = GetIsDoorBlacklisted();
 }
 
 Doors::Doors(const char *model, const glm::vec4 &position, uint8 open_type, uint16 size) :
@@ -901,3 +903,27 @@ bool Doors::IsDestinationZoneSame() const
 {
 	return m_same_destination_zone;
 }
+
+// IsDoorBlacklisted has a static list of doors that are blacklisted
+// from being opened by NPCs. This is used to prevent NPCs from opening
+// doors that are not meant to be opened by NPCs.
+bool Doors::GetIsDoorBlacklisted()
+{
+	std::vector<std::string> blacklist = {
+		"TOGGLE",
+		"PNDRESSER101",
+	};
+
+	for (auto& name : blacklist) {
+		std::string door_name = GetDoorName();
+		if (name == door_name) {
+			return true;
+		}
+	}
+
+	return false;
+}
+
+bool Doors::IsDoorBlacklisted() {
+	return m_is_blacklisted_to_open;
+}

zone/doors.h

@@ -69,7 +69,10 @@ public:
 	bool HasDestinationZone() const;
 	bool IsDestinationZoneSame() const;
 
+	bool IsDoorBlacklisted();
+
 private:
+	bool GetIsDoorBlacklisted();
 
 	bool      m_has_destination_zone = false;
 	bool      m_same_destination_zone = false;
@@ -99,5 +102,6 @@ private:
 	uint8     m_is_ldon_door;
 	int       m_dz_switch_id = 0;
 	uint32    m_client_version_mask;
+	bool      m_is_blacklisted_to_open = false; // is door blacklisted to open by npcs
 };
 #endif