[Release] 23.7.0 (#4902)

* Add custom database version output to CLI from #4892

* Spacing alignment

CHANGELOG.md

@@ -1,3 +1,47 @@
+## [23.7.0] 5/19/2025
+
+### CLI
+
+* Add custom database version output ([#4901](https://github.com/EQEmu/Server/pull/4901)) @joligario 2025-05-18
+* Fix MySQL check in database dumper ([#4897](https://github.com/EQEmu/Server/pull/4897)) @joligario 2025-05-16
+
+### Commands
+
+* Add #zonevariable Command ([#4882](https://github.com/EQEmu/Server/pull/4882)) @Kinglykrab 2025-05-15
+
+### Database
+
+* Add Custom Database Migrations for Operators ([#4892](https://github.com/EQEmu/Server/pull/4892)) @Akkadius 2025-05-16
+* Remove Transaction Wrapped Character Save ([#4894](https://github.com/EQEmu/Server/pull/4894)) @Akkadius 2025-05-16
+
+### Fixes
+
+* Deadlock on failed #copycharacter commands ([#4887](https://github.com/EQEmu/Server/pull/4887)) @Akkadius 2025-05-16
+
+### Logging
+
+* Auto Update Log Category Names ([#4890](https://github.com/EQEmu/Server/pull/4890)) @Akkadius 2025-05-16
+
+### Netcode
+
+* Resend Logic Adjustments ([#4900](https://github.com/EQEmu/Server/pull/4900)) @Akkadius 2025-05-18
+
+### Player Events
+
+* Add rule to ignore configured GM commands ([#4888](https://github.com/EQEmu/Server/pull/4888)) @Akkadius 2025-05-16
+
+### Rules
+
+* Auto Update Rule Notes from Source ([#4891](https://github.com/EQEmu/Server/pull/4891)) @Akkadius 2025-05-16
+
+### World
+
+* Fix Rarer Reload Deadlock ([#4893](https://github.com/EQEmu/Server/pull/4893)) @Akkadius 2025-05-16
+
+### Zone State
+
+* Load New Spawn2 Data When Present ([#4889](https://github.com/EQEmu/Server/pull/4889)) @Akkadius 2025-05-16
+
 ## [23.6.0] 5/14/2025
 
 ### Bots

README.md

@@ -1,79 +1,147 @@
-# EQEmulator Core Server
+<h1 align="center">EQEmulator Server Platform</h1>
-| Drone (Linux x64) | Drone (Windows x64)   |
+
-|:---:|:---:|
+<p align="center">
-|[![Build Status](http://drone.akkadius.com/api/badges/EQEmu/Server/status.svg)](http://drone.akkadius.com/EQEmu/Server)   |[![Build Status](http://drone.akkadius.com/api/badges/EQEmu/Server/status.svg)](http://drone.akkadius.com/EQEmu/Server)   |
+ <img src="https://github.com/user-attachments/assets/11942e15-b512-402d-a619-0543c7f1151e" style="border-radius: 10px">
+</p>
+
+<p align="center">
+<b>EverQuest Emulator (EQEmu) - A Fan-Made Project Honoring the Legendary MMORPG</b>
+</p>
+
+<p align="center">
+<a href="https://github.com/eqemu/server/graphs/contributors"><img src="https://img.shields.io/github/contributors/eqemu/server" alt="Contributors"></a>
+<a href="https://discord.gg/QHsm7CD"><img src="https://img.shields.io/discord/212663220849213441?label=Discord&amp;logo=discord&amp;color=7289DA" alt="Discord"></a>
+<a href="https://docs.eqemu.io"><img src="https://img.shields.io/badge/docs-MkDocs%20Powered-blueviolet" alt="Docs"></a>
+<a href="./LICENSE"><img src="https://img.shields.io/github/license/EQEmu/Server" alt="License"></a> 
+<a href="https://github.com/eqemu/server/releases"><img src="https://img.shields.io/github/v/release/eqemu/server" alt="Latest Release"></a>
+<a href="https://github.com/EQEmu/Server/releases"><img src="https://img.shields.io/github/release-date/EQEmu/Server" alt="Release Date"></a>
+<img src="https://img.shields.io/github/downloads/eqemu/server/total.svg" alt="Github All Releases"></a>
+<a href="http://drone.akkadius.com/EQEmu/Server"><img src="http://drone.akkadius.com/api/badges/EQEmu/Server/status.svg" alt="Build Status"></a> 
+<img src="https://img.shields.io/github/issues-pr-closed/eqemu/server" alt="GitHub Issues or Pull Requests">
+<img src="https://img.shields.io/docker/pulls/akkadius/eqemu-server" alt="Docker Pulls">
+<a href="http://drone.akkadius.com/EQEmu/Server"><img src="http://drone.akkadius.com/api/badges/EQEmu/Server/status.svg" alt="Build Status"></a> <img src="https://jb.gg/badges/official-plastic.svg" alt="Official"> 
+
+</p>
 
 ***
 
-**EQEmulator is a custom completely from-scratch open source server implementation for EverQuest built mostly on C++**
+<p align="center">
- * MySQL/MariaDB is used as the database engine (over 200+ tables)
+EQEmulator is a <b>passion-driven</b>, <b>open source server emulator</b> project dedicated to preserving and celebrating the groundbreaking world of <b>EverQuest</b>, the massively multiplayer online role-playing game originally developed by <b>Verant Interactive</b> and <b>Sony Online Entertainment (now Daybreak Game Company)</b>.
- * Perl and LUA are both supported scripting languages for NPC/Player/Quest oriented events
+</p>
- * Open source database (Project EQ) has content up to expansion OoW (included in server installs)
-  * Game server environments and databases can be heavily customized to create all new experiences
- * Hundreds of Quests/events created and maintained by Project EQ
 
-## Server Installs
+<p align="center">
-| |Windows|Linux|
+For over two decades and continuing, EQEmulator has served as a <strong>fan tribute</strong>, providing tools and technology that allow players to explore, customize, and experience EverQuest’s iconic gameplay in new ways. This project exists solely out of <strong>deep admiration</strong> for the original developers, artists, designers, and visionaries who created one of the most influential online worlds of all time.
-|:---:|:---:|:---:|
+</p>
-|**Install Count**|![Windows Install Count](http://analytics.akkadius.com/?install_count&windows_count)|![Linux Install Count](http://analytics.akkadius.com/?install_count&linux_count)| 
-### > Windows 
 
-* [Install Guide](https://docs.eqemu.io/server/installation/server-installation-windows/)
+<p align="center">
+We do not claim ownership of EverQuest or its assets. <strong>All credit and respect belong to the original creators and Daybreak Game Company</strong>, whose work continues to inspire generations of players and developers alike.
+</p>
 
-### > Debian/Ubuntu/CentOS/Fedora
+<p align="center">
+EQEmulator has for over 20 years and always will be a <strong>fan-based, non-commercial open-source effort</strong> made by players, for players—preserving the legacy of EverQuest while empowering community-driven creativity, learning and joy that the game and its creators has so strongly inspired in us all.
+</p>
 
-* [Install Guide](https://docs.eqemu.io/server/installation/server-installation-linux/)
+***
 
-* You can use curl or wget to kick off the installer (whichever your OS has)
+<h3 align="center">
-> curl -O https://raw.githubusercontent.com/EQEmu/Server/master/utils/scripts/linux_installer/install.sh install.sh && chmod 755 install.sh && ./install.sh
+ Technical Overview & Reverse Engineering Effort
+</h1>
 
-> wget --no-check-certificate https://raw.githubusercontent.com/EQEmu/Server/master/utils/scripts/linux_installer/install.sh -O install.sh && chmod 755 install.sh && ./install.sh 
+<p align="center">EQEmulator represents <strong>over two decades of collaborative reverse engineering</strong>, rebuilding the EverQuest server from the ground up without access to the original source code. This effort was achieved entirely through <strong>community-driven analysis, network protocol decoding, and in-game behavioral research</strong>.</p>
 
-## Supported Clients
+<h1 align="center">
+ 💡 How We Did It
+</h1>
+
+<p align="center">
+ <img src="https://github.com/user-attachments/assets/b6b48cf7-f64a-4497-9750-71f442a3d132" height="300px">
+ </p>
+
+
+<p align="center">
+ <strong>Reverse Engineering</strong> 
+Every system, packet, opcode, and game mechanic has been reconstructed through countless hours of live packet sniffing, client disassembly, and in-game experimentation by dedicated contributors over the years.
+</p>
+
+<p align="center">
+ No proprietary code or server sources were ever used.
+</p>
+
+<p align="center">
+ All implementations are the result of clean-room engineering.
+</p>
+
+
+<h1 align="center">
+ 🛠️ Technology Stack
+</h1>
+
+<p align="center">
+ <img src="https://github.com/user-attachments/assets/df5ea809-86c5-439d-a8fa-651fb04ba477" style="border-radius: 10px">
+</p>
+
+**C++ Core Engine**
+
+* High-performance networking and gameplay logic built in C++
+* Cross-platform support for Linux and Windows
+
+**MySQL / MariaDB Backend**
+
+* Fully structured schema with over 200+ tables
+* Supports content customization, expansions, and custom worlds
+
+**Scripting Engine**
+
+* Native support for **Perl** and **Lua** scripting
+* Powerfully extendable for quests, NPC behaviors, and custom events
+
+**Open Source Content Database**
+
+* Includes ProjectEQ’s world data up through *Dragons of Norrath*
+* 100% customizable to create entirely new game worlds
+
+<h1 align="center">
+ 🚀 Why It Matters
+</h1>
+
+<p align="center">🧬 EQEmulator stands as a <strong>technical preservation project</strong>, ensuring that the magic of classic and custom EverQuest servers lives on for future generations of players, tinkerers, and game designers.
+</p>
+
+> We humbly acknowledge and thank the original developers at **Verant Interactive** and **Sony Online Entertainment (now Daybreak Game Company)** for creating one of the most influential online experiences in gaming history.
+
+<h1 align="center">
+ 🧑‍💻🖥️ Supported Clients
+</h1>
 
 |Titanium Edition|Secrets of Faydwer|Seeds of Destruction|Underfoot|Rain of Fear|
 |:---:|:---:|:---:|:---:|:---:|
 |<img src="http://i.imgur.com/hrwDxoM.jpg" height="150">|<img src="http://i.imgur.com/cRDW5tn.png" height="150">|<img src="http://i.imgur.com/V48kuVn.jpg" height="150">|<img src="http://i.imgur.com/IJQ0XMa.jpg" height="150">|<img src="http://i.imgur.com/OMpHkKa.png" height="100">|
 
-## Bug Reports <img src="http://i.imgur.com/daf1Vjw.png" height="20">
+## 📚 Resources
-* Please use the [issue tracker](https://github.com/EQEmu/Server/issues) provided by GitHub to send us bug
-reports or feature requests.
-* The [EQEmu Forums](http://www.eqemulator.org/forums/) are also a place to submit and get help with bugs.
 
-## Contributions <img src="http://image.flaticon.com/icons/png/512/25/25231.png" width="20">
+| Resource             | Badges                                                                                                                                                    | Link                                                                                              |
+|---------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------|
+| **EQEmulator Docs**  | [![Docs](https://img.shields.io/badge/docs-MkDocs%20Powered-blueviolet)](https://docs.eqemu.io)                                                             | [docs.eqemu.io](https://docs.eqemu.io/)                                                           |
+| **Discord Community**| [![Discord](https://img.shields.io/discord/212663220849213441?label=Discord&logo=discord&color=7289DA)](https://discord.gg/QHsm7CD)                         | [Join Discord](https://discord.gg/QHsm7CD)                                                        |
+| **Latest Release**   | [![Latest Release](https://img.shields.io/github/v/release/eqemu/server)](https://github.com/eqemu/server/releases) <br> [![Release Date](https://img.shields.io/github/release-date/EQEmu/Server)](https://github.com/EQEmu/Server/releases) <br> [![All Releases](https://img.shields.io/github/downloads/eqemu/server/total.svg)](https://github.com/eqemu/server/releases) | [View Releases](https://github.com/eqemu/server/releases)                                         |
+| **License**          | [![License](https://img.shields.io/github/license/EQEmu/Server)](./LICENSE)                                                                                | [View License](./LICENSE)                                                                        |
+| **Build Status**     | [![Build Status](http://drone.akkadius.com/api/badges/EQEmu/Server/status.svg)](http://drone.akkadius.com/EQEmu/Server)                                     | [View Build Status](http://drone.akkadius.com/EQEmu/Server)                                       |
+| **Docker Pulls**     | [![Docker Pulls](https://img.shields.io/docker/pulls/akkadius/eqemu-server)](https://hub.docker.com/r/akkadius/eqemu-server)                                | [Docker Hub](https://hub.docker.com/r/akkadius/eqemu-server)                                      |
+| **Contributions**    | [![GitHub PRs](https://img.shields.io/github/issues-pr-closed/eqemu/server)](https://github.com/eqemu/server/pulls?q=is%3Apr+is%3Aclosed)                   | [Closed PRs & Issues](https://github.com/eqemu/server/pulls?q=is%3Apr+is%3Aclosed)                |
 
-* The preferred way to contribute is to fork the repo and submit a pull request on
+## 🛠️ Getting Started
-GitHub. If you need help with your changes, you can always post on the forums or
-try Discord. You can also post unified diffs (`git diff` should do the trick) on the
-[Server Code Submissions](http://www.eqemulator.org/forums/forumdisplay.php?f=669)
-forum, although pull requests will be much quicker and easier on all parties.
 
-## Contact <img src="http://gamerescape.com/wp-content/uploads/2015/06/discord.png" height="20">
+If you want to set up your own EQEmulator server, please refer to the current [server installation guides](https://docs.eqemu.io/#server-installation). We've had 100,000s of players and developers use our guides to set up their own servers, and we hope you will too!
 
- - Discord Channel: https://discord.gg/QHsm7CD
+## 🗂️ Related Repositories
- - **User Discord Channel**: `#general`
- - **Developer Discord Channel**: `#eqemucoders`
 
-## Resources
+| Repository         | Description                                                                      |
-- [EQEmulator Forums](http://www.eqemulator.org/forums)
+|--------------------|----------------------------------------------------------------------------------|
-- [EQEmulator Wiki](https://docs.eqemu.io/)
+| [ProjectEQ Quests](https://github.com/ProjectEQ/projecteqquests)  | Official quests and event scripts for ProjectEQ                       |
+| [Maps](https://github.com/Akkadius/EQEmuMaps)                     | EQEmu-compatible zone maps                                            |
+| [Installer Resources](https://github.com/Akkadius/EQEmuInstall)   | Scripts and assets for setting up EQEmu servers                       |
+| [Zone Utilities](https://github.com/EQEmu/zone-utilities)         | Utilities for parsing, rendering, and manipulating EQ zone files      |
 
-## Related Repositories
-* [ProjectEQ Quests](https://github.com/ProjectEQ/projecteqquests)
-* [Maps](https://github.com/Akkadius/EQEmuMaps)
-* [Installer Resources](https://github.com/Akkadius/EQEmuInstall)
-* [Zone Utilities](https://github.com/EQEmu/zone-utilities) - Various utilities and libraries for parsing, rendering and manipulating EQ Zone files.
-
-## Other License Info
-
-* The server code and utilities are released under **GPLv3**
-* We also include some small libraries for convienence that may be under different licensing
-  * SocketLib - GPL LibXML
-  * zlib - zlib license
-  * MariaDB/MySQL - GPL
-  * GPL Perl - GPL / ActiveState (under the assumption that this is a free project)
-  * CPPUnit - GLP StringUtilities - Apache
-  * LUA - MIT
 
 ## Contributors
 

common/CMakeLists.txt

@@ -17,6 +17,7 @@ SET(common_sources
     database.cpp
     database_instances.cpp
     database/database_update_manifest.cpp
+    database/database_update_manifest_custom.cpp
     database/database_update_manifest_bots.cpp
     database/database_update.cpp
     dbcore.cpp

common/database.cpp

@@ -1920,6 +1920,7 @@ bool Database::CopyCharacter(
 	std::vector<std::string> tables_to_zero_id = {
 		"keyring",
 		"data_buckets",
+		"character_evolving_items",
 		"character_instance_safereturns",
 		"character_expedition_lockouts",
 		"character_instance_lockouts",
@@ -1951,6 +1952,12 @@ bool Database::CopyCharacter(
 			)
 		);
 
+		if (!results.Success()) {
+			LogError("Transaction failed [{}] rolling back", results.ErrorMessage());
+			TransactionRollback();
+			return false;
+		}
+
 		std::vector<std::string> columns      = {};
 		int                      column_count = 0;
 
@@ -1969,6 +1976,12 @@ bool Database::CopyCharacter(
 			)
 		);
 
+		if (!results.Success()) {
+			LogError("Transaction failed [{}] rolling back", results.ErrorMessage());
+			TransactionRollback();
+			return false;
+		}
+
 		std::vector<std::vector<std::string>> new_rows;
 
 		for (auto row : results) {
@@ -2036,13 +2049,18 @@ bool Database::CopyCharacter(
 			LogInfo("Copying table [{}] rows [{}]", table_name, Strings::Commify(rows_copied));
 
 			if (!insert.ErrorMessage().empty()) {
+				LogError("Error copying table [{}] [{}]", table_name, insert.ErrorMessage());
 				TransactionRollback();
 				return false;
 			}
 		}
 	}
 
-	TransactionCommit();
+	auto r = TransactionCommit();
+	if (!r.Success()) {
+		LogError("Transaction failed [{}] rolling back", r.ErrorMessage());
+		return false;
+	}
 
 	LogInfo(
 		"Character [{}] copied to [{}] total rows [{}]",

common/database/database_dump_service.cpp

@@ -50,7 +50,7 @@ bool DatabaseDumpService::IsMySQLInstalled()
 {
 	std::string version_output = GetMySQLVersion();
 
-	return version_output.find("mysql") != std::string::npos && version_output.find("Ver") != std::string::npos;
+	return version_output.find("mysql") != std::string::npos && (version_output.find("Ver") != std::string::npos || version_output.find("from") != std::string::npos);
 }
 
 /**

common/database/database_update.cpp

@@ -7,6 +7,7 @@
 #include "../http/httplib.h"
 
 #include "database_update_manifest.cpp"
+#include "database_update_manifest_custom.cpp"
 #include "database_update_manifest_bots.cpp"
 #include "database_dump_service.h"
 
@@ -14,7 +15,7 @@ constexpr int BREAK_LENGTH = 70;
 
 DatabaseVersion DatabaseUpdate::GetDatabaseVersions()
 {
-	auto results = m_database->QueryDatabase("SELECT `version`, `bots_version` FROM `db_version` LIMIT 1");
+	auto results = m_database->QueryDatabase("SELECT `version`, `bots_version`, `custom_version` FROM `db_version` LIMIT 1");
 	if (!results.Success() || !results.RowCount()) {
 		LogError("Failed to read from [db_version] table!");
 		return DatabaseVersion{};
@@ -25,6 +26,7 @@ DatabaseVersion DatabaseUpdate::GetDatabaseVersions()
 	return DatabaseVersion{
 		.server_database_version = Strings::ToInt(r[0]),
 		.bots_database_version = Strings::ToInt(r[1]),
+		.custom_database_version = Strings::ToInt(r[2]),
 	};
 }
 
@@ -33,6 +35,7 @@ DatabaseVersion DatabaseUpdate::GetBinaryDatabaseVersions()
 	return DatabaseVersion{
 		.server_database_version = CURRENT_BINARY_DATABASE_VERSION,
 		.bots_database_version = (RuleB(Bots, Enabled) ? CURRENT_BINARY_BOTS_DATABASE_VERSION : 0),
+		.custom_database_version = CUSTOM_BINARY_DATABASE_VERSION,
 	};
 }
 
@@ -43,6 +46,7 @@ constexpr int LOOK_BACK_AMOUNT = 10;
 // this check will take action
 void DatabaseUpdate::CheckDbUpdates()
 {
+	InjectCustomVersionColumn();
 	InjectBotsVersionColumn();
 	auto v = GetDatabaseVersions();
 	auto b = GetBinaryDatabaseVersions();
@@ -59,6 +63,15 @@ void DatabaseUpdate::CheckDbUpdates()
 		m_database->QueryDatabase(fmt::format("UPDATE `db_version` SET `version` = {}", b.server_database_version));
 	}
 
+	if (UpdateManifest(manifest_entries_custom, v.custom_database_version, b.custom_database_version)) {
+		LogInfo(
+			"Updates ran successfully, setting database version to [{}] from [{}]",
+			b.custom_database_version,
+			v.custom_database_version
+		);
+		m_database->QueryDatabase(fmt::format("UPDATE `db_version` SET `custom_version` = {}", b.custom_database_version));
+	}
+
 	if (b.bots_database_version > 0) {
 		if (UpdateManifest(bot_manifest_entries, v.bots_database_version, b.bots_database_version)) {
 			LogInfo(
@@ -344,6 +357,16 @@ bool DatabaseUpdate::CheckVersionsUpToDate(DatabaseVersion v, DatabaseVersion b)
 		);
 	}
 
+	if (b.custom_database_version > 0) {
+		LogInfo(
+			"{:>8} | database [{}] binary [{}] {}",
+			"Custom",
+			v.custom_database_version,
+			b.custom_database_version,
+			(v.custom_database_version == b.custom_database_version) ? "up to date" : "checking updates"
+		);
+	}
+
 	LogInfo("{:>8} | [server.auto_database_updates] [<green>true]", "Config");
 
 	LogInfo("{}", Strings::Repeat("-", BREAK_LENGTH));
@@ -353,7 +376,10 @@ bool DatabaseUpdate::CheckVersionsUpToDate(DatabaseVersion v, DatabaseVersion b)
 	// bots database version is optional, if not enabled then it is always up-to-date
 	bool bots_up_to_date   = RuleB(Bots, Enabled) ? v.bots_database_version >= b.bots_database_version : true;
 
-	return server_up_to_date && bots_up_to_date;
+	// custom database version is optional, if not enabled then it is always up-to-date
+	bool custom_up_to_date = v.custom_database_version >= b.custom_database_version;
+
+	return server_up_to_date && bots_up_to_date && custom_up_to_date;
 }
 
 // checks to see if there are pending updates
@@ -373,3 +399,12 @@ void DatabaseUpdate::InjectBotsVersionColumn()
 		m_database->QueryDatabase("ALTER TABLE db_version ADD bots_version int(11) DEFAULT '0' AFTER version");
 	}
 }
+
+void DatabaseUpdate::InjectCustomVersionColumn()
+{
+	auto results = m_database->QueryDatabase("SHOW COLUMNS FROM `db_version` LIKE 'custom_version'");
+	if (!results.Success() || results.RowCount() == 0) {
+		LogInfo("Adding custom_version column to db_version table");
+		m_database->QueryDatabase("ALTER TABLE `db_version` ADD COLUMN `custom_version` INT(11) UNSIGNED NOT NULL DEFAULT 0");
+	}
+}

common/database/database_update.h

@@ -17,6 +17,7 @@ struct ManifestEntry {
 struct DatabaseVersion {
 	int server_database_version;
 	int bots_database_version;
+	int custom_database_version;
 };
 
 class DatabaseUpdate {
@@ -38,6 +39,7 @@ private:
 	Database *m_content_database;
 	static bool CheckVersionsUpToDate(DatabaseVersion v, DatabaseVersion b);
 	void InjectBotsVersionColumn();
+	void InjectCustomVersionColumn();
 
 };
 

common/database/database_update_manifest_custom.cpp

@@ -0,0 +1,55 @@
+#include "database_update.h"
+
+std::vector<ManifestEntry> manifest_entries_custom = {
+	ManifestEntry{
+		.version = 1,
+		.description = "2025_05_16_new_database_check_test",
+		.check = "SHOW TABLES LIKE 'new_table'",
+		.condition = "empty",
+		.match = "",
+		.sql = R"(
+CREATE TABLE `new_table`  (
+  `id` int NOT NULL AUTO_INCREMENT,
+  PRIMARY KEY (`id`)
+);
+)",
+		.content_schema_update = false,
+	},
+// Used for testing
+//	ManifestEntry{
+//		.version = 9229,
+//		.description = "new_database_check_test",
+//		.check = "SHOW TABLES LIKE 'new_table'",
+//		.condition = "empty",
+//		.match = "",
+//		.sql = R"(
+//CREATE TABLE `new_table`  (
+//  `id` int NOT NULL AUTO_INCREMENT,
+//  PRIMARY KEY (`id`)
+//);
+//CREATE TABLE `new_table1`  (
+//  `id` int NOT NULL AUTO_INCREMENT,
+//  PRIMARY KEY (`id`)
+//);
+//CREATE TABLE `new_table2`  (
+//  `id` int NOT NULL AUTO_INCREMENT,
+//  PRIMARY KEY (`id`)
+//);
+//CREATE TABLE `new_table3`  (
+//  `id` int NOT NULL AUTO_INCREMENT,
+//  PRIMARY KEY (`id`)
+//);
+//)",
+//	}
+
+};
+
+// see struct definitions for what each field does
+// struct ManifestEntry {
+// 	int         version{};     // database version of the migration
+// 	std::string description{}; // description of the migration ex: "add_new_table" or "add_index_to_table"
+// 	std::string check{};       // query that checks against the condition
+// 	std::string condition{};   // condition or "match_type" - Possible values [contains|match|missing|empty|not_empty]
+// 	std::string match{};       // match field that is not always used, but works in conjunction with "condition" values [missing|match|contains]
+// 	std::string sql{};         // the SQL DDL that gets ran when the condition is true
+// };

common/dbcore.cpp

@@ -189,9 +189,9 @@ void DBcore::TransactionBegin()
 	QueryDatabase("START TRANSACTION");
 }
 
-void DBcore::TransactionCommit()
+MySQLRequestResult DBcore::TransactionCommit()
 {
-	QueryDatabase("COMMIT");
+	return QueryDatabase("COMMIT");
 }
 
 void DBcore::TransactionRollback()

common/dbcore.h

@@ -32,7 +32,7 @@ public:
 	MySQLRequestResult QueryDatabase(const std::string& query, bool retryOnFailureOnce = true);
 	MySQLRequestResult QueryDatabaseMulti(const std::string &query);
 	void TransactionBegin();
-	void TransactionCommit();
+	MySQLRequestResult TransactionCommit();
 	void TransactionRollback();
 	std::string Escape(const std::string& s);
 	uint32 DoEscapeString(char *tobuf, const char *frombuf, uint32 fromlen);

common/eqemu_logsys.cpp

@@ -682,14 +682,33 @@ EQEmuLogSys *EQEmuLogSys::LoadLogDatabaseSettings(bool silent_load)
 		if (is_missing_in_database && !is_deprecated_category) {
 			LogInfo("Automatically adding new log category [{}] ({})", Logs::LogCategoryName[i], i);
 
-			auto new_category = LogsysCategoriesRepository::NewEntity();
+			auto e = LogsysCategoriesRepository::NewEntity();
-			new_category.log_category_id          = i;
+			e.log_category_id          = i;
-			new_category.log_category_description = Strings::Escape(Logs::LogCategoryName[i]);
+			e.log_category_description = Strings::Escape(Logs::LogCategoryName[i]);
-			new_category.log_to_console           = log_settings[i].log_to_console;
+			e.log_to_console           = log_settings[i].log_to_console;
-			new_category.log_to_gmsay             = log_settings[i].log_to_gmsay;
+			e.log_to_gmsay             = log_settings[i].log_to_gmsay;
-			new_category.log_to_file              = log_settings[i].log_to_file;
+			e.log_to_file              = log_settings[i].log_to_file;
-			new_category.log_to_discord           = log_settings[i].log_to_discord;
+			e.log_to_discord           = log_settings[i].log_to_discord;
-			db_categories_to_add.emplace_back(new_category);
+			db_categories_to_add.emplace_back(e);
+		}
+
+		// look to see if the category name is different in the database
+		auto it = std::find_if(
+			categories.begin(),
+			categories.end(),
+			[i](const auto &c) { return c.log_category_id == i; }
+		);
+		if (it != categories.end()) {
+			if (it->log_category_description != Logs::LogCategoryName[i]) {
+				LogInfo(
+					"Updating log category [{}] ({}) to new name [{}]",
+					it->log_category_description,
+					i,
+					Logs::LogCategoryName[i]
+				);
+				it->log_category_description = Logs::LogCategoryName[i];
+				LogsysCategoriesRepository::ReplaceOne(*m_database, *it);
+			}
 		}
 	}
 

common/eqemu_logsys.h

@@ -194,8 +194,8 @@ namespace Logs {
 		"Web Interface (Deprecated)",
 		"World Server (Deprecated)",
 		"Zone Server (Deprecated)",
-		"QueryErr",
+		"MySQL Error",
-		"Query",
+		"MySQL Query",
 		"Mercenaries",
 		"Quest Debug",
 		"Legacy Packet Logging (Deprecated)",
@@ -211,15 +211,15 @@ namespace Logs {
 		"Traps",
 		"NPC Roam Box",
 		"NPC Scaling",
-		"MobAppearance",
+		"Mob Appearance",
 		"Info",
 		"Warning",
 		"Critical (Deprecated)",
 		"Emergency (Deprecated)",
 		"Alert (Deprecated)",
 		"Notice (Deprecated)",
-		"AI Scan",
+		"AI Scan Close",
-		"AI Yell",
+		"AI Yell For Help",
 		"AI CastBeneficial",
 		"AOE Cast",
 		"Entity Management",
@@ -237,7 +237,7 @@ namespace Logs {
 		"DialogueWindow",
 		"HTTP",
 		"Saylink",
-		"ChecksumVer",
+		"Checksum Verification",
 		"CombatRecord",
 		"Hate",
 		"Discord",

common/net/daybreak_connection.cpp

@@ -1127,16 +1127,37 @@ void EQ::Net::DaybreakConnection::ProcessResend(int stream)
 		auto time_since_first_sent = std::chrono::duration_cast<std::chrono::milliseconds>(now - first_packet.first_sent).count();
 
 		if (time_since_first_sent >= m_owner->m_options.resend_timeout) {
+			auto now_ms = std::chrono::duration_cast<std::chrono::milliseconds>(now.time_since_epoch()).count();
+			auto first_sent_ms = std::chrono::duration_cast<std::chrono::milliseconds>(first_packet.first_sent.time_since_epoch()).count();
+			LogNetClient(
+				"Closing connection for m_endpoint [{}] m_port [{}] time_since_first_sent [{}] >= m_owner->m_options.resend_timeout [{}] now [{}] first_packet.first_sent [{}]",
+				m_endpoint,
+				m_port,
+				time_since_first_sent,
+				m_owner->m_options.resend_timeout,
+				now_ms,
+				first_sent_ms
+			);
 			Close();
 			return;
 		}
 
+		if (m_last_ack - now > std::chrono::milliseconds(1000)) {
+			LogNetClient(
+				"Resetting m_acked_since_last_resend flag for m_endpoint [{}] m_port [{}]",
+				m_endpoint,
+				m_port
+			);
+			m_acked_since_last_resend = true;
+		}
+
 		// make sure that the first_packet in the list first_sent time is within the resend_delay and now
 		// if it is not, then we need to resend all packets in the list
 		if (time_since_first_sent <= first_packet.resend_delay && !m_acked_since_last_resend) {
-			LogNetClient(
+			LogNetClientDetail(
-				"Not resending packets for stream [{}] packets [{}] time_first_sent [{}] resend_delay [{}] m_acked_since_last_resend [{}]",
+				"Not resending packets for m_endpoint [{}] m_port [{}] packets [{}] time_first_sent [{}] resend_delay [{}] m_acked_since_last_resend [{}]",
-				stream,
+				m_endpoint,
+				m_port,
 				s->sent_packets.size(),
 				time_since_first_sent,
 				first_packet.resend_delay,
@@ -1146,15 +1167,16 @@ void EQ::Net::DaybreakConnection::ProcessResend(int stream)
 		}
 	}
 
-	if (LogSys.IsLogEnabled(Logs::Detail, Logs::Netcode)) {
+	if (LogSys.IsLogEnabled(Logs::General, Logs::NetClient)) {
 		size_t    total_size = 0;
 		for (auto &e: s->sent_packets) {
 			total_size += e.second.packet.Length();
 		}
 
-		LogNetClient(
+		LogNetClientDetail(
-			"Resending packets for stream [{}] packet count [{}] total packet size [{}] m_acked_since_last_resend [{}]",
+			"Resending packets for m_endpoint [{}] m_port [{}] packet count [{}] total packet size [{}] m_acked_since_last_resend [{}]",
-			stream,
+			m_endpoint,
+			m_port,
 			s->sent_packets.size(),
 			total_size,
 			m_acked_since_last_resend
@@ -1165,9 +1187,12 @@ void EQ::Net::DaybreakConnection::ProcessResend(int stream)
 		if (m_resend_packets_sent >= MAX_CLIENT_RECV_PACKETS_PER_WINDOW ||
 			m_resend_bytes_sent >= MAX_CLIENT_RECV_BYTES_PER_WINDOW) {
 			LogNetClient(
-				"Stopping resend because we hit thresholds m_resend_packets_sent [{}] max [{}] m_resend_bytes_sent [{}] max [{}]",
+				"Stopping resend because we hit thresholds for m_endpoint [{}] m_port [{}]  m_resend_packets_sent [{}] max [{}] in_queue [{}] m_resend_bytes_sent [{}] max [{}]",
+				m_endpoint,
+				m_port,
 				m_resend_packets_sent,
 				MAX_CLIENT_RECV_PACKETS_PER_WINDOW,
+				s->sent_packets.size(),
 				m_resend_bytes_sent,
 				MAX_CLIENT_RECV_BYTES_PER_WINDOW
 			);
@@ -1204,11 +1229,11 @@ void EQ::Net::DaybreakConnection::ProcessResend(int stream)
 	}
 
 	m_acked_since_last_resend = false;
+	m_last_ack = now;
 }
 
 void EQ::Net::DaybreakConnection::Ack(int stream, uint16_t seq)
 {
-
 	auto now = Clock::now();
 	auto s = &m_streams[stream];
 	auto iter = s->sent_packets.begin();
@@ -1224,12 +1249,14 @@ void EQ::Net::DaybreakConnection::Ack(int stream, uint16_t seq)
 			m_rolling_ping = (m_rolling_ping * 2 + round_time) / 3;
 
 			iter = s->sent_packets.erase(iter);
-			m_acked_since_last_resend = true;
 		}
 		else {
 			++iter;
 		}
 	}
+
+	m_acked_since_last_resend = true;
+	m_last_ack = now;
 }
 
 void EQ::Net::DaybreakConnection::OutOfOrderAck(int stream, uint16_t seq)
@@ -1247,6 +1274,9 @@ void EQ::Net::DaybreakConnection::OutOfOrderAck(int stream, uint16_t seq)
 
 		s->sent_packets.erase(iter);
 	}
+
+	m_acked_since_last_resend = true;
+	m_last_ack = now;
 }
 
 void EQ::Net::DaybreakConnection::UpdateDataBudget(double budget_add)

common/net/daybreak_connection.h

@@ -186,6 +186,7 @@ namespace EQ
 			size_t m_resend_packets_sent = 0;
 			size_t m_resend_bytes_sent = 0;
 			bool m_acked_since_last_resend = false;
+			Timestamp m_last_ack;
 
 			struct DaybreakSentPacket
 			{

common/rulesys.cpp

@@ -502,6 +502,19 @@ bool RuleManager::UpdateInjectedRules(Database *db, const std::string &rule_set_
 		}
 	}
 
+	// update rules in the database where the description is different
+	for (auto &e : RuleValuesRepository::All(*db)) {
+		auto i = rule_data.find(e.rule_name);
+		if (i != rule_data.end()) {
+			// if notes are different, update them
+			if (i->second.second != nullptr && *i->second.second != e.notes) {
+				LogInfo("Updating rule [{}] notes to [{}]", i->first, *i->second.second);
+				e.notes = *i->second.second;
+				RuleValuesRepository::ReplaceOne(*db, e);
+			}
+		}
+	}
+
 	if (injected_rule_entries.size()) {
 		if (!RuleValuesRepository::InjectRules(*db, injected_rule_entries)) {
 			return false;

common/ruletypes.h

@@ -1073,6 +1073,7 @@ RULE_CATEGORY(Logging)
 RULE_BOOL(Logging, PrintFileFunctionAndLine, false, "Ex: [World Server] [net.cpp::main:309] Loading variables...")
 RULE_BOOL(Logging, WorldGMSayLogging, true, "Relay worldserver logging to zone processes via GM say output")
 RULE_BOOL(Logging, PlayerEventsQSProcess, false, "Have query server process player events instead of world. Useful when wanting to use a dedicated server and database for processing player events on separate disk")
+RULE_STRING(Logging, PlayerEventsIgnoreGMCommands, "help,show", "This is a comma delimited list of commands to ignore when recording GM command player events.")
 RULE_INT(Logging, BatchPlayerEventProcessIntervalSeconds, 5, "This is the interval in which player events are processed in world or qs")
 RULE_INT(Logging, BatchPlayerEventProcessChunkSize, 10000, "This is the cap of events that can be inserted into the queue before a force flush. This is to keep from hitting MySQL max_allowed_packet and killing the connection")
 RULE_CATEGORY_END()

common/version.h

@@ -25,7 +25,7 @@
 
 // Build variables
 // these get injected during the build pipeline
-#define CURRENT_VERSION "23.6.0-dev" // always append -dev to the current version for custom-builds
+#define CURRENT_VERSION "23.7.0-dev" // always append -dev to the current version for custom-builds
 #define LOGIN_VERSION "0.8.0"
 #define COMPILE_DATE    __DATE__
 #define COMPILE_TIME    __TIME__
@@ -44,6 +44,7 @@
 
 #define CURRENT_BINARY_DATABASE_VERSION 9323
 #define CURRENT_BINARY_BOTS_DATABASE_VERSION 9054
+#define CUSTOM_BINARY_DATABASE_VERSION 0
 
 #endif
 

package.json

@@ -1,6 +1,6 @@
 {
   "name": "eqemu-server",
-  "version": "23.6.0",
+  "version": "23.7.0",
   "repository": {
     "type": "git",
     "url": "https://github.com/EQEmu/Server.git"

world/cli/database_version.cpp

@@ -14,6 +14,7 @@ void WorldserverCLI::DatabaseVersion(int argc, char **argv, argh::parser &cmd, s
 
 	v["database_version"]        = CURRENT_BINARY_DATABASE_VERSION;
 	v["bots_database_version"]   = RuleB(Bots, Enabled) ? CURRENT_BINARY_BOTS_DATABASE_VERSION : 0;
+	v["custom_database_version"] = CUSTOM_BINARY_DATABASE_VERSION;
 
 	std::stringstream payload;
 	payload << v;

world/cli/version.cpp

@@ -14,6 +14,7 @@ void WorldserverCLI::Version(int argc, char **argv, argh::parser &cmd, std::stri
 	j["bots_database_version"]   = CURRENT_BINARY_BOTS_DATABASE_VERSION;
 	j["compile_date"]            = COMPILE_DATE;
 	j["compile_time"]            = COMPILE_TIME;
+	j["custom_database_version"] = CUSTOM_BINARY_DATABASE_VERSION;
 	j["database_version"]        = CURRENT_BINARY_DATABASE_VERSION;
 	j["server_version"]          = CURRENT_VERSION;
 

world/eqemu_api_world_data_service.cpp

@@ -162,7 +162,8 @@ void EQEmuApiWorldDataService::reload(Json::Value &r, const std::vector<std::str
 	for (auto &t: ServerReload::GetTypes()) {
 		if (std::to_string(t) == command || Strings::ToLower(ServerReload::GetName(t)) == command) {
 			message(r, fmt::format("Reloading [{}] globally", ServerReload::GetName(t)));
-			zoneserver_list.SendServerReload(t, nullptr);
+			LogInfo("Queueing reload of type [{}] to zones", ServerReload::GetName(t));
+			zoneserver_list.QueueServerReload(t);
 		}
 		found_command = true;
 	}

world/zonelist.cpp

@@ -132,6 +132,16 @@ void ZSList::Process() {
 			).c_str()
 		);
 	}
+
+	if (!m_queued_reloads.empty()) {
+		m_queued_reloads_mutex.lock();
+		for (auto &type : m_queued_reloads) {
+			LogInfo("Sending reload of type [{}] to zones", ServerReload::GetName(type));
+			SendServerReload(type, nullptr);
+		}
+		m_queued_reloads.clear();
+		m_queued_reloads_mutex.unlock();
+	}
 }
 
 bool ZSList::SendPacket(ServerPacket* pack) {
@@ -1003,3 +1013,10 @@ void ZSList::SendServerReload(ServerReload::Type type, uchar *packet)
 		++counter;
 	}
 }
+
+void ZSList::QueueServerReload(ServerReload::Type &type)
+{
+	m_queued_reloads_mutex.lock();
+	m_queued_reloads.emplace_back(type);
+	m_queued_reloads_mutex.unlock();
+}

world/zonelist.h

@@ -9,6 +9,7 @@
 #include <vector>
 #include <memory>
 #include <deque>
+#include <mutex>
 
 class WorldTCPConnection;
 class ServerPacket;
@@ -74,7 +75,10 @@ public:
 	const std::list<std::unique_ptr<ZoneServer>> &getZoneServerList() const;
 	inline uint32_t GetServerListCount() { return zone_server_list.size(); }
 	void SendServerReload(ServerReload::Type type, uchar *packet = nullptr);
+	std::mutex m_queued_reloads_mutex;
+	std::vector<ServerReload::Type> m_queued_reloads = {};
 
+	void QueueServerReload(ServerReload::Type &type);
 private:
 	void OnTick(EQ::Timer *t);
 	uint32 NextID;

zone/client.cpp

@@ -1024,8 +1024,6 @@ bool Client::Save(uint8 iCommitNow) {
 		m_pp.endurance = current_endurance;
 	}
 
-	database.TransactionBegin();
-
 	/* Save Character Currency */
 	database.SaveCharacterCurrency(CharacterID(), &m_pp);
 
@@ -1109,8 +1107,6 @@ bool Client::Save(uint8 iCommitNow) {
 		database.botdb.SaveBotSettings(this);
 	}
 
-	database.TransactionCommit();
-
 	LogInfo("Save for [{}] took [{}]", GetCleanName(), timer.elapsed());
 
 	return true;

zone/command.cpp

@@ -514,7 +514,15 @@ int command_realdispatch(Client *c, std::string message, bool ignore_status)
 		parse->EventPlayer(EVENT_GM_COMMAND, c, message, 0);
 	}
 
-	if (player_event_logs.IsEventEnabled(PlayerEvent::GM_COMMAND) && message != "#help") {
+	bool log_command = true;
+	for (auto &cmd: Strings::Split(RuleS(Logging, PlayerEventsIgnoreGMCommands), ",")) {
+		if (Strings::Contains(command, Strings::ToLower(cmd))) {
+			log_command = false;
+			break;
+		}
+	}
+
+	if (player_event_logs.IsEventEnabled(PlayerEvent::GM_COMMAND) && log_command) {
 		auto e = PlayerEvent::GMCommandEvent{
 			.message = message,
 			.target = c->GetTarget() ? c->GetTarget()->GetName() : "NONE"

zone/zone_save_state.cpp

@@ -6,6 +6,7 @@
 #include "zone.h"
 #include "zone_save_state.h"
 #include "../common/repositories/spawn2_repository.h"
+#include "../common/repositories/criteria/content_filter_criteria.h"
 
 // IsZoneStateValid checks if the zone state is valid
 // if these fields are all empty or zero value for an entire zone state, it's considered invalid
@@ -531,6 +532,69 @@ bool Zone::LoadZoneState(
 		}
 	}
 
+	// compare state spawns to spawn2 list, if there are any missing, we need to add them
+	// this is to cover the rare case where a spawn2 is created in the database but not in the zone state
+	auto zone_spawns = Spawn2Repository::GetWhere(
+		content_db, fmt::format(
+			"TRUE {} AND zone = '{}' AND (version = {} OR version = -1) ",
+			ContentFilterCriteria::apply(),
+			zone->GetShortName(),
+			zone->GetInstanceVersion()
+		)
+	);
+
+	for (auto &s: zone_spawns) {
+		bool found = false;
+		for (auto &ss: spawn_states) {
+			if (ss.spawn2_id == 0 || ss.spawngroup_id == 0 || ss.is_corpse || ss.is_zone) {
+				continue;
+			}
+			if (s.id == ss.spawn2_id) {
+				found = true;
+				break;
+			}
+		}
+
+		if (!found) {
+			bool spawn_enabled = true;
+
+			for (auto &ds: disabled_spawns) {
+				if (ds.spawn2_id == s.id) {
+					spawn_enabled = !ds.disabled;
+				}
+			}
+
+			LogZoneState("Missing spawn2 [{}] in zone state, this NPC spawn was newly created", s.id);
+			uint32 spawn_time_left = 0;
+			if (spawn_times.count(s.id) != 0) {
+				spawn_time_left = spawn_times[s.id];
+				LogInfo("Spawn2 [{}] Respawn time left [{}]", s.id, spawn_time_left);
+			}
+
+			auto new_spawn = new Spawn2(
+				s.id,
+				s.spawngroupID,
+				s.x,
+				s.y,
+				s.z,
+				s.heading,
+				s.respawntime,
+				s.variance,
+				spawn_time_left,
+				s.pathgrid,
+				(bool) s.path_when_zone_idle,
+				s._condition,
+				(int16) s.cond_value,
+				spawn_enabled,
+				(EmuAppearance) s.animation
+			);
+
+			new_spawn->SetStoredLocation(glm::vec4(s.x, s.y, s.z, s.heading));
+			spawn2_list.Insert(new_spawn);
+			new_spawn->Process();
+		}
+	}
+
 	// dynamic spawns, quest spawns, triggers etc.
 	for (auto &s: spawn_states) {
 		if (s.spawngroup_id > 0 || s.is_zone) {