54 Commits

Author SHA1 Message Date
Mitch Freeman
8f4f8368df
[Player Event Logs] Migrate and Deprecate QS Legacy Logging (#4542)
* First pass of player_event_loot_items

* Second pass of player_event_loot_items

* Third pass of player_event_loot_items

* Example without RecordDetailEvent template

* Cleanup the removal of the template

* Fourth Pass

Add retention for etl tables
Rename tables/fields to etl nomenclature
Combine database work to one atomic load

* Reposition to reduce db tasks

* Refactor etl processing for easier additions

* Add merchant purchase event
testing passed though appears that the event itself has a few bugs.  Will fix them in another commit

* Fix PlayerEventMerchantPurchase in client_packet.cpp

* WIP - Handin

* Handin Event added

* Cleanup

* All a rentention period of 0 days which deletes all current records.

* Updates

Cleanup and refactor a few items.

* Cleanup and Formatting

Cleanup and Formatting

* Add etl for
Playerevent::Trade
PlayerEvent::Speech (new event to mirror functionality of qs_speech

* Add etl for
Playerevent::KilledNPC, KilledNamedNPC and KilledRaidNPC

* Add etl for Playerevent::AA_purchase

Add etl for Playerevent::AA_purchase

* Cleanup before PR

* Review comment updates.

* Add world cli etl:settings to output a json on all player event details.

* Add reserve for all etl_queues
Correct a failed test case for improper next id for etl tables when table is first created.

* Potential solution for a dedicated database connection for player events.

* Simple thread for player_events.  Likely there is a better way to do this.

* Add zone to qs communications for recordplayerevents

First pass of enabling zone to qs direct transport to allow for PlayerEvents to bypass world.

* Cleanup a linux compile issue

* Add augments to LOOT ITEM and DESTROY ITEM

* Add augments to ITEMCREATION, FORAGESUCCESS, FISHSUCCESS, DESTROYITEM, LOOTITEM, DROPPEDITEM, TRADERPURCHASE, TRADERSELL, GUILDTRIBUTEDONATE and cleaned up the naming convention of augments

* Formatting fixes

* Swap out GetNextTableId

* Statically load counter

* Add counter.clear() since the counter is static

* Upload optional QS conversion scripts

* Remove all qs_tables and code referencing them

* Update database.cpp

* Simplify ProcessBatchQueue

* Simplify PorcessBatchQueue

* Simplify event truncation

* Build event truncation to bulk query by retention groups

* Post rebase

* Update player_events.h

* Fix build

* Update npc.cpp

* First pass of direct zone to qs sending for player events

* Remove keepalive logic

* Fix event ordering

* Cleanup

* Update player_event_logs.cpp

* Wipe event data after ETL processed

* Split up database connections, hot reload logs for QS

* Load rules from database vs qs_database

* Update player_event_logs.cpp

* Hot toggle queryserv connect

---------

Co-authored-by: Akkadius <akkadius1@gmail.com>
2025-02-05 02:02:16 -06:00
Fryguy
748e37dbdf
[Crash Fix] Reverting PR #3877 (#3997)
Reports of #3877 causing crashes on windows. No reports on Linux, but reverting for safety.
2024-01-17 18:22:11 -05:00
Fryguy
2ed4effbe3
[Bug Fix] Prevent QS Crashes (#3877) 2024-01-07 04:47:29 -05:00
Alex King
53169ae217
[Cleanup] Delete errmsg.h in common and zone (#3821)
* [Cleanup] Delete errmsg.h in common and zone

# Notes
- These files were mostly unused.
- Moved the stuff that was used to the `dbcore.h` file since it's already used.

* Remove references
2023-12-30 11:21:57 -05:00
Chris Miles
d9f545a5ec
[Logging] Implement Player Event Logging system (#2833)
* Plumbing

* Batch processing in world

* Cleanup

* Cleanup

* Update player_event_logs.cpp

* Add player zoning event

* Use generics

* Comments

* Add events

* Add more events

* AA_GAIN, AA_PURCHASE, FORAGE_SUCCESS, FORAGE_FAILURE

* FISH_SUCCESS, FISH_FAILURE, ITEM_DESTROY

* Add charges to ITEM_DESTROY

* WENT_ONLINE, WENT_OFFLINE

* LEVEL_GAIN, LEVEL_LOSS

* LOOT_ITEM

* MERCHANT_PURCHASE

* MERCHANT_SELL

* SKILL_UP

* Add events

* Add more events

* TASK_ACCEPT, TASK_COMPLETE, and TASK_UPDATE

* GROUNDSPAWN_PICKUP

* SAY

* REZ_ACCEPTED

* COMBINE_FAILURE and COMBINE_SUCCESS

* DROPPED_ITEM

* DEATH

* SPLIT_MONEY

* TRADER_PURCHASE and TRADER_SELL

* DISCOVER_ITEM

* Convert GM_COMMAND to use new macro

* Convert ZONING event to use macro

* Revert some code changes

* Revert "Revert some code changes"

This reverts commit d53682f997e89a053a660761085913245db91e9d.

* Add cereal generation support to repositories

* TRADE

* Formatting

* Cleanup

* Relocate discord_manager to discord folder

* Discord sending plumbing

* Rename UCS's Database class to UCSDatabase to be more specific and not collide with base Database class for repository usage

* More discord sending plumbing

* More discord message formatting work

* More discord formatting work

* Discord formatting of events

* Format WENT_ONLINE, WENT_OFFLINE

* Add merchant purchase event

* Handle Discord MERCHANT_SELL formatter

* Update player_event_discord_formatter.cpp

* Tweaks

* Implement retention truncation

* Put mutex locking on batch queue, put processor on its own thread

* Process on initial bootup

* Implement optional QS processing, implement keepalive from world to QS

* Reload player event settings when logs are reloaded in game

* Set settings defaults

* Update player_event_logs.cpp

* Update player_event_logs.cpp

* Set retention days on boot

* Update player_event_logs.cpp

* Player Handin Event Testing.

Testing player handin stuff.

* Cleanup.

* Finish NPC Handin.

* set a reference to the client inside of the trade object as well for plugins to process

* Fix for windows _inline

* Bump to cpp20 default, ignore excessive warnings on windows

* Bump FMT to 6.1.2 for cpp20 compat and swap fmt::join for Strings::Join

* Windows compile fixes

* Update CMakeLists.txt

* Update CMakeLists.txt

* Update CMakeLists.txt

* Create 2022_12_19_player_events_tables.sql

* [Formatters] Work on Discord Formatters

* Handin money.

* Format header

* [Formatters] Work on Discord Formatters

* Format

* Format

* [Formatters] More Formatter work, need to test further.

* [Formatters] More Work on Formatters.

* Add missing #endif

* [Formatters] Work on Formatters, fix Bot formatting in ^create help

* NPC Handin Discord Formatter

* Update player_event_logs.cpp

* Discover Item Discord Formatter

* Dropped Item Discord Formatter

* Split Money Discord Formatter

* Trader Discord Formatters

* Cleanup.

* Trade Event Discord Formatter Groundwork

* SAY don't record GM commands

* GM_Command don't record #help

* Update player_event_logs.cpp

* Fill in more event data

* Post rebase fixes

* Post rebase fix

* Discord formatting adjustments

* Add event deprecation or unimplemented tag support

* Trade events

* Add return money and sanity checks.

* Update schema

* Update ucs.cpp

* Update client.cpp

* Update 2022_12_19_player_events_tables.sql

* Implement archive single line

* Replace hackers table and functions with PossibleHack player event

* Replace very old eventlog table since the same events are covered by player event logs

* Update bot_command.cpp

* Record NPC kill events ALL / Named / Raid

* Add BatchEventProcessIntervalSeconds rule

* Naming

* Update CMakeLists.txt

* Update database_schema.h

* Remove logging function and methods

* DB version

* Cleanup SendPlayerHandinEvent

---------

Co-authored-by: Kinglykrab <kinglykrab@gmail.com>
Co-authored-by: Aeadoin <109764533+Aeadoin@users.noreply.github.com>
2023-02-12 21:31:01 -06:00
Chris Miles
9e3539295b
[QS] Database class name change (#2743) 2023-01-15 14:37:39 -06:00
Chris Miles
dfd8f84cac
[Strings] Refactor Strings Usage (#2305)
* Initial commit checkpoint

* More functions converted

* Commify

* More functions

* Fin

* Sort declarations

* Split functions between files

* Bots

* Update strings.h

* Split

* Revert find replaces

* Repository template

* Money

* Misc function

* Update CMakeLists.txt

* Saylink

* Update strings.cpp

* Swap Strings::Saylink for Saylink::Create since saylink is coupled to zone database

* API casings
2022-07-14 02:10:52 -05:00
cybernine186
4de5a7b86d
[Bug] Fixed trade items record log (#2003)
LogPlayerHandin was not recording the item details of the trade. It should be inserting the trade details into `qs_player_handin_record_entries` but it was not running the query because of missing QueryDatabase function.
2022-02-16 21:52:14 -06:00
Chris Miles
bcb0e43d13
[Logging] Simplify Log Settings Initialization (#1394)
* Simplify logging loading

* Fix log injections and reduce verbosity
2021-06-13 18:06:43 -05:00
Akkadius
acae5d81ab Logs::QSServer to LogInfo 2019-09-02 04:23:50 -05:00
Akkadius
859252a270 Move status log calls to use LogInfo 2019-09-01 23:22:40 -05:00
Akkadius
98cbb7d781 Simplify log calls 2019-09-01 21:04:58 -05:00
Akkadius
d3803045bc Add log aliases to make logging much easier to use (All FMT driven) 2019-09-01 20:47:26 -05:00
Akkadius
eaf31fb14b Merge branches 'lsid' and 'master' of https://github.com/EQEmu/Server into lsid 2019-08-10 03:26:40 -05:00
Michael Cook (mackal)
8df7bcc2c8 Fix double free in queryserv 2019-08-08 14:26:08 -04:00
Akkadius
3eb102a006 Fix issue with adding the real values of new logging categories 2019-07-05 19:43:10 -05:00
Trust
f8f1061ced [Queryserv] Dropped Item Logging 2018-07-22 12:39:34 -04:00
Akkadius
7aa1d243b0 [Performance] Reworked how all log calls are made in the source, see changelog.txt for more details 2017-04-01 03:51:46 -05:00
Michael Cook (mackal)
60da544d3a clang-tidy modernize-use-auto 2016-05-25 16:10:28 -04:00
Michael Cook (mackal)
700e801821 Fix mismatched delete in QueryServ 2016-04-12 13:44:33 -04:00
JJ
318a664b09 No "sigs". [skip ci] 2015-02-23 19:57:47 -05:00
Akkadius
387b445685 Fix the 'is_category_enabled' in each of the database loaders 2015-02-01 02:08:46 -06:00
Akkadius
46ac254b3a Add struct member LogSettings::is_category_enabled so that it can be used for fast checking in intense portions of code 2015-02-01 01:58:10 -06:00
Akkadius
46010fbfdf Rename LoadLogSysSettings to LoadLogSettings 2015-01-21 19:04:13 -06:00
Akkadius
a64c21eb96 Undo Queryserv refactoring because our database class stuff is stupid 2015-01-20 05:30:19 -06:00
Akkadius
1bbbb28218 Refactor some of the database stuff for QueryServ for uniformity, should probably be done right later 2015-01-20 04:30:27 -06:00
Akkadius
0d9b6703a6 Rename debug.h to global_define.h, update cmakelists and such 2015-01-19 04:12:09 -06:00
Akkadius
467b359d0c Moved all EQEmuLogSys:: enum references used in Log.Out to a namespace 'Logs' for shortening of syntax 2015-01-18 02:20:16 -06:00
Akkadius
c025765283 Renamed DoLog to Out as the aggregate logging function for simplicity of use and shortened syntax of Log.Out 2015-01-18 02:00:15 -06:00
Akkadius
1c048cb1d1 Renamed DebugCategory to DoLog as the aggregate logging function for simplicity of use and shortened syntax of Log.DoLog 2015-01-18 01:54:09 -06:00
Akkadius
e691735a2d Consolidate 'LogType' Error logs over to 'LogCategory' 2015-01-18 01:30:25 -06:00
Akkadius
e9f8d5fa6d Port Status messages from Type to Category 2015-01-18 01:27:52 -06:00
Akkadius
7dbde36b03 Rename reference logger to Log 2015-01-18 00:41:18 -06:00
Akkadius
132fbbb0c6 Rename LogDebugType to DebugCategory 2015-01-16 03:09:02 -06:00
Akkadius
f26f49c2a1 Convert 'QueryServ' debugging _log to logger.LogDebugType 2015-01-12 22:37:20 -06:00
Akkadius
17d9b9199c Replace rest of EQEmuLog->write with EQEmuLogSys Log 2015-01-10 19:36:15 -06:00
Akkadius
be381b4e56 Renamed EQEMuLog class to EQEmuLog 2015-01-08 19:46:28 -06:00
Arthur Ice
0438042844 GeneralQueryReceive converted to QueryDatabase 2014-10-07 09:02:13 -07:00
Arthur Ice
0a9732a267 LogMerchantTransaction converted to QueryDatabase 2014-08-28 00:10:55 -07:00
Arthur Ice
1bc06c9c24 LogPlayerMove converted to QueryDatabase 2014-08-28 00:05:28 -07:00
Arthur Ice
e0acc937b3 LogPlayerDelete converted to QueryDatabase 2014-08-28 00:04:20 -07:00
Arthur Ice
3a10a0129a LogPlayerNPCKill converted to QueryDatabase 2014-08-27 23:53:37 -07:00
Arthur Ice
18dbcf16cc LogPlayerHandin converted to QueryDatabase 2014-08-27 23:52:36 -07:00
Arthur Ice
bed8dc7d34 LogPlayerTrade converted to QueryDatabase 2014-08-27 23:52:35 -07:00
Arthur Ice
a0fc9844fd AddSpeech converted to QueryDatabase 2014-08-27 23:21:09 -07:00
Uleat
18a4f831be Tweaked QS code for Client::FinishTrade() and QueryServ handlers. 2014-08-26 06:37:40 -04:00
akkadius
7f89191ffc Changed zone process window title format, example: 'crushbone :: clients: 6 inst_id: 1 inst_ver: 0 :: port: 7015'
Most of the following changes are QueryServ related, fully implemented its original functionality to be able to offload
	intensive or metric based logging to a remote server process that could exist on another server entirely
Implemented Player Event Logging Types (Go to table `qs_player_events`):
		1 = Player_Log_Quest,
		2 = Player_Log_Zoning,
		3 = Player_Log_Deaths,
		4 = Player_Log_Connect_State,
		5 = Player_Log_Levels,
		6 = Player_Log_Keyring_Addition,
		7 = Player_Log_QGlobal_Update,
		8 = Player_Log_Task_Updates,
		9 = Player_Log_AA_Purchases,
		10 = Player_Log_Trade_Skill_Events,
		11 = Player_Log_Issued_Commands,
		12 = Player_Log_Money_Transactions,
		13 = Player_Log_Alternate_Currency_Transactions,
		- All QueryServ logging will be implemented with a front end in EoC 2.0 very soon
Changed all QS Error related logging to 'QUERYSERV__ERROR'
(Natedog) (Crash Fix) Legacy MySQL bug revert for loading AA's COALESCE( from COALESCE (
Implemented Perl Quest objects (LUA still needed to be exported):
	- quest::qs_send_query("MySQL query") - Will send a raw query to the QueryServ process, useful for custom logging
	- quest::qs_player_event(char_id, event_desc); - Will process a quest type event to table `qs_player_events`
Added MySQL Tables:
	- `qs_player_aa_rate_hourly`
	- `qs_player_events`
	- Source table structures from:
		- utils\sql\git\queryserv\required\08_23_2014_player_events_and_player_aa_rate_hourly
		To get the complete QueryServ schema, source from here:
		- utils\sql\git\queryserv\required\Complete_QueryServ_Table_Structures.sql
Added rules for each logging type, source rules here with them enabled by default:
	- utils\sql\git\queryserv\required\Complete_QueryServ_Rules_Enabled.sql
Spawn related logging cleanup
General code cleanup
Added queryserv.cpp and queryserv.h with QueryServ class
2014-08-23 23:59:20 -05:00
KimLS
7fc21b9e3a Tons of renames 2014-08-21 19:33:02 -07:00
KimLS
e6fba5ba82 Merging 2013-05-23 00:31:02 -07:00
Michael Cook
0fdfe025cb Remove 'using namespaces std' fixes #61 2013-05-22 16:17:19 -04:00