Dev Diary - Game Stats

iwCSA26.png

Developer Diary - Game Stats

In the next release we will be adding the capacity for game servers to send back statistics for analysis by the development team. This is one of the most important features that we have ever added to MBII. Up until now all design decisions and feedback we have had has come from anecdotal evidence and different people often disagree wildly. Gathering statistics will allow these discussions to be data informed rather than relying on different people's often irreconcilable points of view.

To that end, with the next release of MBII libcurl will be a required component for server hosts. For windows users this library will be supplied, pre-packaged in the usual way either via launcher or the manual installations. For Linux server hosts the 32bit libcurl library will need be installed or the game will not load.

What data is being gathered?

Whilst this will be an ever evolving component of MBII we are starting off by gathering the readily accessible but most meaningful data. This is the data which was quick to implement but will give us the biggest capability when it comes assisting decision making in the short term. As time goes on extra pieces of information will be gathered, which could be more of the generic data as the first release is, or highly targeted data to assist with answering specific questions.

The first release will gather the following information:

For each Game:
  • Server Name/IP Address
  • Map
  • Game Mode
  • Game Duration
  • Winning Team
  • Are Teams Swapped?
  • MBII Version
  • GameStats Version
  • Game Date and Time
  • Progressive Siege Area
  • List of Kills (The specific data for this is specified in the "For each Kill section")
  • List of Players (The specific data for this is specified in the "For each Player section")
For each Kill:
  • Client Slot Killed
  • Client Slot Killed By
  • Client Slot Assisted By
  • Time between Assist Damage Dealt and Death
  • Location of dead Player
  • Location of killer Player
  • Location of Assisting Player
  • Means of Death
  • Means of Assist
For each Player:
  • Client Slot
  • Player Abilities
  • Player Class

What data is not being gathered?

Whilst it is impossible to list every piece of information that is not being gathered, hopefully the following list will show the types of data that are not being gathered and will never be added to the game stats we do gather. These are:
  • Any information from the client. All data is gathered from the servers. Only dedicated servers will send data (Where g_dedicated is not 0) - local games will not send data back.
  • Any personal information that the server sees - including but not limited to: Player Names, Player IP addresses, Player Scores, any information that could be used to identify a given player.

Analyzing the Data

For the first release we will be looking to analyse the data in two ways. Firstly for game play design and secondly to provide information back to level designers.

Level Designers

From the information gathered level designers will be able to access heat maps for each map built into MBII. A separate heat map will be available for kills, deaths and assists and level designers will be able to access these either at the level for individual games, or for all games played on that individual map. The sort of conclusions that might be drawn from this are:
  • Are players generally being killed before they get too close to the objective?
  • Are players being killed in an unanticipated place?
  • Are there places which are not intended to be reachable which can be exploited for kills?
Here is a heat map for a few sessions played by the beta team on Lunar Base:
HeatMap.png


Gameplay

Gameplay statistics are useful in a huge variety of ways, and how we can analyse the data effectively will be an on going learning activity for us. As a starting point we have reports written which will answer a few basic questions.
  • What is the relative popularity of the classes?
  • Which abilities are most commonly chosen for each class? At what level are they taken?
  • Which builds are very commonly chosen?
  • Which weapons are getting the most kills?
  • Which weapons are getting the most assists?
Importantly this give us a base point to analyse the changes we make and determine how effective they have been. For example, if Elite Trooper wasn't often selected and didn't often get kills when it was played, we might decide to look at if it was under powered. If it was, and we made changes to buff it, we would be able to look after the release and see if the amount of people choosing the class has increased and the class was being more effective in combat. A sustained improvement over time would indicate that what we had changed was successful, where as a short term peak followed by a return to the old values might show people who have tried the changes we've made but it not being enough to make the class competitive.

Here is what our statistics from a few beta sessions look like - try not to pay attention to the numbers too much as this is a highly skewed group who would have been testing specific other things working their way through development. As an example of how useful the statistics are, you could probably work out which classes they have been doing extra testing on...

ClassStats.png


FcfStats.png


MODStats.png


Server Owners
All this new functionality requires use of a third party library - in this case we are using libcurl to perform the network functionality. If you are a Windows server owner or a player self hosting on Windows then the required library will be provided by launcher in the flavor of libcurl.dll. For Linux server owners (And anyone self hosting on linux via OpenJK) you will need to install the i386 version of curl in order to start a game.

For Ubuntu/Debian systems:
Code:
sudo apt-get install curl:i386

For Older Ubuntu (At least 16.04 and Prior)/Debian
Code:
sudo apt-get install libcurl3:i386

For CentOS systems:
Code:
sudo yum install libcurl.i686

If this library is not available the following errors will appear in the console output:
Code:
Sys_LoadDll(<homepath>/GameData/MBIITest/jampgamei386.so) failed: "libcurl.so.4: cannot open shared object file: No such file or directory"
Sys_LoadDll(<executable>/MBII/jampgamei386.so) failed: "<executable>/MBII/jampgamei386.so: cannot open shared object file: Not a directory"
Sys_Error: Sys_LoadDll (jampgame) failed dlopen() completely!
 

KarusBadders

Donator
Posts
336
Reaction score
467
Not sure why anyone would/should be opposed to this...
I think this is great news. Anything to help the devs should be a welcome addition at this point.
Also at least now they'll be able to actually anaylse player trends rather than hearing us buttcrying on the forums about it.
 

MaceMadunusus

Level Designer
Donator
Movie Battles II Team Retired
Posts
1,651
Reaction score
2,214
I disagree with not providing statistics to players. Not data act friendly

Like defiant said, no private data is being gathered so I don't think it applies? Plus, we would have to bump donation requirements significantly to do that. A large chunk of games do this same thing, but do not have it accessible to the public, its nothing new, even with GDPR and other things being in existence.
 
Posts
58
Reaction score
76
I would like to point out that if your data starts showing high deaths due to grip its probably just me please dont nerf it again
 

Hexodious

Movie Battles II Team Retired
Posts
705
Reaction score
696
This is cool - I am looking forwards to seeing a future dev post where you break down some findings.

Are you able to separate findings based on server geoip, for example seeing meta differences/popularity of classes/abilities based on region? Similar to how we can filter EU/NA maps played in grafana.
 
Top