Experiments in Online Gaming Platforms: A Personal Reflection and Interference Analysis
Abstract: Experiments in Online Gaming Platforms: A Personal Reflection and Interference Analysis
In this talk, I will delve into my journey within the video game industry, discussing the role of analytics, the questions that matter to gaming platforms, and the challenges that emerge. I will recount my experiences in conducting field experiments on online gaming platforms and introduce a specific challenge in causal inference: the interdependence of user experiences. To analyze this issue theoretically, we construct a stochastic market model and develop its mean field limit to examine user dynamics on these platforms. We concentrate on two prevalent designs: user and match randomization strategies. Under Markovian behavior and homogeneous treatment effects, match randomization provides unbiased estimates, but biases arise in other cases. On the other hand, user randomization tends to be generally biased but is more robust against model inaccuracies. We propose a linear regression estimator under user randomization, which consistently outperforms alternatives. In conclusion, I will highlight other potential challenges in experimental designs on gaming platforms based on my experiences.
Talker Bio
Xiao Lei is an assistant professor at HKU Business School. He received his doctoral degree in Operations Research at Columbia University. His research interests include revenue management and pricing, digital economy, and human-centric operations. Xiao's work has been acknowledged with the George Dantzig Dissertation Award, INFORMS Service Science Best Student Paper Award, the Jeff McGill Best Student Paper Award, and CSAMSE Best Paper Award. He has research and consulting experience with several leading firms in the industry, such as Activision and Tencent.