You've been logged out of GDC Vault since the maximum users allowed for this account has been reached. To access Members Only content on GDC Vault, please log out of GDC Vault from the computer which last accessed this account.

Click here to find out about GDC Vault Membership options for more users.


Session Name:

Don't trust the HiPPOs: A/B Testing Online Games


A/B Testing is very important in any modern online application. Designers learn to trust metrics (rather than the HiPPOs – the Highest Paid Person's Opinions) in assessing the efficacy of new features. Testing provides a rational basis for evaluation of the feature variations, and can result in significant improvement in the performance of an application. This presentation will introduce techniques used in experimental design in engineering, bio-medical, pharmaceutical contexts and provide an overview of how these may be adapted for use in optimizing online games, employing statistics which provides us with the tools to determine a confidence in the results we observe. We will examine the infrastructure required to implement testing efficiently within a game production pipeline for mobile and online games. We'll also explore the challenges you'll face when implementing A/B Testing, and hopefully speed you on your way to creating a successful testing culture within your organization.

Did you know free users get access to 30% of content from the last 2 years?

Get your team full access to the most up to date GDC content

  • GDC 2012
  • Steven Collins
  • Swrve
  • free content
  • Game Design
  • Design