mike ambinder, phdmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• platform for recording...
TRANSCRIPT
![Page 1: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/1.jpg)
MIKE AMBINDER, PhD VALVE
DATA TO DRIVE DECISION-MAKING
![Page 2: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/2.jpg)
HOW AND WHY VALVE USES DATA TO DRIVE
THE CHOICES WE MAKE
![Page 3: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/3.jpg)
Data to Drive Decision-Making
• Decision-Making at Valve
• Introduction to experimental design
• Data collection/analysis infrastructure
• Examples
—Playtesting (L4D)
—DOTA 2
—CS:GO
![Page 4: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/4.jpg)
![Page 5: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/5.jpg)
DECISION-MAKING AT VALVE
http://www.thumotic.com/seven-ways-the-red-pill-will-improve-your-life/
![Page 6: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/6.jpg)
Decision-Making at Valve
• No formal management structure
• Decision-making is a meritocracy
• All data is available to every employee
• We just want to make the best decisions possible.
• We don’t want to rely on ‘instinct’ it is fallible
![Page 7: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/7.jpg)
![Page 8: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/8.jpg)
Decision-Making
• Explicit
• Data-driven
• Theory-driven
• Measurable Outcomes
• Iterative
http://sarahmjamieson.wordpress.com/2012/06/10/the-solo-runner-quantum-meditation-5/
![Page 9: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/9.jpg)
Explicit
• What problem are you trying to solve?
• Define terminology/constructs/problem space
• Ask the ‘second’ question
• Force yourself to be specific
• Force yourself to be precise
![Page 10: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/10.jpg)
Data-Driven
• What do we know about the
problem?
• What do we need to know
before we decide?
• What do we still not know after
we decide?
![Page 11: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/11.jpg)
Theory-Driven
• What does the data mean?
—Is it consistent with expectations?
—Is it reliable?
• Model derived from prior experience/analysis
• Coherent narrative
• Prove a hypothesis right (or wrong)
• Want result AND explanation
![Page 12: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/12.jpg)
Measureable Outcomes
• Define ‘Success’
• How will we know we made the right choice?
• Know the ‘outcome’ of your decision
![Page 13: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/13.jpg)
Iterative
Gather
Data
Formulate
Hypothesis
Analyze
Data
![Page 14: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/14.jpg)
Iterative
Run
Experiment
in TF2
Run
Experiment
in CS:GO
Run
Experiment
in DOTA 2
Steam
![Page 15: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/15.jpg)
INTRODUCTION TO EXPERIMENTAL DESIGN
http://www.sas.com/en_us/insights/analytics.html
If it can be destroyed by the truth, it
deserves to be destroyed by the truth.
– Carl Sagan
![Page 16: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/16.jpg)
THE SCIENTIFIC METHOD
http://www.tomatosphere.org/teacher-resources/teachers-guide/principal-investigation/scientific-method.cfm
![Page 17: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/17.jpg)
Experimental Design
• Observational
—Retrospective vs. Prospective
—Correlational not causal
• Experiment
—Control Condition and Experimental Condition
—Account for confounding variables
—Measure variable of interest
![Page 18: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/18.jpg)
![Page 19: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/19.jpg)
Experimental Design
• What have we learned?
• What biases are present?
• How are future experiments informed?
• What other hypotheses need to be ruled out?
• What should we do next?
![Page 20: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/20.jpg)
DATA COLLECTION/ANALYSIS INFRASTRUCTURE
http://dorkutopia.com/wp-content/uploads/2013/06/Servers-Server-Farm-Engine-Room.jpg
![Page 21: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/21.jpg)
Valve Data Collection
• Record lots and lots (and lots) of user behavior
• If we’re not recording it, we’ll start recording it
• Define questions first, then schema
• Collection Analysis Communication
![Page 22: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/22.jpg)
Data Collection - Games
• OGS – Operational Game Stats
• Platform for recording gameplay metrics
• Kills, Deaths, Hero Selection, In-Game Purchases,
Matchmaking wait times, Bullet trajectories, Friends in
Party, Low-Priority Penalties, etc.
![Page 23: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/23.jpg)
Data Collection - Games
• Organizational schemas defined for each game
• Data sent at relevant intervals
• Daily, Monthly, Lifetime Rollups, Views,
Aggregations
![Page 24: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/24.jpg)
ValveStats
![Page 25: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/25.jpg)
Data Collection - Steam
• Steam Database – Raw data
• SteamStats Database – Analysis/Summary of Raw Data
• Record all relevant data about Steam user behavior
![Page 26: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/26.jpg)
![Page 27: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/27.jpg)
PLAYTESTING
![Page 28: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/28.jpg)
Valve’s Game Design Process
Goal is a game that makes customers happy
Game designs are hypotheses
Playtests are experiments
Evaluate designs based off playtest results
Repeat
![Page 29: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/29.jpg)
Content
Creation
+
Game Design
Playtesting
Hypothesis
Feedback
![Page 30: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/30.jpg)
Playtest Methodologies
• Traditional
—Direct Observation
—Verbal Reports
—Q&As
![Page 31: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/31.jpg)
Playtest Methodologies
• Technical
—Stat Collection/Data Analysis
—Design Experiments
—Surveys
—Physiological Measurements
(Heart Rate, Eyetracking, etc.)
![Page 32: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/32.jpg)
LEFT 4 DEAD
![Page 33: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/33.jpg)
Enabling Cooperation
• Coop Game where competing gets you killed
• Initial playtests were not as enjoyable as hoped
• Initial playtests were not as cooperative as hoped
—Players letting their teammates die
—Ignoring cries for help
![Page 34: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/34.jpg)
Enabling Cooperation
• Explicit: Players letting teammates die
• Data-Driven: Surveys, Q&As, high death rates
• Theory-Driven: Lack awareness of teammate location
• Measurements: Surveys, Q&As, death rates
• Iterative:
Hypothesis: Give better visual cues to teammate location
![Page 35: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/35.jpg)
![Page 36: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/36.jpg)
![Page 37: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/37.jpg)
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Pre Post
Deaths in 'No Mercy - The Apartments'
~40%
Decrease
![Page 38: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/38.jpg)
Results
• Survey ratings of enjoyment/cooperation increased
• Anecdotal responses decreased
• Deaths decreased
![Page 39: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/39.jpg)
Enabling Cooperation
• Explicit: Players letting teammates die
• Data-Driven: Surveys, Q&As, high death rates
• Theory-Driven: Lack awareness of teammate location
• Measurements: Surveys, Q&As, death rates
• Iterative: Where else can visual cues aid gameplay?
![Page 40: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/40.jpg)
DOTA 2
![Page 41: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/41.jpg)
![Page 42: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/42.jpg)
Improve Player Communication
• Explicit: Reduce negative communication
• Data-Driven: Chat, reports, forums, emails, quitting
• Theory-Driven: No feedback loop to punish negativity
• Measurements: Chat, reports, ban rates, recidivism
• Iterative: Will this work in TF2? Do these systems scale?
Hypothesis: Automating communication bans will reduce
negativity in-game
![Page 43: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/43.jpg)
![Page 44: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/44.jpg)
Results
• 35% fewer negative words used in chat
• 32% fewer communication reports
• 1% of active player base is currently banned
• 61% of banned players only receive one ban
![Page 45: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/45.jpg)
CS:GO
![Page 46: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/46.jpg)
![Page 47: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/47.jpg)
Weapon Balance
• Explicit: M4A4 usage is high; few choices in late-game
• Data-driven: Purchase rates
• Theory-driven: Greater tactical choice Player retention
• Measurements: Purchase rates, playtime, efficacy
• Iterative: Inform future design choices
Hypothesis: Creating a balanced alternative weapon will
increase player choice and playtime
![Page 48: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/48.jpg)
![Page 49: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/49.jpg)
Results
• ~ 50/50 split between new and old favorites
• Increase in playtime
—Conflated with other updates
—Difficult to isolate
• Open question as to whether or not increased
weapon variability increases player retention
![Page 50: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/50.jpg)
Where Can You Begin?
• Start asking questions
• Gather data—any data
—Playtests
—Gameplay metrics
—Steamstats
—Forum posts/emails/Reddit
• Tell us what data you’d like us to provide
![Page 51: MIKE AMBINDER, PhDmedia.steampowered.com/apps/steamdevdays/slides/data.pdf• Platform for recording gameplay metrics • Kills, Deaths, Hero Selection, In-Game Purchases, Matchmaking](https://reader033.vdocuments.site/reader033/viewer/2022060411/5f108ff97e708231d449b88f/html5/thumbnails/51.jpg)
THANKS!!!