How we know what worked
The measurement playbook
How to tell whether a change worked: what to measure, whether your traffic can answer the question, and how to read a result honestly.
Most product changes move metrics less than teams expect, and many early wins fade. So every move Gamify It suggests comes with a way to read it, sized to the traffic you actually have.
When traffic is too low for a reliable A/B test, we say so, recommend bigger swings and label the read as directional. A result that is inconclusive is still a result.
Measurement
The playbook, step by step
The measurement entries in our library, grouped by the question they answer.
01
Pick what to measure
One metric that reflects delivered value, the signals that move it, and the guardrails that keep it honest.
- North Star Metric and input metricsone value metric plus its inputs
- HEART and Goals-Signals-Metricsuser-centered metric design
- Evidence grade A. Robust evidence: replicated, peer-reviewed research.Metric roles: decision metric, guardrails and revenue qualitysuccess metrics and counter-metrics
- Activation moments and the aha thresholdsetup, aha and habit moments
- Reading retention curvescohort curve shapes and definitions
- Growth accounting and the user state modelwhere active users come from and go
- Evidence grade A. Robust evidence: replicated, peer-reviewed research.Measuring habit strength (SRHI and SRBAI)self-reported automaticity
- Evidence grade A. Robust evidence: replicated, peer-reviewed research.NPS and better-targeted survey metricssatisfaction and effort surveys
- PMF survey and segment enginemust-have survey by segment
02
Know what your traffic can tell you
Before testing, check what your data supports and how many weeks a readable result would take.
- Measurement readiness levelswhat the data can support
- Evidence grade A. Robust evidence: replicated, peer-reviewed research.Sample size, minimum detectable effect and weeks to readcan this step be tested in time
- The low-traffic loop (directional reads)testing with small samples
- Using benchmarks honestlybenchmark bands with definitions
03
Run tests you can trust
Decide the rules before the data arrives, expect most ideas to lose, and check that wins last.
- Evidence grade A. Robust evidence: replicated, peer-reviewed research.Experiment hygiene: trust checks before reading resultstrustworthy A/B testing checklist
- Experiment win rates and the winner's cursehow often tests really win
- Evidence grade A. Robust evidence: replicated, peer-reviewed research.Holdouts and long-term measurementdoes the effect last
- Evidence grade B. Moderate evidence: solid studies with some limits.Qualitative methods behind the numbersinterviews, usability tests and diaries
Limits
What we don’t claim
These caveats travel with the frameworks everywhere they appear, on this site and in every answer Gamify It gives in your AI assistant.
- The Hook Model is a practical design tool, not a validated or complete theory of behavior.Hooked, Introduction
- Octalysis is a practitioner framework built from observing games, not a validated scientific model.Actionable Gamification Ch.15, pp.399-400
- Left Brain and Right Brain are a metaphor for extrinsic and intrinsic motivation, not a claim about neuroscience.Actionable Gamification Ch.13, pp.342-343
- The evidence is correlational until it is tested in your product.
- Habit metrics need the longer habit horizon (weeks to months) before they can be read.
- The threshold is a heuristic, not a measured cut-off.
- Bartle himself doubted his player types outside the settings he studied.
Go deeper
More from the knowledge base
- Framework · Behavioral science
Realistic effect sizes for behavioral levers
what to expect from a single change
Single behavioral tweaks usually move a metric by low single digits. Across 126 nudge-unit trials (23 million people) the average lift was 1.4 percentage points, about 8% relative, versus 8.7 points in journals; and most program effects fade once the program ends. Plan bundled changes and durability checks.
- Framework · Behavioral science
Claims never used as mechanisms
replication-crisis do-not-use list
Several popular behavioral claims failed replication or were retracted: willpower as a depleting resource, subtle priming, the sign-at-the-top honesty nudge, the Zeigarnik memory effect, 21-day habits, and choice overload or decoys as universal laws. They are never presented as working mechanisms; the nearest valid lever is offered instead.
- Framework · Behavioral scienceEvidence grade B. Moderate evidence: solid studies with some limits.
Streaks and streak repair
streak design with repair
People repeat a logged behavior more when the record shows an intact streak than a broken one, even with the same past behavior; the drop after a break shrinks when the streak can be repaired. Match the streak unit to the natural cadence and offer freezes or repair windows. Moderate evidence (B).
Sources
Actionable Gamification · Yu-kai Chou
Ch.17, p.463 · Ch.10, p.253 · Ch.12, p.307
Influence · Robert Cialdini
Ch.3, p.76
Hooked · Nir Eyal
Ch.3 · Ch.8 · Ch.1 · Ch.7
The North Star Playbook · Amplitude
Benchmark confidence B. A reputable dataset.Measuring the User Experience on a Large Scale (HEART) · Rodden, Hutchinson and Fu, CHI 2010 (Google), 2010
Evidence grade A. Robust evidence: replicated, peer-reviewed research.Risk-aware product decisions in A/B tests with multiple metrics · Schultzberg, Ankargren and Franberg (Spotify), 2024
Evidence grade A. Robust evidence: replicated, peer-reviewed research.Email Volume Optimization at LinkedIn · Gupta et al., KDD 2016, 2016
Evidence grade A. Robust evidence: replicated, peer-reviewed research.Define user activation moments · Reforge (Casey Winters and Shaun Clowes)
Benchmark confidence B. A reputable dataset.The 7% retention rule (2025 Product Benchmark Report) · Amplitude, 2025
Benchmark confidence B. A reputable dataset.What is good retention · Lenny's Newsletter (Lenny Rachitsky and Casey Winters), 2020
Benchmark confidence B. A reputable dataset.Diligence at Social Capital, Part 1: Accounting for User Growth · Social Capital (Jonathan Hsu), 2015
Benchmark confidence A. A primary source for the benchmark.Meaningful metrics: the Duolingo growth model · Duolingo Blog (Erin Gustafson), 2023
Benchmark confidence A. A primary source for the benchmark.What is the quick ratio hiding? · Tom Tunguz
Benchmark confidence C. A secondary source.Towards parsimony in habit measurement: the SRBAI · Gardner, Abraham, Lally and de Bruijn, IJBNPA 9:102, 2012
Evidence grade A. Robust evidence: replicated, peer-reviewed research.How are habits formed: modelling habit formation in the real world · Lally et al., European Journal of Social Psychology, 2010
Evidence grade B. Moderate evidence: solid studies with some limits.Has the Net Promoter Score been discredited in the academic literature? · MeasuringU (Jeff Sauro), 2019
Benchmark confidence A. A primary source for the benchmark.How Superhuman built an engine to find product-market fit · First Round Review (Rahul Vohra), 2018
Benchmark confidence B. A reputable dataset.Trustworthy Online Controlled Experiments · Kohavi, Tang and Xu, Cambridge University Press, 2020
Evidence grade A. Robust evidence: replicated, peer-reviewed research.Improving the sensitivity of online controlled experiments by utilizing pre-experiment data (CUPED) · Deng, Xu, Kohavi and Walker, WSDM 2013, 2013
Evidence grade A. Robust evidence: replicated, peer-reviewed research.Why you only need to test with 5 users · Nielsen Norman Group (Jakob Nielsen), 2000
Evidence grade B. Moderate evidence: solid studies with some limits.Diagnosing Sample Ratio Mismatch in online controlled experiments · Fabijan et al., KDD 2019, 2019
Evidence grade A. Robust evidence: replicated, peer-reviewed research.Peeking at A/B tests: why it matters, and what to do about it · Johari, Koomen, Pekelis and Walsh, KDD 2017, 2017
Evidence grade A. Robust evidence: replicated, peer-reviewed research.Evaluating for the Long Term: Learnings from Industry · Sigerson et al., arXiv, 2026
Evidence grade A. Robust evidence: replicated, peer-reviewed research.A/B-test power, base rates of success and replication · Kohavi et al., Econ Journal Watch 23(1), 2026
Evidence grade A. Robust evidence: replicated, peer-reviewed research.Focus on the Long-Term: It's Better for Users and Business · Hohnhold, O'Brien and Tang, KDD 2015, 2015
Evidence grade A. Robust evidence: replicated, peer-reviewed research.Megastudies improve the impact of applied behavioural science · Milkman et al., Nature, 2021
Evidence grade A. Robust evidence: replicated, peer-reviewed research.RCTs to Scale: Comprehensive Evidence From Two Nudge Units · DellaVigna and Linos, Econometrica, 2022
Evidence grade A. Robust evidence: replicated, peer-reviewed research.Estimating the reproducibility of psychological science · Open Science Collaboration, Science, 2015
Evidence grade A. Robust evidence: replicated, peer-reviewed research.A multilab preregistered replication of the ego-depletion effect · Hagger et al., Perspectives on Psychological Science, 2016
Evidence grade X. Contested: the evidence conflicts or failed to replicate.Choice overload: a conceptual review and meta-analysis · Chernev, Bockenholt & Goodman, Journal of Consumer Psychology, 2015
Evidence grade C. Limited evidence: few or small studies.On or off track: how (broken) streaks affect consumer decisions · Silverman and Barasch, Journal of Consumer Research 49(6), 2023
Evidence grade B. Moderate evidence: solid studies with some limits.On or off track: how broken streaks affect consumer decisions · Silverman & Barasch, Journal of Consumer Research, 2023
Evidence grade B. Moderate evidence: solid studies with some limits.How Duolingo reignited user growth · Lenny's Newsletter (Jorge Mazal, Duolingo), 2023
Case grade B. Company-reported numbers.Close your rings, but in watchOS 11 it's okay if you don't · 9to5Mac, 2024
Case grade C. Press or secondary reporting.
Source IDs: OCT-M03F26INF-K-03HK-T13HK-MEA04HK-MEA11F15HK-MEA03HK-TRG37F07F04F12F13F03GT#68F22CD8CASE-M01CASE-M25
Book content is paraphrased, never quoted at length. Last verified October 24, 2026.
Your product
Find the stage your product is stuck in
Gamify It reads your product, picks the one move that matters at its stage, and sizes the test to your traffic. Connecting is free; a Pro subscription starts it.