Set the lens
Write the ideal customer profile and this year’s one goal before sorting, and score strategic fit against them. Without a written lens, each person scores fit against their own idea of the strategy.
Innovation NavigatorStrategic Game
Customer Signal vs Noise
from the loudest input wins→to the recurring, strategic signal wins
The Goal
Find the customer signal worth acting on: the recurring, strategic pattern under the loudest feedback, with a kill criterion set before the team commits.
Ideal For
Product and go-to-market teams whose roadmap bends to the latest escalation, roadmap and planning cycles, customer council and post-launch debriefs, and any team serving customers, partners or a community that hears more feedback than it can act on.
The Shift
From the loudest input wins to the recurring, strategic signal wins.
The Science
We tend to judge how common something is by how easily examples come to mind, so vivid, recent inputs feel more frequent than they are, and the feedback people volunteer tends to come from the most delighted and the most frustrated. Rating every input on the same two criteria tends to give more consistent judgments than gut feel, and deciding in advance what would change our mind tends to guard against seeing only the evidence that supports a favored idea.
Customer Signal vs Noise, step by step
- Set the lensWrite the ideal customer profile, this year’s one goal and any market the team has chosen to enter at the top of the canvas, and agree what “recurring” means (for example, three or more ICP accounts in 60 days).
- Pool the inputsSilently, one input per sticky from the last 60 days: what it is, where it came from, and how many distinct accounts raised it. Merge duplicates and add up the accounts.
- Run three checksMark anything here mainly because it’s recent (R) or because of who said it or how loudly (S), and add a sticky for every voice that’s missing (M): lost deals, churned accounts, quiet customers.
- Plot on two filtersPlace each input on frequency (distinct ICP accounts) × strategic fit (against the lens), in silence first; the most senior person places last. Signal: investigate. Worth watching: track. Distraction: reframe or ignore. Noise: drop.
- Pick and killCircle one signal and write the investigation (the question, who you’ll ask, by when) and the kill criterion: “We will reconsider this priority if we discover that ___.”
- CommitOne owner, the next step written as “By [day], [owner] will [step],” and a 14-day review to build it, test it or move it.
The techniques, and why they work
Count accounts, not mentions
Frequency is the number of distinct customers in the ICP who raised it, checked for recency, salience and missing voices. We tend to judge how common something is by how easily it comes to mind, and volunteered feedback leans to the extremes.
Kill before you commit
Pick one signal, write the investigation and the evidence that would prove it was noise, then name an owner and book a 14-day review. Deciding in advance what would change our mind tends to guard against seeing only the evidence we want.
Selected research
- Tversky, A., & Kahneman, D. (1973). Availability: A heuristic for judging frequency and probability. Cognitive Psychology, 5(2), 207–232.
- Hu, N., Zhang, J., & Pavlou, P. A. (2009). Overcoming the J-shaped distribution of product reviews. Communications of the ACM, 52(10), 144–147.
- Dawes, R. M. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34(7), 571–582.
- Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175–220.
- Sleesman, D. J., Conlon, D. E., McNamara, G., & Miles, J. E. (2012). Cleaning up the big muddy: A meta-analytic review of the determinants of escalation of commitment. Academy of Management Journal, 55(3), 541–562.
From the research
The thinking behind this lab, published in full: