The dashboard says we're fine but I don't believe it.

What to do when the dashboard says fine but you do not believe it

Lower Manhattan, nineteen hundred and six. Hetty Green is seventy-one years old and operating from a desk in the Chemical Bank's New York offices. She holds no formal title at the bank. She conducts her business in plain dress, severe black, the same outfit she has worn since her husband died fifteen years earlier. The bank's clerks frequently fail to recognise her on her way in.

She is, by nineteen hundred and six, the wealthiest woman in the United States. The financial press has been writing about her for thirty years as the Witch of Wall Street — a caricature that is both convenient and inaccurate.

The dashboard, in nineteen hundred and six, says American capital markets are fine.

Equity prices are rising. Trust companies are expanding. The wealthy New Yorkers Green watches daily — she walks through lower Manhattan as part of her routine, observing the auction houses, the real-estate offices, the luxury showrooms — are unloading. Her phrase, recorded in coverage that survives in The Literary Digest in nineteen sixteen and reconstructed in Janet Wallach's two thousand and twelve biography, is that they are getting rid of palatial residences to automobiles — the visible kit of conspicuous wealth — for cash.

Ahead of something they are not yet naming.

Green's response is not to argue with the rising-market dashboard. It is to act on the territory the dashboard isn't describing. She sells out of equity positions and accumulates cash through nineteen hundred and six and into nineteen hundred and seven. Through a year of a rising market, she takes the unpopularity that comes with sitting in cash while everyone around her is making money.

When the Panic of nineteen hundred and seven hits in October — the run on the Knickerbocker Trust Company, the cascade through the trust-company sector, the freeze in the call-loan market — Green is one of a handful of New York principals with substantial uncommitted liquid capital.

In her own voice, recorded afterwards: When the crash came I had money, and I was one of the very few who really had it. They had to come to me.

What comes is the City of New York. The municipal government cannot raise capital through the normal channels; the trust companies that have been the backbone of municipal short-term financing are insolvent or paralysed. Green lends the City roughly one-point-one million dollars at six per cent interest, at a moment when comparable lenders are charging forty per cent or refusing entirely. She is the only woman invited to the J. P. Morgan crisis meeting that coordinates the wider response.

The decision Green is remembered for is the loan. The decision that mattered was the one she made twelve months earlier — to read the rising-market dashboard against the lived market, trust the lived market, and accept the unpopularity of sitting in cash through a bull year.

Why an accurate dashboard can still measure the wrong thing

When the dashboard says you are fine and you don't believe it, the question is rarely whether the dashboard is accurate. The question is whether what it is accurate about is the thing your business actually depends on. The dashboard is reading the market the rising market believes it is. Green was reading the market the falling market would shortly reveal.

So let's go to the office and work through it.

Start by reading what kind of scepticism this is

"The dashboard says we're fine but I don't believe it."

The feeling is sceptical.

The numbers all look reasonable. The team is hitting the metrics they have been asked to hit. And something about the experience of being in the market is telling you a story the dashboard isn't, and you cannot yet name what the gap is.

Two choices. They look like the same scepticism. Different fundamental causes.

When the dashboard is wrong in a way you can measure

Choice one: the dashboard is wrong in a way you can measure. You suspect specific metrics are decorative — they look healthy because they are designed to look healthy, and they are not actually driving any decisions.

If that's the read, turn the dashboard off for a week. Not the underlying data collection. Just the dashboard itself. Then see who complains.

G. K. Chesterton, on the question of fences you find in a field, was clear about the symmetric move. Don't remove a fence until you know why it's there. And — conversely — do remove the fence if nobody can tell you why it's there. The metrics that nobody misses are the ones that weren't driving any decisions anyway, and you have just saved yourself the time you were spending watching them. The metrics that someone genuinely cannot work without will be back on the dashboard within the hour, and now you know which ones they are.

This is an unusually clean test because it does not require you to argue with anyone. The absence of complaints is the evidence.

When the numbers look right but feel wrong

Choice two: the dashboard is wrong in a way you can only feel. The numbers all look plausible, but something about the experience of being in the market is telling you the story the dashboard isn't.

If that's the read, leave your desk. Go and sit with one customer, in their workplace, doing their actual work with your actual product, and watch without commentary.

Steve Blank, formalising what he calls getting out of the building, has been clear about this for thirty years. The gap between what a dashboard shows and what a user experiences is real. It is specifically the thing that can't be captured in any aggregate metric, because aggregates average across the friction that individual users feel moment by moment. One hour of watching one customer will usually tell you more about whether your dashboard is lying than a week of drilling into the data.

Measurable, or only-feelable. Same scepticism. Two different first moves.

How to test the dashboard against the real world

Three tools. The discipline is to verify the dashboard against reality rather than against itself. Two of the three tools were unpacked earlier in the toolkit; the third is new.

The first is

Work against raw data, not the vanity layer

Analytics and Data Analysis.

We unpacked it at scenario twenty-two — the operations-research-and-modern-data-warehouse tradition, applied with the discipline of stripping the vanity layer and working against raw data rather than headline summaries.

The reason it matters here is that aggregates hide everything that matters when you don't trust the dashboard. The dashboard's job is to summarise. Summaries average out variance, drop-offs, and the specific flows where individual users abandon. If you suspect the dashboard is lying, the cure is the disaggregated data underneath it.

The discipline for this scenario is to look at individual user trajectories rather than aggregated dashboards. The interesting information is in the variance — the cluster of users who behave differently from the average, the specific funnel stages where drop-off concentrates, the cohorts whose curves diverge. The aggregates tell you everything is fine; the disaggregated data tells you whether the fine is broadly distributed or concentrated in a way the dashboard is hiding.

The second is

Pull the qualitative signal the dashboard never reaches

Voice of Customer.

We unpacked it at scenario twenty-one — the Griffin-and-Hauser MIT formalisation of structured listening to capture customer needs in product-development decisions, drawing on the Toyota and kansei engineering lineage.

The reason it matters here is that the qualitative signals about whether the product is working are sitting in places the dashboard does not reach. Support tickets. Reviews. Customer conversations. App store comments. Cancellation surveys. The places where users are already telling you what they think, in their own words, without being asked to fit into the dashboard's structure.

The discipline for this scenario is to pull qualitative signals out of the systems that are already collecting them. The team's support team is reading messages every day; they have a different view of the product than the dashboard. The reviews team is reading the reviews; they have a different view again. The cancellation team has talked to people leaving. The Voice of Customer pull aggregates these into a picture the dashboard could not produce, because the dashboard is not built for unstructured signal.

The combination of disaggregated quantitative data and aggregated qualitative signal is what turns dashboard scepticism into structural diagnosis.

The third is

Watch for the workarounds users never report

Customer Interviews — and this one we are unpacking properly.

Customer Interviews as a structured discipline traces through Steve Blank's Four Steps to the Epiphany, two thousand and five, and the wider customer-development tradition, with deeper roots in ethnographic interviewing and the contextual inquiry tradition from human-computer interaction work in the nineteen-eighties and nineteen-nineties. Where the Problem Interviews we covered at scenario twenty-eight probe the customer's pain abstractly, Customer Interviews probe how they actually use the product — concretely, in their actual workflow.

The reason Customer Interviews matter when you don't trust the dashboard is that the dashboard cannot capture what users do that the product wasn't designed to support. Workarounds. Abandoned attempts. Half-completed flows. The dashboard sees what fits its instrumentation; everything outside the instrumentation is invisible. Customer Interviews surface what the instrumentation is missing.

The unique insight is the specific-question discipline. The interview is not how do you feel about our product?, which produces polite generalities. It is walk me through the last time you used it, what were you trying to do, what happened, what did you do instead? Concrete. Recent. Behavioural. The answers are usable because they describe behaviour rather than opinion.

What you get when the interviews are done well is the one detail that makes the dashboard's green light look different. Not the customer's overall view of the product, which the dashboard could approximate. The specific friction the customer encountered last Tuesday at four in the afternoon, which the dashboard did not record because the dashboard was watching different things.

So. How to run them.

Pick. Across active, churned, and never-converted segments. Five is enough to find patterns; ten is the maximum useful.

Schedule. Less than an hour each and you don't get past the polite layer. More and the conversation drifts into hypothetical territory.

Last. Not their general experience. Tell me about the last time you used our product. What were you trying to do? What happened? What did you do next?

Workarounds. The interesting moments are the ones where the user tells you they used your product to do something it wasn't designed for, or used a different product alongside yours, or gave up and did something offline. The workaround is the diagnostic.

Aggregate. The same workarounds will keep surfacing across five interviews. Or different ones will, which is information about how diverse your user base is.

Don't argue. When a customer tells you they don't use the feature you were proud of, don't defend the feature. Note it. The interview is data collection, not sales.

The combination of all three tools — disaggregated data, qualitative signal, direct interviews — produces a picture the dashboard alone could not.

That's the toolkit. One more story before we close.

The Green story we opened with showed an investor reading the rising-market dashboard against the lived market — and trusting the lived market a year before the lived market became the dashboard's next reading. The story we close with shows the same move a hundred and ten years later, at the scale of an agricultural cooperative — a chief executive whose strategy dashboards had been reporting on a condition that didn't exist outside the building.

A precedent: moving the company into the gap the dashboard hid

Minneapolis, two thousand and eighteen onwards. Beth Ford becomes Chief Executive Officer of Land O'Lakes in August two thousand and eighteen — the first openly gay woman to lead a Fortune Five Hundred company, and the chief executive of a fourteen-billion-dollar member-owned agricultural cooperative.

Within her first eighteen months she has visited member farms in forty-eight states.

What she finds in the field — and what she returns to repeatedly in her own voice, on the Fortune Leadership Next podcast in August two thousand and twenty — is a diagnostic the corporate dashboards in Minneapolis cannot see.

The strategy dashboards at headquarters are reporting progress on the assumption of reliable broadband in rural America. And reliable broadband doesn't exist. Her framing of the gap is plain. I can't auto-steer the tractor and I can't pull in this data. The data Land O'Lakes's strategy depends on has been, in her phrase, a ghost — present in the reports, absent in the field.

Her response is not to adjust the dashboards. It is to move the company into the infrastructure question itself.

She convenes the American Connection Project in two thousand and twenty with more than one hundred and seventy-five organisations. Land O'Lakes makes rural broadband access a public-policy fight the company will visibly lead.

The result is tangible by late two thousand and twenty-three. The American Connection Project's lobbying is cited in reporting on the sixty-five-billion-dollar rural broadband provisions of the Infrastructure Investment and Jobs Act. Ford's two thousand and twenty-two Gardiner Lecture at Kansas State codifies the underlying posture — executive judgment had to be exercised against lived conditions, not against the reporting layer that abstracted them.

When the dashboard says you are fine and you don't believe it, the useful move is rarely a different metric. It is to go and find out what the people the dashboard is meant to describe would tell you the dashboard is missing.

So. Your Next Move from this playbook.

Which chart on your dashboard does everyone look at and nobody acts on — and what would happen if you deleted it next Monday?