There is a comfortable myth in data work: that the difficult part is the modelling. Pick the right algorithm, tune the right hyperparameters, wrangle the pipeline into shape, and the value falls out the other end. It is a myth because it puts the hardest decision — what are we actually trying to decide? — quietly to one side, and gets on with the part that feels like progress.

In our experience, the majority of analytics projects that fail do so before anyone writes a line of SQL. They fail at the framing stage, where ambiguity is cheap to create and expensive to discover later.

The symptom looks technical. The cause rarely is.

A team comes to us with a model that “isn’t working.” It predicts churn, or demand, or risk, and the numbers look fine in validation — but nobody trusts it, and nobody uses it. The instinct is to reach for a technical explanation: data leakage, drift, an unrepresentative training set.

Sometimes that is the problem. More often, the model answers a question nobody asked. It predicts whether a customer will churn when the business needed to know which intervention would change their mind. Those are different questions, and the gap between them is invisible in any accuracy metric. You can be 95% accurate at the wrong thing.

A model that answers the wrong question precisely is more dangerous than one that answers the right question roughly.

What “framing” actually means

Framing is not a workshop with sticky notes. It is the disciplined work of turning a vague ambition into a specific, falsifiable decision. We push every engagement through four questions before we agree to build anything:

  • What decision will change as a result of this work? If the honest answer is “none — we just want visibility,” that is a reporting problem, not a modelling one, and it should be scoped as such.
  • Who makes that decision, and what do they need in order to trust the answer? A number a director will not act on is not an asset.
  • What does the answer have to beat? Every model competes with the status quo — a rule of thumb, a spreadsheet, an experienced human. If it cannot beat that, it does not ship.
  • How will we know it worked — in business terms, measured after the fact, not in cross-validation?

None of these are technical questions. All of them determine whether the technical work will matter.

A worked example

A lender once asked us to “improve the credit model.” Reasonable enough. But improvement against what? Pressed, the real goal emerged: they were declining too many applicants who would have repaid, and the cost was growth, not defaults.

That reframing changed everything downstream. The objective function shifted from minimising default rate to maximising approved volume at a fixed loss tolerance. The evaluation moved from a single AUC figure to a profit curve the credit committee could read. The model we eventually built was, in algorithmic terms, unremarkable. It worked because it was pointed at the right target.

Had we started with the data, we would have built a better predictor of default — and missed the point entirely.

Why teams skip it

If framing is so decisive, why is it so often skipped? Three reasons, in our experience.

First, it is uncomfortable. Framing surfaces disagreement — between functions, between executives, between what is wanted and what is possible. Building a model is a way to avoid that conversation while looking productive.

Second, it is unglamorous. No one is promoted for a sharply-defined question. They are promoted for a model in production, even one nobody uses.

Third, it is hard to bill for. A week spent deciding not to build something can feel, to a client, like a week of nothing. We think it is the most valuable week in the engagement — and we have learned to say so plainly.

The practical takeaway

If you are about to commission analytics work — internally or with a partner — spend the first week as if the data did not exist. Write down the decision in a single sentence. Name the person who owns it. State what the answer has to beat. Define the after-the-fact measure of success.

If you cannot do those four things, you are not ready to model. And that is not a failure — it is the most useful thing the project will tell you all quarter.

The question comes first. Everything else is implementation.