Two oranges and a lemon

In 1747, a Scottish naval surgeon called James Lind was trying to work out how to cure scurvy, and there was no shortage of advice.

So what has an 18th-century naval surgeon got to do with your marketing budget?

Everyone had a theory. Cider. Vinegar. Seawater. Rather than back the most persuasive one, Lind took 12 sailors with similar symptoms, split them into six pairs and gave each pair a different remedy. As he told it, the pair given two oranges and a lemon a day recovered far faster than the rest.

It’s one of the first controlled trials on record. (Historians have since questioned some of the details - Lind’s own account is brief - but the method is what stuck.)

One small test beat centuries of opinion.

And that, for me, is what predictability really is. Not a forecast. A habit.

The operating model

Running hypotheses and testing them has been central to every role I’ve had. I like a very structured approach - structured enough that I’ve built an operating model for it, to help marketers write hypotheses that can actually be tested.

The hardest part to teach wasn’t the method. It was getting people to see a failed test as just as positive as a successful one, and to write both down and share them with the wider team. A win you don’t document gets forgotten. A failure you don’t document gets run again by someone else next year.

Why does this matter so much? Because without it you can’t scale. You just spend more money, faster, on tactics nobody has proven work.

So I’ve broken predictability into three parts: write the model down, test it properly, and keep the loop running.

1. Write the model down

Every marketing plan is a hypothesis, whether or not anyone calls it one.

Work back from the revenue target and you get a chain of assumptions: this many opportunities, at this win rate, from this much pipeline, from these programmes. Each link is a guess until it’s been tested. Predictability means knowing which links you’ve proven and which you’re hoping for.

Most teams can’t tell the difference. And it shows. When CSO Insights tracked the deals that sellers forecast to close, fewer than half of them did - year after year. A forecast built on untested assumptions is a wish with a decimal point.

Writing the model down also shows you where not to spend. Brian Balfour’s Four Fits framework makes the point that some channels simply can’t pay their way at your price point, however well you run them. That’s far cheaper to find out with a small test than with a full-year budget.

2. Test it properly

In the 1920s a colleague of the statistician Ronald Fisher claimed she could taste whether the milk had gone into her cup before the tea. Rather than argue, Fisher designed a test: eight cups, four made each way, served in random order.

The clever part came first. Fisher worked out that guessing all eight correctly by luck was a 1 in 70 chance, so he knew what result would convince him before a single cup was poured. (She got them all right, by most accounts.)

Most marketing “tests” skip that step. Three things change at once, there’s no control, and a winner gets declared on gut feel a fortnight later. So in my operating model every test starts with a written hypothesis - if we do X, then Y will change, because Z - and a line saying what result would change our minds.

You also have to expect to be wrong. A lot. When Ronny Kohavi looked back at experiments run at Microsoft, about a third improved the metric they targeted, a third made no difference and a third made things worse. Across companies, he found 60% to 90% of ideas don’t improve what they were designed to improve.

That’s why a failed test is a good result. It’s money you didn’t spend scaling the wrong thing.

It also keeps you humble about your own instincts. I’ve been surprised again and again - customer proof points that lowered conversion in some situations, ugly, hard-to-read pages beating sleek ones. It all comes down to context, and context is really hard to work out in your head. That’s exactly why you test.

3. Keep the loop running

Here’s the part most people miss. Testing isn’t one and done.

One of my favourite results came from removing the phone number field from some critical forms. It felt scary - how are the SDRs meant to follow up? But the data suggested that, in one region at least, it helped both the self-serve business and the sales-led business.

Would it hold everywhere, forever? I doubt it. People change. Customer experiences and expectations change. The test that’s right today becomes the assumption that’s wrong in two years.

So the tests that form the foundation of your operating model need to be revisited, on purpose, as part of how you go to market. This is the bit that never gets thrown away once you find a channel that works. It’s how a revenue team stays predictable for its whole life, not just its first year.

What good looks like

A few golden rules I use to check whether testing is really part of the engine:

• A written hypothesis for every test. If, then, because - plus what result would change your mind.
• One change at a time, against a control. Otherwise you learn nothing you can repeat.
• Kill criteria agreed up front. Decide when to stop before you’re attached to the idea.
• Failures reported like wins. Same template, same meeting, same airtime.
• A shared test log. Anyone can search what’s been tried, by whom, and what happened.
• A re-test calendar. The tests your plan depends on get run again on a schedule.
• Assumptions labelled. Every conversion rate in the plan is marked as tested or assumed.

For me, yes and no

There’s a fair challenge here. B2B isn’t Microsoft. You might close a few dozen enterprise deals a year, the cycles run for months, and you can’t A/B test a buying committee. Even the statistical tools struggle - Gartner notes marketing mix models have less credibility in B2B because of long, variable deal cycles.

All true. You’ll rarely get the clean, significant results a consumer website gets.

But that’s an argument for testing smarter, not testing less. Test where the volume is - forms, pages, early stages of the funnel. Use leading indicators you can read in weeks rather than quarters. Hold out a set of accounts when you can’t split individual visitors. And treat what you learn as directional evidence that builds up over time.

For me, a directional test beats a confident opinion every time.

Where to start on Monday

1. Mark your assumptions. Take three conversion rates from your current plan and label each one tested or assumed.
2. Write one proper hypothesis. If, then, because - and what result would change your mind. Then run it.
3. Start a test log. Add last quarter’s failures first. They’re the entries people will thank you for.

What’s the test result that surprised you most? I’d love to hear the ones that broke other people’s assumptions too.

Next in the series: Lind had his answer in 1747, but it was 1795 before the Royal Navy made citrus standard - 48 years later. Proving something works is one job. Getting an organisation to change is another. That’s optimisation and scale.