{{ p.title }}

A catalog of ten thousand products suggests ten thousand decisions, and many AI programs are scoped accordingly: a platform first, then integrations with every channel, then months of data work before a single recommendation reaches a customer. By the time the first decision ships, several things have changed at once, and nobody can say with confidence which of them moved the numbers. The program has spent its budget and still has to argue for its value.

The teams that reach production tend to do the opposite. They pick one decision, put it on one surface, and measure it against a control group before building anything else. The choice of that first decision matters more than its size, because the goal of a first deployment is a result the business can trust.

Why broad rollouts are hard to evaluate

When recommendations launch on the homepage, product pages, search and email in the same quarter, every change interacts with every other. A lift on product pages might come from better recommendations, from a seasonal shift in traffic, or from the email campaign that pushed returning customers back to the site. Without a clean comparison, the result becomes a matter of interpretation, and interpretation tends to favor whoever is presenting it.

Broad rollouts also delay the first useful feedback. Data pipelines, catalog clean-up and integration work all have to finish before anything reaches a customer, so the earliest signal about whether the system makes good decisions arrives months after the decision to build it. Problems that a small deployment would have exposed in its second week surface only after the architecture is fixed.

What makes a good first decision

The best first decision is chosen for how clearly it can be measured. In our experience it meets five conditions:

• Visible. Customers see it directly, so its effect shows up in their behavior rather than in an internal metric.
• Frequent. It happens often enough that a month of traffic produces a readable result. A product-page recommendation on a busy category qualifies; a quarterly bulk-order flow usually does not.
• Tied to a number the business already tracks. Conversion, order value or return rate, defined before launch, so nobody has to invent a success metric afterwards.
• Contained. One surface and one owner, so the change can be reasoned about and everyone knows who decides when to expand or stop.
• Reversible. It can be switched off without side effects, which makes the pilot easy to approve.

On a fragrance storefront such as Uquda, which we designed and run, a decision like “which three scents to show next to the one the shopper is viewing” meets all five. On a grocery site, the equivalent might be the substitute suggested when an item is out of stock.

The architecture of starting small

A small first deployment does not have to be a prototype that gets thrown away. The useful pattern is a single integration point that later surfaces reuse. In our case that point is one call, POST /v1/match, available as a REST call, an SDK or a drop-in widget. The first surface calls it with the shopper’s context and receives a ranked decision with its reason. The second surface calls the same endpoint with its own context.

Because every surface calls the same decision path, the work done for the first decision carries forward: the dimensions defined for the category, the guardrails the merchandising team configured, and the vocabulary used for reasons. The second surface needs its own integration and nothing more. This is also why the first integration is quick. Connecting one surface typically takes an afternoon, and the first decision is usually live within about two weeks, most of which is spent agreeing on the metric and configuring guardrails.

Measuring the first decision

The pilot runs against a holdout. For the first month, half of the shoppers on the chosen surface receive the new decisions and half keep the current experience, assigned at random. The difference between the two groups on the agreed metric is the incremental effect of the decision, separated from seasonality, campaigns and everything else that changed in the same period.

A 50/50 split gives up some upside during the pilot, since half the traffic does not get the new experience. In exchange, the result is readable in weeks rather than months, which is the point of the exercise. After the pilot, the holdout drops to 10% and stays in place, so the effect keeps being measured as the catalog, the season and the models change. A system that stops measuring after launch gradually loses the ability to show that it still works.

It also matters which number is reported. Revenue that passed through a recommendation is influenced revenue. Revenue that the holdout shows would not have happened otherwise is incremental. Only the second justifies expansion.

From one decision to the next

Expansion repeats the same steps. The second decision goes on a new surface or into a new part of the journey, gets its own holdout and its own metric, and reuses what the first one established. Each step costs less than the last because the integration, the dimensions and the guardrails already exist, and each produces its own evidence, so the case for the next step rests on results rather than projections.

The order is worth planning. Decisions that share dimensions with the first one are cheaper to add. A team that started with product-page recommendations can move to search ranking or post-purchase email relatively easily, since the same reading of the shopper applies. Moving to a category with different dimensions is closer to starting again.

Trade-offs

Starting small has costs. A narrow pilot can understate the value of the system, because some benefits only appear across surfaces: a shopper who sees consistent choices and reasons on the site and in email behaves differently from one who sees them in one place. A single-surface result is a floor, and it should be presented as one.

Traffic is a real constraint. If the chosen surface does not see enough sessions, a month will not produce a result anyone should trust, and the pilot will either run long or end inconclusive. Choosing the first decision by traffic and margin, rather than by internal enthusiasm, avoids most of this.

A small first result can also be politically fragile. If the metric and the threshold for success are not agreed in writing before launch, a modest but genuine improvement is easy to dismiss. The discipline that makes the measurement credible has to extend to the decision about what counts as success.

What to settle before the first deployment

Six questions are worth answering in writing before any integration work starts: which decision, on which surface, measured by which metric, over what period, with what threshold for expanding, and who owns the switch. If any of them lacks a clear answer, the first decision is not yet small enough.

How a first build runs, from scoping to the first live decision, is laid out step by step on our How it works page.

See how a first build runs