Start from the problem, not the technology

The most common failure in AI architecture happens before any model is chosen: the team starts from the technology and works backward to a problem. A durable design runs the other way. You begin with the job to be done, the users, and the constraints, and only then ask what class of solution fits.

Write the problem as a sentence a non-engineer would recognize, such as: support agents spend too long finding the right policy answer. Now attach the constraints that will actually decide the architecture — how fast the answer must arrive, how accurate it must be, what data it may touch, and what a wrong answer costs. These four constraints, not the model, shape almost every later decision.

Notice that none of this mentions a model name yet. That is deliberate. When you lead with the problem, the same feature might turn out to need search, a rules engine, a small classifier, or a large language model, and you keep the option to combine them. When you lead with the model, you tend to force every problem into it.

A useful habit: for each feature, name the single hardest constraint. If the hardest constraint is latency, that rules out some approaches immediately. If it is auditability, that rules out others. The architecture is largely the shape of the constraints, made concrete.

The tradeoff angle: every hour spent sharpening the problem statement saves days of rework, because a fuzzy problem invites an over-engineered solution that satisfies nobody.