At first, I tried simply asking an artificial intelligence model what it thought about a campaign that was performing poorly. The answer was always plausible and almost always useless: increase the budget, try a broader audience, test new creatives. These are recommendations you could give to virtually any campaign, in any industry, at any point in time. The problem wasn’t that they were wrong in theory. It was that they said nothing about that specific campaign.
This article explains how I moved from an AI that comments to a system that diagnoses, and why the most important part of the work was deciding what to forbid it from saying.
The Problem With Generic Advice
Generic advice is dangerous precisely because it isn’t false. “Test new creatives” is almost always a reasonable idea, which also makes it an easy answer to give when you haven’t actually figured out what’s happening. If the exact same recommendation comes up for a campaign that generates plenty of clicks but no leads and one that isn’t generating clicks in the first place, it means whoever is producing it, human or AI, isn’t distinguishing between two completely different problems that require completely different solutions.
So the first job was to eliminate the advice that works everywhere, leaving only recommendations that apply to a specific case.
How to Tell a Creative Problem From a Landing Page or Form Problem
The journey of someone who sees an ad has clear stages: they see the creative, potentially click, arrive on a page or open a form, and potentially submit their information. Every point where people drop off represents a different problem.
- If few people see the ad and even fewer click, the problem is probably with the image or copy: they’re not capturing attention.
- If many people click but the page or form doesn’t convert, the problem comes after the click: the creative has already done its job by bringing interested people in. It’s what happens afterward that isn’t working.
- If the frequency with which the same ad is shown to the same people becomes too high, the problem is neither of the above. The audience has been “worn out,” and what is needed is fresh reach, not a change to the copy or landing page.
These are three problems that, from the outside, produce the same symptom: few leads. But they require opposite interventions. A system that stops at the symptom will always recommend the same thing. A system that breaks the journey down into stages can identify which stage is failing.
Why I Had to Forbid the AI From Giving Certain Types of Advice
The least obvious part of the work wasn’t teaching the system to read the data. That part is relatively straightforward. It was explicitly defining what the system shouldn’t suggest.
The most concrete example: in the first version, when analyzing an ad’s copy, it would sometimes suggest alternative copy that had nothing to do with the actual product, generic marketing-manual phrases that could have been used to sell anything and therefore, in practice, weren’t particularly good at selling anything. The fix was to impose a simple rule: every suggestion has to stay within the same offer, the same audience, and the same industry as the existing copy. Not “write copy that converts better” in the abstract, but “write an alternative for this product, for these people.”
The same logic applies to higher-cost decisions, such as changing an audience or campaign structure. Here, the system flags that something needs to be evaluated, but it doesn’t prescribe the solution. These are decisions that require context, such as commercial agreements, seasonality, and available budget, that the model simply doesn’t have. Pretending it does would create confidence, not accuracy.
A Concrete Example of a Correct Diagnosis
The clearest example I’ve seen was a campaign with a huge number of clicks and very few leads, a ratio that at first glance looked like a creative failure. The automated diagnosis indicated the opposite: the creative was performing well, given how many clicks it was generating. The problem was between the click and the lead, meaning it had to be looked for on the landing page or in the form. When I checked manually, the form was indeed too long compared with what the offer promised.
Without that distinction, the obvious intervention would have been to change the image, the only visible part of the problem, while leaving the actual cause untouched.
Where AI Really Helps and Where the Human Eye Is Still Needed
The system is reliable when it has to read numbers and place them at a specific stage of the journey. This is a mechanical, repeatable task that AI handles well and faster than a person scrolling through accounts one by one. It becomes less reliable when the question requires knowledge that isn’t present in the data, such as a specific agreement with a client, a seasonal event, or a positioning decision made somewhere else.
The practical rule I use is this: the system can say, “There is a problem here, and it is probably at this point in the journey.” I don’t ask it to say, “So do exactly this,” when the answer requires knowledge it has no way of having.
