Over the past few months, I’ve built two systems that use Claude for very different jobs: one manages advertising campaigns for multiple clients, while the other searches for and evaluates job listings. You don’t need to know either project in detail to understand the five things I learned building them, because none of them are strictly technical. They’re about how to think about the problem before you even open a chat with AI.
A Prompt Solves a Question. A System Solves a Problem That Keeps Repeating
Asking AI to write a piece of text, summarize a document, or come up with an idea is useful, and then it’s done. The answer is valuable for that particular instance. A different problem emerges when the same question would be useful to ask every day, with different data but the same evaluation criteria. In that case, manually repeating the question every time is a waste. It’s worth building something that can do it on its own, using the same logic and updated data.
The signal that you’re dealing with this type of problem is simple: if you notice yourself making the same evaluation, based on the same criteria, with data that changes every week, you probably don’t need a better prompt. You need a process.
AI Is Only as Reliable as the Criteria You Give It
The biggest difference between a system that works and one that produces noise isn’t the model you use. It’s how precise the criteria are for evaluating something. “Tell me if this campaign is performing poorly” is a vague question and produces vague answers. “Tell me if this campaign has increasing spend without a proportional increase in leads over the past few days” is a precise criterion and produces a verifiable answer.
The most important work in both systems I built wasn’t writing instructions for the AI. It was precisely defining, before even discussing the problem with a model, what makes something “a problem worth flagging” and what doesn’t.
The First Results Almost Always Need Fixing, and That’s Fine
Neither system worked well in its first version. The one for advertising campaigns initially flagged too much. Any minor variation ended up on the list of things to check, making the list just as long as the manual review it was supposed to replace. The job application system initially searched using the wrong criteria and returned off-target results.
In both cases, the correction only came after looking at enough of the wrong results to recognize the pattern behind the error. Building with AI is more like setting up a process and observing it than writing a finished piece of text: the first version is mainly there to help you discover what’s wrong, not to already be the right one.
Decisions That Carry a Cost Stay With the User, Not the AI
In both systems, there’s a clear line between what the AI decides and what the person decides. The campaign system flags where to intervene and suggests alternatives within precise limits, but it never changes an audience or budget on its own. The job application system evaluates and ranks opportunities, but it never submits an application on someone’s behalf.
The criterion I use to draw this line is simple: if getting it wrong costs little and is easy to fix, AI can make the suggestion. If getting it wrong has a real cost, money spent or an opportunity lost, the final decision stays with the person who has the context to make it, and that context rarely exists entirely in the data the AI can see.
The Time It Takes to Build a System Only Pays Off If the Problem Really Repeats
Building one of these systems takes more time than doing the same thing manually once. The advantage only appears if the problem keeps coming back: every day, every week, with different data but the same underlying logic. If it’s a problem that happens once and never again, the fastest system is still to ask directly without building anything around it.
That’s the question I now ask myself before starting any similar project: will I have to do this dozens of times with different data, or is this something I’m about to do just once? In the first case, it’s worth building. In the second, a well-written question is enough.
