I build AI agents for my own companies and for clients. For the past 95 days, I have kept a written log of every time my agents got something wrong. In this article I am going to open that log and show you what is inside, because nobody selling you agents will ever do that. And here is my position: the agent fleets the big software platforms are about to sell you will disappoint most of the companies that turn them on. Not because the AI is bad. Because reliability was never in the AI.
Let me show you what I mean.
An agent is a smart model wrapped in plumbing. The plumbing is the product.
Every AI agent has two parts.
The first part is the model. It reads, writes, and decides. It is genuinely impressive, and it is the only part you see in a demo.
The second part is everything around it: the rules about what it is allowed to touch, the checks on its work, the record of what it did, and the human who approves anything important before it happens. Call that the plumbing.
The demo shows you the model. Your company lives with the plumbing.
Here is what my plumbing has cost me so far. My build log has 771 entries across 95 days. Roughly a third of them added nothing new. They were detective work: figuring out what an agent actually did, why it did it, and what has to change so it never happens again.
That is the price for agents I trust. And that is with one operation, mine, whose every quirk I know.
My agents failed in three ways. Yours will too.
First: they make things up, and they sound sure.
Day one of my log. My agent drafted an email confirming a meeting for Monday at 3:00 PM. The woman it was confirming with had never replied. Not one word. The agent invented her answer and wrote it up with complete confidence. My approval step caught it before it went out.
The very next day it happened again, straight through the new safety check I had built overnight.
Here is the lesson. An AI given thin information does not say "I don't know." It writes something believable. And when a believable guess belongs to an agent that can act, the guess becomes a real event on a real calendar, in your name.
Second: they fail silently.
I run an alarm that watches my outgoing email for a certain kind of mistake. This month I discovered the alarm had never been connected. One wrong line in its startup script, there since the day it was born. Every failure it ever had was invisible.
Worse: a backup job for a client's records passed every automated check I had, and never actually ran. For six nights, that company's records existed in exactly one place, which was the machine they were supposed to be backed up from.
The dangerous failures make no noise. And silence looks exactly like health. You only find these by auditing things that appear to be working, which means someone has to be paid to distrust green lights.
Third: every part works, and the result is still wrong.
One morning I opened a follow-up email my system had drafted after a meeting. It was warm, polite, correctly signed, and said nothing. Not one reference to anything we discussed. Two recording bots had joined the same call, my system set aside one copy as a duplicate, and the drafting step happened to read the empty one. Handed a blank page, the model still wrote a friendly email.
Every step reported success. And a warm, empty email over my name is worse than no email, because it looks like the system worked.
Then there was the morning I asked my own scheduling agent, in plain English, to book a specific meeting. Nothing happened. The agent understood me perfectly. Then three of my own safety checks, each doing its job, threw my instruction away. Ninety minutes later I booked the meeting myself and wrote one sentence in my log: "I can't be babysitting her."
An agent system does not fail like a program. It fails like an organization. Every part passes its own test, and the thing that reaches a real person is wrong anyway.
The hard part is your company's quirks, and no vendor can ship those.
My system had to learn that the meeting transcription service garbles names, so that a banker I know well kept getting flagged as a stranger. It had to learn that dragging a calendar event to a new day does not actually reschedule the meeting, because the other person's reminders follow the booking, not the calendar. At one point it held two different definitions of "available" and told a generous contact my week was full while my calendar sat wide open.
Not one of those fixes would help your company. You have your own garbled names, your own calendar habits, your own definition of available.
The last mile of agent reliability is made of facts about your business. And the last mile is the whole product.
One more finding from my log, and it is the one I would tattoo on every sales deck. When I audited the code I had written for future, generic customers, every single bug lived in the parts no real customer was using yet. The out-of-the-box settings are exactly the settings that nobody who built them lives with.
This is why the fleets will disappoint.
Your ERP, your CRM, the biggest software platform in your industry: all of them are about to sell you a fleet of agents that deploys in a day.
It took me 771 logged working sessions to make agents trustworthy for one company I know completely. A platform vendor cannot do that work for ten thousand customers. So they will ship the defaults. And the defaults are unproven.
Then there is the question of who catches the failures. Every important draft my agents produce crosses my desk before it moves. Over one 90-day stretch, I rejected 44 percent of them. Nearly half needed my hand.
That review step is not a weakness of my system. The review step is the product. A fleet without one does not have fewer failures than mine. It has invisible ones, going out under your company's name.
One last thing from the log. At one point I deleted over a thousand lines of my own safety code, because the data showed my automated checks were good at spotting problems and bad at fixing them. A check can see something is off. It cannot know which fix I want. Deleting that code was safe for exactly one reason: I personally review everything that matters. A vendor fleet has neither thing. Not checks tuned to your judgment, and not you in the loop.
Five questions to ask anyone selling you agents.
Ask these in the sales meeting and watch the room.
- When this agent fails, what does the failure look like, and who sees it? If the answer never mentions silent failures, they have not run agents in the real world.
- Where does a human approve important actions, and what did your customers reject last quarter? A vendor who cannot name a rejection rate has no review step.
- What record does every action leave? When an agent does something strange, "what did it do and why" must be answerable from a log. If the answer is "ask the agent," walk.
- Who watched the first live batch? Not the pilot. The first real workload. In my system, nothing new runs its first real batch unwatched. Ask whose job that is in the fleet.
- What happens when the agent hits something it does not recognize? The only good answer is: it stops and asks. A system that guesses on strange input will guess in your name.
Every one of these questions is a scar in my log.
Own your agents.
Owning your agents does not mean building everything yourself. It means the agents run on your company's facts. The checks encode your judgment, not a vendor's average customer. The records are yours to read. The person who distrusts the green lights works for you.
The real comparison is not build cost versus subscription cost. It is this: someone reviews your agents' judgment, or no one does. If someone does, you want that person inside your walls. If no one does, go back and reread the story of the warm, empty, perfectly signed email, and picture it landing in your biggest customer's inbox.
After 95 days of receipts, here is where I have landed. I trust my agents more than I ever have. Not because they stopped failing, but because their failures now land where I can see them, every action leaves a record, and every hard lesson became a permanent rule. Trust in an agent system is built the same way it is built with people: visibility, accountability, time.
No one can ship you that in a fleet.
The vendors will sell you the agent. Nobody can sell you the every time.
