Companies tend to pick their first agent job the same way. Somebody asks what AI could do for us, and the room converges on the work everyone can picture: write our blog posts, answer our customers, handle the inbox. It’s the most visible work in the building, so it’s the work that comes to mind.
Six months later the pilot is quietly parked and the conclusion is that the technology wasn’t ready. The technology was fine. The job selection was the problem, and visibility is what made it a bad one.
Three things that make a job hard, and the visible ones have all three
A job is easy for an agent when there’s a checkable right answer, when a bad output is cheap, and when somebody in the building already does the job and can tell good from bad on sight.
Customer-facing writing fails the first test.
There is no right answer to “is this blog post good,” only a preference held by a person who will recognize the wrong version instantly and struggle to specify the right one in advance. That gap is where most quality complaints actually live.
Customer-facing anything fails the second test.
A bad internal summary costs somebody four minutes. A bad reply to a customer costs a relationship, and occasionally a compliance conversation. The cost of a failure sets how much supervision the job needs, and supervision is the expensive part of running agents.
And the visible jobs usually fail the third test in a way nobody notices until late.
The person who would judge the output is a senior person whose attention is the scarcest thing in the company. Handing them a review queue involves moving work onto your most expensive calendar.
What the successful first jobs look like
They’re boring. That’s the pattern, and it’s consistent enough to plan around.
When NTT DATA Group expanded its agent tooling across the organization, one of the early wins that shifted internal opinion was an internal engineering job, not a customer experience. OpenAI’s account of the deployment describes the automation of a complex incident analysis for a critical system — work that had previously required five experienced engineers and taken three days, completed in 30 minutes. That result, the write-up says, “quickly gained attention from senior leaders and became an early proof point.”
Look at the shape of that job rather than the headline number. Incident analysis has a right answer, and it’s an answer the organization can check, because the work already existed and the people who used to do it are still there.
It repeats. It sits inside the company’s own systems rather than in front of a customer. And the standard for “done” is whether the analysis holds up against the evidence, which is a question with a defensible answer, not a matter of taste.
Compare that to the job most companies nominate first. A blog post has no right answer, no in-house judge whose time is cheap, and an audience.
The sequencing matters as much as the pick. NTT DATA rolled out ChatGPT Enterprise across the company first and, in an internal survey, more than 96% of respondents said they were satisfied with it and more than 95% reported productivity gains.
Through that everyday use, the write-up says, “employees built experience working with AI for research, writing, and content creation” — and those habits “prepared the organization for the next step: delegating clearly defined tasks.”
The leadership lessons published with the case study lead with exactly that: make AI part of daily work, help people build the habits to collaborate with it. The company built a broad base of familiarity with AI output before the incident-analysis work landed.
That habit-building phase is doing more work than it looks like it is. When general AI tooling becomes ordinary, people start applying it well outside the boundaries of their job titles, and my read is that this is where the good candidate jobs surface — from the people doing the work, not from a planning meeting. It’s also where skepticism gets worked out, and how frontline staff actually feel about the tools is not something a single high-stakes project resolves.
The exception proves the rule, if you read the whole case
The obvious objection is that customer-facing agents demonstrably work. They do. It’s worth reading one closely.
The shopping agent avatarin built with Yamada Holdings is about as visible as an agent job gets — 24/7 multilingual support by voice and text, guiding shoppers from product discovery to purchase decisions. In a two-week public campaign on Yamada Denki’s online store, roughly 30,000 people used it and 92% of survey responses were positive. Real result, real customers, front of house.
Worth being straight about one thing: OpenAI’s write-up calls this “its first major opportunity to bring those lessons directly to customers.” So this was avatarin’s first customer-facing agent, which reads like an argument against everything above.
Look at what “those lessons” refers to. avatarin is an AI customer service company spun out of ANA Holdings, and the write-up notes it had worked with OpenAI “well before the Yamada Denki project, using the OpenAI API for speech recognition, inquiry analysis, and employee training.”
The agent’s answers are grounded by a retrieval-augmented generation system, which means responses are anchored to actual product information rather than the model’s memory. The retailer’s customer service knowledge was deliberately encoded into conversation flows and prompting. Guardrails help keep the conversation on the shopping experience. And OpenAI worked directly with the team to structure the prompts and bring down the cost of running an always-on voice service.
So it was a first customer-facing job for a company whose entire business is AI customer service, arriving after prior work with the same vendor’s tools, with a grounding layer, encoded domain expertise, guardrails, and the vendor’s own people helping. That’s the real price of the visible job, and almost none of it is visible in the result.
Read the two accounts together and they’re a sequencing lesson rather than two success stories. One organization earned the right to delegate defined work by making AI ordinary first. The other reached the customer-facing job as a specialist, on the back of component work it had already done. Neither one started where the planning meeting wants to start.
The selection rule
Score every candidate job on four questions before you pick.
1. Is there a checkable right answer?
Not a good answer, a right one. Reconciliation, classification, extraction, and analysis-with-a-conclusion all qualify. Anything judged on taste does not, at least not first.
2. What does one bad output cost?
If the honest answer involves a customer, a regulator, or a number in a filing, the job needs the kind of architecture and review rails that you almost certainly haven’t built yet.
3. Does volume justify the setup?
A job done twice a month will never repay the weeks of iteration. The economics come from repetition, and repetition is also what teaches you where the thing fails.
4. Who already does this and can spot a bad result instantly?
That person is your reviewer, and they need to have the time. If the only qualified judge is the founder, pick a different job.
The work that scores well on all four is almost never the work that came up in the meeting. It’s usually something a team has been doing manually for two years and has stopped complaining about, because they no longer notice it as work.
Where this leaves you
Take the list of jobs you’re considering and score them honestly this week. The candidate that wins will feel like a disappointment. Automate it anyway.
The point of a first agent job isn’t the value it returns. It’s that your organization learns what these systems do well, where they break, and what supervising them actually costs, on work where being wrong is survivable. That knowledge is what makes the second job a real decision instead of a guess, and the third one is usually where the money is.
Pick the boring job. Earn the visible one.
Credit: Source link
