Agentic AI is rewriting how software ships. Assistants draft pull requests, agents open migrations, and review bots claim a change is safe before a human finishes coffee. The industry often frames this as a path to fewer people in quality. That framing is backward. When machines author more of the codebase, the scarce skill is not generation. It is independent verification.
The 2025 Stack Overflow Developer Survey found that 84% of respondents are using or planning to use AI tools in development, and 51% of professional developers use them daily. Positive sentiment fell to 60%, and more developers actively distrust AI accuracy (46%) than trust it (33%). The top frustration, cited by 66%, was solutions that are almost right but not quite. Google Cloud’s 2025 DORA research reported AI adoption at 90% among software development professionals, with over 80% saying AI improved productivity, yet only 24% reported a great deal or a lot of trust. Usefulness without trust is exactly the condition that expands the tester’s job, not shrinks it.
Speed Is Not Proof
I work as a Software Quality Engineer focused on agentic testing and CI gates for AI-authored code. Internally, we shipped a PR reviewer and a test case generator. Both are draft tools. Neither grades its own homework. That rule matters because agents speak with certainty. Certainty is not correctness.
We learned that the expensive way. In an AI-built PR reviewer pilot, one change cleared CI in under 4 minutes. Two days later, a downstream dashboard showed impossible temperatures. A Celsius-to-Fahrenheit scaling flip lived on one code path the suite never exercised. The pipeline was green. The product was wrong. Green CI that never asked the hard question is theater.
That incident sits next to a broader pattern. CodeRabbit’s State of AI vs Human Code Generation Report analyzed 470 open-source GitHub pull requests (320 AI-co-authored, 150 human-only). AI-authored changes produced about 1.7× more issues overall, with logic and correctness problems 75% more common and security findings up to 2.74× higher. Veracode’s 2026 GenAI Code Security Report tested models across security tasks and found an average security pass rate of 56%, barely changed from 55% in the prior report, while roughly 44% of AI code generation tasks introduced a known vulnerability. Syntax is nearly solved. Secure, correct behavior is not. Testers are the people who refuse to confuse those two.
Hallucinated Dependencies Are a Testing Problem
Generation also invents supply. USENIX Security 2025 research, We Have a Package for You!, evaluated package hallucinations across 16 code-generating LLMs and 576,000 samples. Commercial models hallucinated packages at least 5.2% of the time on average; open-source models averaged 21.7%, with 205,474 unique hallucinated names. Persistent phantom packages create a ready target for slop squatting: attackers register the invented name and wait for a developer or agent to install it.
Dependency provenance checks belong in the same tier as functional tests. If an agent invents a NuGet or npm package, the merge should fail before anyone debates whether the unit tests were thorough. Provenance is not optional hygiene. It is a quality gate for AI-authored change.
Gates That Scale With Agents
When we ran an org-wide .NET 8 to .NET 10 upgrade in about two weeks with AI-assisted automation, I owned the quality gates. Speed was real. So was the need for checks that do not believe the agent’s summary. The gates I run on changed code include:
- Mutation score targets in the 60% to 70% range on touched logic, so the suite has to kill real mutants rather than only cover lines
- Property-based tests on high-risk logic, where examples alone under-specify behavior
- Dependency provenance checks against slop squatting and hallucinated packages
- Duplication and churn signals that flag copy-paste thrash from agents
- Independent verification of agent claims such as “migration complete,” because a claim is a hypothesis until evidence lands
Mutation score is not a vanity metric. It asks whether your tests notice a deliberate fault. Coverage can rise while the suite still misses the Celsius path. Property-based checks force the model of the system to hold across ranges, not one happy path. Provenance closes the package pantry. Churn and duplication catch the agent habit of rewriting the same idea three ways. Together they form a posture: assume the agent is wrong until the suite proves otherwise. I keep that calibration line in review.
None of this replaces API testing, auth testing, smoke testing, or UI automation. FlaUI and PowerShell still catch what APIs miss. Packaging through NuGet, MSI, and MSM still needs install and silent-path validation. Agentic workflows amplify volume. They do not retire those disciplines.
More Testers, Different Work
The fear is that agents write the tests and humans become optional. Agents can draft cases. They should not certify their own drafts. Stack Overflow respondents planning future AI use for testing still put human distrust near the center: when they would ask another person for help, the top reason was not trusting AI’s answers. That instinct is the profession’s asset.
Hiring and enabling people who came through nontraditional paths matters here. I switched into quality after an HR ops and data career, completed an MS in Information Systems at UMBC in May 2024, and now spend my days making agentic gates practical. Agentic AI will not wait for a single pedigree. It needs more people who can design evidence, not fewer people who rubber-stamp green builds.
The product question is simple. If agents ship faster than your verification can think, you do not have a productivity win. You have a deferred incident. Invest in testers who treat every agent claim as suspect, who wire mutation and provenance into CI, and who keep independent judgment where the model is most fluent and least reliable. Agentic AI needs more testers, not fewer.
Credit: Source link
