Agentic testing is testing where an AI agent does the testing work in a loop: it plans a test, runs it, reads the result and decides what to do next, such as writing another test or updating one that broke. The difference from earlier AI testing is autonomy across many steps, not a smarter way to find one button. That autonomy is useful only when the agent works inside a format a human can read and a gate a human controls.
An agent runs a loop, a script runs a list
A test script is a fixed list of instructions. It does the same thing every time, and it stops when something unexpected happens.
An agent (a model that plans, acts, observes the outcome and chooses its next action) works differently. Given a goal such as "cover the checkout flow", it can explore the page, write steps, run them, see which step failed and try again. In testing, agents usually do three things:
- Write tests from a description, a code change or an existing flow.
- Run tests and read the results, including screenshots and logs.
- Update tests when the interface changes, or report that the product itself changed.
The loop is what makes agents productive. It is also what makes them risky. An agent that keeps trying until a test passes can end up changing the test to match a bug.
Coding agents made agentic testing necessary
Agentic testing grew because agentic coding grew. In the JetBrains developer ecosystem survey, 85% of developers regularly use AI tools for coding and development, and 62% rely on at least one AI coding assistant, agent, or code editor [JetBrains]. Qodo reports that 82% of developers use AI coding tools daily or weekly [Qodo].
When an agent writes a feature in minutes, a human writing end-to-end (E2E) tests for it over days becomes the bottleneck. Either tests fall behind, or review slows everything down. The second problem is described in the code review bottleneck with coding agents. Letting the same class of tool write and run tests is the obvious response. The question is how to keep it honest.
The spec file is the contract between agent and team
Our recommendation: let agents write tests only in a format that a reviewer can read in a pull request diff. Plain-English steps in a versioned file meet that bar. Generated test code in a general-purpose language often does not, because a reviewer has to trace logic to know what is being checked.
A readable spec makes three things possible. A reviewer can see what the agent intends to test. A second agent can pick up the file and extend it. And a failure points to a step a person understands, not to line forty of a helper function.
1name: Sign up with email2url: https://app.example.com3tags: [signup, smoke]4steps:5 - Click "Sign up"6 - Fill in the email field with a new address7 - Click "Create account"8 - assert: The welcome page shows the new account emailMore on shaping the suite for agents is in a testing strategy for AI coding agents.
Agents should not grade their own work
The agent that wrote a change should not be the one that decides a failing test is wrong. Keep two rules in place.
First, the merge gate belongs to the team. On GitHub, once required status checks are enabled, all required status checks must pass before collaborators can merge changes into the protected branch [GitHub]. Make the E2E run one of those checks, and do not let an agent change the rule.
Second, separate healing (re-finding an element after its locator changed) from behaviour changes. A renamed button can be healed automatically. A different order total cannot. When behaviour changes, the test fails and a person decides whether the product or the test is wrong. See the CI docs for how a run reports to a pull request.
Agentic testing has limits a loop cannot remove
An agent can only test what it can observe and what it was asked to cover. It does not prove that an application has no bugs. It does not replace exploratory testing, where a person follows curiosity, or accessibility and security review, which need specialist judgment. Those limits are listed on what we cannot do.
Agents are also only as good as their goal. A vague goal produces vague tests. Give the agent a concrete flow, a clear expected outcome and a place to report what it could not resolve.
How AI Testing Platform works with agents
AI Testing Platform gives agents a fixed loop: run npx aitp init, read tests/*.test.yaml, write new specs in the same format, run npx aitp run, and read the report at .aitp/last-run.json. The platform heals locator and layout changes and fails the test on behaviour changes, so an agent cannot quietly accept a regression. Setup details are on the for agents page.
npx aitp initnpx aitp run