Achieving 12x faster test authoring with Al QA agents
.png)
5 min
A full QA suite written, against an hour by hand
22 hrs
Back on a QA engineer's calendar every week
~$64k
Capacity returned per engineer, per year
Every ticket starts with a specification and someone reading it. A user story, a design frame, a pull request. Whoever picks it up works out what the change touches elsewhere, thinks through the edge cases, and writes the cases out in a form the rest of the team can act on.
On anything non-trivial that is close to an hour of work, and it happens before a single test has been executed. Across a team, several people digest the same document separately and reach slightly different answers, which is where requirements go missing.
QA flow does that first pass. It reads whatever spec exists, writes the cases a QA engineer would write, runs them unattended, and files what breaks. Five minutes instead of an hour, with one engineer reviewing a complete draft rather than starting from a blank page.
The challenge
Every ticket runs the same loop before anything gets tested, and almost none of it is creative. It is careful, necessary work, and it takes most of the week.
Authoring from a blank page
Read the user story, work out what the change touches elsewhere, think through the edge cases, then write the cases out in a form the team can act on. On anything non-trivial that is close to an hour, and it happens before a single test has been executed.
Execution with someone present throughout
Open the case, read step one, go do it, come back, read step two. Roughly eight minutes per test case, with attention on it the whole time.
Writing the failure up a second time
Anything that fails becomes a bug report, which is a second pass over work already done once, in a different format for a different reader.
The judgment goes last
Edge cases, impact analysis, the scenario nobody thought to write down. That work is what a QA engineer is actually for, and it gets squeezed because authoring and execution fill the day first.
QA flow runs the front half of the ticket. It reads whatever spec exists, writes the cases a QA engineer would write, runs them unattended, and files what breaks. Three things stay with the engineer: approving the suite, adding the edge cases only they would think of, and fixing the bug. Everything between those decisions happens without anyone watching.
What we built
QA flow runs the front half of the ticket. It reads whatever spec exists, writes the cases a QA engineer would write, runs them unattended, and files what breaks. Three things stay with the engineer: approving the suite, adding the edge cases only they would think of, and fixing the bug. Everything between those decisions happens without anyone watching.
It was built in-house rather than for a client, by a four-person pod working full time for six months across the front end, the back end, and the agent itself. The target was the part of the role that does not need a person present, so that the part that does gets the time back.
.png)
Writes the tests from whatever spec exists
The type of test is a choice too. Take the blend a QA engineer would assemble, or name the one you want, from end to end and integration down to unit, smoke, and accessibility. Review starts from a complete draft rather than a blank page.
.png)
Runs them without anyone watching
One place still wants a person. A case that signs in as a particular user needs those steps edited first, because the run picks up one set of credentials rather than choosing between several. It is the one part of the flow that stayed manual.
.png)
Shows what failed, and what fails most
There is no execution report. A list of the cases goes out once they are written, and the dashboard covers the rest. Ticket formatting got its own pass during the build, from a bare first version to something carrying enough context that the next person does not have to come back and ask.
Getting the agent accurate took longer than building it
So false positives and false negatives went into a shared spreadsheet, run by run, and the model was tuned against that record until the rate came down, then moved to a newer version. That accounting is where most of the six months went. It is also the part that never shows up in a demo, which is why the accuracy work is the reason this one is worth writing about.
.svg)
Three stages of manual work come off the ticket, and the three decisions worth making stay. What comes back is about twenty two hours a week per engineer, spent on testing that never fit before.
Want this pipeline running on your backlog? Get in touch.
Conclusion
Three stages of manual work come off the ticket, and the three decisions worth making stay. Authoring drops from about an hour per feature to five minutes. Execution drops from roughly eight minutes per test case with an engineer watching to a three-minute run that needs nobody. Writing up the failure stops being a separate pass.
What comes back is about twenty two hours a week per engineer, spent on testing that never fit before. The suite that gets written first is the one any competent QA engineer would have written. What the engineer does with the hour they got back is the part a tool was never going to do.
Want this pipeline running on your backlog? Get in touch.
5 min
A full QA suite written, against an hour by hand
22 hrs
Back on a QA engineer's calendar every week
~$64k
Capacity returned per engineer, per year
Spec sources: Jira, Linear, GitHub repositories, screenshots.
Test types: end-to-end, integration, unit, smoke, accessibility.
Agent: fine-tuned model, false positive and negative rates tracked run by run across six months.
Output: generated test cases, unattended execution, automatic bug tickets, results dashboard.
Build: four-person pod, full time, six months across front end, back end and agent.
.svg)
.svg)
.svg)
Read more
.png)
.png)
.png)
You’re all set!
.svg)
