Quality review for every change, from engineers or AI coding agents
I’m Arthur Flam. I started QA-Board, the open-source quality tracking platform, in 2018. I set it up for teams on their own machines and connect their projects, so that every change is reviewed with its outputs and metrics next to the reference’s, whoever wrote it.
What tests and averages miss
Six regressions from QA-Board’s public demo projects, which use made-up data.
Test suites
Green CI, half the tests gone
Asked to fix CI, a coding agent deleted the failing tests. The build is green; the comparison with main shows 12 tests fewer and 2 newly skipped.

Latency
Less CPU, and seconds of tail latency
A shared cache written by a coding agent saves CPU. Under the same load, p99 latency goes to 1.5 seconds, 26 times the reference’s.

LLM agents
The assistant stopped asking before acting
A coding agent shortened the assistant’s tool descriptions and turned off a confirmation check. Now the assistant reboots a robot without asking, even after the operator said no.
The demo project: The assistant stopped asking before acting

ML models
Smaller model, people missing
An int8 detector is 72% smaller than the release model and loses 35% of its mAP on indoor scenes, which the overall average hides.

Robotics
The robot thinks it’s somewhere else
New loop closures match identical shelf bays in a warehouse, and one false match pulls the robot’s estimated trajectory away from the ground truth.

Imaging
The metric improved, the image got worse
A new denoiser wins 1.6 dB of PSNR and draws dark halos around every leaf. Drag the divider to compare with the fix.


Reviewing changes from AI coding agents
In three of these cases, a coding agent wrote the change. With QA-Board, the agent runs your test set and reads the comparison with the reference itself, so it can fix what it broke before anyone reviews it. A quality gate in CI, experimental for now, fails the job and comments on the pull request. Merging stays a person’s decision.
Services
- A first project, end to end
- A server on your machines and one project connected to it, running in CI. You judge it on your own data.
- More projects and tools
- Runners for LSF, Slurm, Kubernetes or Celery, the quality gate on pull requests, viewers for your file formats.
- Running the server
- Docker Compose or Helm, single sign-on, backups, upgrades, networks without internet access.
- Training, reviews and coding agent setup
- For the people who review results, the engineers who integrate projects, and the agents that check their own changes.
How a first project goes
- A call about what you measure and where your code runs. I follow up with a written scope.
- I set up the server and the first project with someone from your team. From then on it runs in CI on every push.
- Your team reviews real changes for a few weeks, and we adjust what it shows to what they look at.
- You decide what comes next. The server and the integration are yours.
Who you’ll work with
Arthur Flam
Me, directly. My day job is platform engineering for research teams, which covers GPU clusters, LLM serving and the coding agents they use. More about me
Contact
Write a few lines about what you measure and how you review changes today.