The Lab
The Lab studies how AI systems behave, learn, and hold up in production - measured, not assumed. What we learn here feeds every product and every Studio build.
Everything here is our own work in progress, labelled by where it stands; the finished and announced record lives on the blog.
An agent that runs inside a workflow can generate a lot of visible activity without moving the work forward. This note is about the gap between activity and help, and how we are trying to measure the difference rather than assume it.
Trusting an agent once is a demo. Trusting it on day two hundred, through drift, quiet regressions, and failures that do not announce themselves, is the actual problem. This note collects what we watch for and what we still cannot see.
A system can arrive at the right answer for reasons that will not hold the next time the problem shifts. This planned paper is about separating genuine reasoning from surface pattern-matching that happens to look the same on a familiar test.
Training can produce a better system without anyone being able to say why, or repeat it. This planned paper is about the reproducibility of improvement itself: whether a gain came from the change we made, and whether making that change again would produce it again.
If an agent is doing a real chunk of the work, it needs the things any team member needs: a defined role, clean handoffs, and a trail of what it did and why. This note, circulated inside Redomic, sets out how we are trying to design for that.