What five hundred assessments revealed about hiring AI engineers

In the first day of the DevMesh beta, one AI-focused role drew more than two thousand candidates, and we completed over five hundred assessments in twenty-four hours. We were not trying to rank people. We were testing whether the things everyone screens for actually predict the thing everyone cares about. They did not. Three findings held across the entire cohort.

Almost nobody verifies what they ship

The most consistent failure was tests. Across every level of experience, from fresh graduates to working engineers, the pattern was identical: everyone was happy to ship with AI, and almost nobody was willing to verify what they had shipped. When a model writes most of the code, checking it is not a nicety. It is the job, and it is the part most people skipped.

Experience did not predict performance

Years on a resume told us nothing about how someone scored. Not a weak signal, a missing one. The skills that separate a strong AI engineer from a weak one are new enough that seniority earned before them does not transfer cleanly. The proxy the industry leans on hardest turned out to carry the least information.

Nobody was excellent, and that is the story

Most candidates landed between 4.6 and 6.6 out of ten. The best score in the entire cohort was 8.3. Not one person cracked nine. A distribution with no top end does not mean the people are weak. It means the skill is not being taught, or even named, so almost everyone is self-taught and roughly halfway there.

Before you can raise the bar, you have to be able to see where it currently sits. Most hiring pipelines cannot.

Redomic Labs

There is a hopeful reading here and a hard one. The hopeful one: the ceiling is wide open, because no one is near it yet. The hard one: most pipelines are measuring the wrong things, confidently. Closing that gap is the entire reason DevMesh exists.