Start as a managed evaluation operations service for AI product and engineering teams. The first engagement creates a use-case-specific eval pack: golden tasks, labeled examples, acceptance criteria, scoring rubrics, baseline runs, error taxonomy, and a launch-readiness report. Ongoing retainers run weekly or per-release eval cycles, coordinate expert reviewers, maintain datasets, monitor metric drift, and translate results into prioritized product fixes. Lightweight software can emerge later as internal tooling for intake, result history, reviewer QA^?, and release gates.

AI products are moving from demos into production workflows, but evaluation practices are lagging behind model deployment speed. The job signals show companies are turning evaluation into a dedicated operating function across product, engineering, research, and operations.