Start as a delivered evaluation and reliability service for AI teams building agents. The first engagement maps one agent workflow, creates a task-specific eval suite, instruments traces, defines quality metrics, and delivers a repeatable testing pipeline plus weekly failure reports. Over time, the repeatable pieces can become templates, managed eval infrastructure, and eventually software for agent regression testing and reliability operations.

The job signals show a broad shift from generic LLM^? applications toward production agent systems with tool use, memory, RAG^?, planning, and closed-loop evaluation. Many companies are hiring this capability internally, which suggests urgent demand and a shortage of proven operating patterns.