AutoSynthData Builds Training Data for Enterprise Agents
ServiceNow CoreAI has introduced AutoSynthData, a system designed to create training data for agents operating in enterprise environments. The approach focuses on weaknesses that arise from specific tools, workflows, policies and data states, rather than relying only on general-purpose capabilities.
AutoSynthData evaluates a target model on diagnostic tasks and compares its performance with that of a stronger teacher. It summarizes the resulting capability gaps in specification cards, then generates new tasks with varied prompts, entities, initial states, tool combinations and solution paths. Each task includes a system specification, a user request and a verifier.
Candidates are checked for feasibility, realism and difficulty before acceptance. The pipeline executes reference solutions, tests whether incorrect outcomes are rejected, and sends failed samples through critique and limited repair. Accepted examples can be used for supervised fine-tuning, while a separate multiplication phase creates additional variants. Batch-level review monitors coverage, diversity and repetition, directing later generation toward capabilities the model still struggles with. The released EnterpriseOps Gym dataset is used to illustrate the pipeline.
Written by Sakhanda Wire AI from reporting by Hugging Face. Facts, quotes and full context are in the original article below.