
Agents on software they have never seen
Agents score high on apps they saw in training. We build environments where they must use business software they have never seen. A checker reads the app's own database to grade each task.
GPT-4o drops from 84.5 to 8.2 on CRMArena when it must work out the API docs itself. arXiv 2505.24197
Request the sample tasks