Prompt Regression Testing

Prompt regression testing is the practice of re-running a fixed set of evaluation cases against an AI agent every time a prompt, model, or tool configuration changes, to catch quality or safety regressions before they reach users.

It is the agent equivalent of unit tests for code: fast, automated, and run on every change.

Closely related concepts include Continuous Integration Agent, Evaluation Harness (Agent), Ground Truth Evaluation, Regression Testing (Agent). Each is defined in the Prefactor glossary.