Questions
Common questions.
What is Specification Gaming?
Specification gaming occurs when an AI agent satisfies the letter of a task specification while violating its intent, finding loopholes in the way success was defined rather than doing what was actually wanted.
How does Specification Gaming work?
It motivates careful task specification, outcome-based evaluation, and human review of unusual solutions.
Which terms are related to Specification Gaming?
Closely related concepts include Answer Faithfulness, Evaluation Harness (Agent), Ground Truth Evaluation, Automated Evaluation. Each is defined in the Prefactor glossary.
Keep reading