Evaluate AI answers with a task-specific reference set

A generic benchmark cannot tell a purchasing team whether an assistant reads its approval policy correctly. A task-specific reference set links evaluation to the decisions users actually face. It should describe what a satisfactory answer must contain, what evidence supports it, and when the assistant should ask for more information instead.

Describe the task before collecting questions

Choose a bounded activity, such as explaining purchase approval requirements. List the user roles, available documents, and information the assistant may access. NIST’s AI Risk Management Framework calls for documented test sets and metrics, and evaluation under conditions similar to deployment.[7] The practical design below applies that principle to document answers.

Collect questions from the workflow, not only from the implementation team. Include incomplete requests, conflicting references, and questions outside the approved collection. Remove unnecessary personal information. Preserve the distinctions that make the task difficult, such as different approval rules for different purchase categories.

Write reference criteria, not a magic sentence

For each case, save the question, relevant source passage, required answer elements, and prohibited conclusions. Accept equivalent wording when the meaning remains correct. Exact wording is useful for identifiers or amounts, but it should not become the only grading method for an explanation.

Separate retrieval checks from answer checks. Reviewers should know whether the system failed to find the evidence or misrepresented evidence it already found. Mark a confident unsupported answer differently from a cautious request for clarification. Agree on these labels before comparing models or prompt revisions.

Create a usable scoring sheet

Use a short rubric that reviewers can apply consistently. Ask two reviewers to score ambiguous cases independently, then discuss disagreements and refine the guidance. This is a recommended process, not a claim that agreement alone establishes correctness.

• Require the applicable rule and its material exceptions.
• Check that each citation supports the statement beside it.
• Include cases where missing evidence requires an explicit limitation.
• Keep development examples separate from held-out acceptance cases.
• Record error severity and reviewer reasoning, not just a combined score.

Hypothetical example: a purchase assistant

A hypothetical manufacturer tests an assistant that explains approval routes for equipment requests. One case omits whether the equipment is leased or purchased. The reference expects a clarification because the supplied policy distinguishes those categories.

An answer that confidently names an approver fails that case even if its tone sounds professional. Another answer passes if it asks the necessary question and cites the relevant distinction. The test evaluates a defined behavior. It does not establish that the assistant can approve purchases or handle every procurement exception.

Use results to choose the next change

Review failures by task type before accepting a release. Repair document coverage when evidence is missing, and revise answer behavior when supported rules are misstated. A small reference set can miss rare situations. Expand it with reviewed incidents, while retaining stable cases that reveal regressions across successive changes.