Choose your first AI pilot: one workflow, a clear test

Start with a task, not a chatbot

A useful first AI pilot tests a specific business question. Can a draft assistant help staff answer routine requests without increasing review work? That question defines users, inputs and an outcome. A request to add AI everywhere does not.

List recurring tasks with the people who perform them. Favor work with clear inputs, an available reviewer and a practical manual fallback. Avoid a first pilot that autonomously approves payments or makes decisions about people's rights.

• Name the task and the person accountable for it.
• Record today's effort, common mistakes and review steps.
• Choose one workflow that can run without AI if necessary.

Check data before choosing a model

Identify which records the pilot needs and who may access them. Confirm permission before sending business or personal information to an external service. Review retention terms and access controls with the responsible owner.

Create a test set that includes typical work, unclear requests and cases the system should refuse. Keep a separate evaluation set outside prompt development. Remove unnecessary personal details, and document where each example came from.

• Identify an owner for access and deletion decisions.
• Exclude data that lacks a clear permission basis.
• Define when the assistant must stop and ask a person.

Agree on the test before the demo

Write acceptance criteria before testing. Compare the AI-assisted workflow with the current process on comparable tasks. Count total effort, including review and corrections, rather than generation time alone.

For a draft assistant, check factual accuracy, missing information and whether reviewers can identify the supporting source. Track rejected drafts and requests that need escalation. Set your own limits for cost and response time based on the workflow.

• Have a domain reviewer assess outputs against a written rubric.
• Record failures as well as successful examples.
• Separate quality, review effort, cost and risk in the results.

Make a go, revise or stop decision

Give the pilot a fixed scope and a review date. Start with selected users, keep human approval and retain the manual process. Assign someone to inspect failures and changes in output quality.

At review, expand only when the agreed evidence supports expansion. Revise a weak workflow or stop a pilot that adds cost without useful value. NIST's voluntary AI Risk Management Framework offers broader guidance on risk across design, use and evaluation. This checklist is a starting point, not a claim of compliance.

• Document the decision and remaining limitations.
• Name the owner of monitoring and the fallback process.
• Retest after material changes to the model, prompt or data.