Before an Internal AI Assistant Answers: Check Access and Sources

UDT Insights

From UDT’s knowledge archive: RAG learning materials, August 2026. Newly edited for publication.

Start with the questions the assistant may answer

An internal assistant should not turn every document into an answer for every employee. Before a pilot, decide which questions it supports, which sources it may use, and which people may access those sources.

This article proposes readiness checks for a pilot. The underlying learning materials discuss retrieval-augmented generation, or RAG. RAG retrieves relevant material and supplies it as context for a generated answer.

Access checks must precede answer generation

OWASP recommends permission-aware stores and access partitioning for vector and embedding data.[3] Treat document access as a system responsibility, not a request that the language model can choose to follow.

Our recommended design checks the current user and permitted documents before retrieved text reaches the model. Apply the same boundary to answer snippets, source links, and reused responses. A restricted title can disclose information even without the full document.

A filter for harmful questions serves a different purpose. It does not establish whether a person may read a particular document. Test access changes explicitly, including a user who loses permission after an earlier successful query.

Prepare the sources before the demonstration

Create a small, approved collection first. Ask the document owners to confirm what is authoritative and what needs correction.

• Scope: choose supported questions and exclude topics the pilot cannot responsibly answer.
• Ownership: identify the source owner, review status, and treatment of conflicting documents.
• Freshness: define how updates, withdrawals, and access changes reach the retrieval system.
• Traceability: preserve document identity, relevant passage, and version information for review.
• Answer rules: separate supported statements from uncertainty and request clarification when evidence is insufficient.
• Evaluation: test permitted answers, denied access, missing evidence, outdated sources, and misleading retrieved text.

Check the answer, not just the search result

Evaluate retrieval and generation separately. Did the system find the relevant permitted passage? Does the answer stay within that passage? Does it address the actual question? Does it agree with the approved reference answer?

A citation helps a reviewer inspect a statement. It does not prove the statement follows from the cited passage. Review the connection and show uncertainty when documents conflict. Keep human review for consequential decisions rather than treating fluent wording as authority.

A hypothetical example: an equipment request

This example is hypothetical, not a UDT implementation. An employee asks who approves a request for shared equipment.

The assistant finds an approved guide within that employee’s access. It answers with the relevant passage and a source link. It does not include a restricted planning note merely because the note shares the same keywords.

Later, the guide changes but the index still contains the old approver. The test should expose that mismatch. Until the source update is confirmed, the assistant should report uncertainty and direct the employee to the responsible owner, not invent an answer.

A four-step readiness flow

Use this sequence as a pilot acceptance check before expanding the document collection.

• Select: approve the questions, document owners, and source collection.
• Authorize: enforce the user’s current access before retrieval reaches generation.
• Answer: use supported passages and make the evidence inspectable.
• Review: test correctness, access changes, and source freshness before expansion.