AI Agent Audit Checklist

An AI agent audit checklist gives an independent reviewer a repeatable way to test whether agent controls worked during a defined period.

The review should go beyond reading policies and checking that logs exist. An auditor needs a stated objective, clear criteria, complete populations, defensible samples, evidence tied to each sample, and pass conditions written before testing starts.

This checklist is an audit workpaper for a defined scope. Start with one agent workflow in one environment. A fleet audit may cover a named segment of agents, workflows, and environments when the criteria, period, populations, and exclusions remain explicit. The procedures test design and operating effectiveness across ownership, identity, context, tools, data, releases, runtime evidence, incidents, and retirement.

It is Alignbase’s recommended baseline, not an audit standard, certification, or legal opinion. Tailor the criteria, sample sizes, severity rules, and required reviewers to your organization’s risk, duties, and assurance program.

AI Agent Audit Checklist at a Glance

Run these ten procedures:

1. Reconcile the agent inventory and ownership records.
2. Verify risk classification and control obligations.
3. Test identity, authorization, and delegation.
4. Test context, Skills, Memory, permissions, and routes.
5. Test tools, approvals, and external outcomes.
6. Test data, privacy, secrets, and retention.
7. Test releases, changes, and software supply chain controls.
8. Test runtime monitoring and point-in-time evidence.
9. Test incidents, suspension, recovery, and pending work.
10. Test retirement and residual access.

For each procedure, record the population, selected sample, evidence, test steps, result, exception, owner, and reviewer. A pass applies only to the scope and period tested.

Start With an Audit Plan

Write the plan before requesting evidence. This keeps the review from turning into a search for records that support a preferred result.

The plan should name:

• Audit objective
• Agent, workflow, environment, and legal entity in scope
• Audit period and evidence cutoff
• Control criteria and source
• Risk, authority, data, and autonomy profile
• In-scope systems, identities, tools, context sources, models, and integrations
• Populations and population owners
• Sampling methods and sample sizes
• Materiality and finding severity rules
• Auditor, technical specialists, and conflicts of interest
• Evidence access, handling, retention, and deletion rules
• Reporting date, recipients, and remediation process

The NIST SP 800-53A Rev. 5 assessment guide describes a customizable method for assessing controls and building assessment plans. Its examine, interview, and test methods are useful here: inspect records and configurations, ask responsible people how the process operates, then execute a test that proves or disproves the control.

NIST’s AI Risk Management Framework Core also calls for testing before deployment and during operation. Measure 1.3 recommends regular involvement from internal experts who were not front-line developers or from independent assessors. Independence should match the risk and assurance claim. A reviewer should not certify a control based only on their own implementation work.

Contain audit testing

Run adversarial, replay, credential, cross-tenant, suspension, and former-access tests only under written authorization in a disposable, isolated environment. Use synthetic data, inert integrations, scoped test identities, restricted egress, bounded resources, active monitoring, and verified cleanup.

A production check needs separate approval, a non-destructive method, explicit targets and time limits, and a tested stop path. Do not send a real high-impact action, expose a secret, access another tenant’s data, or reactivate retired authority to prove that a control blocks it.

Define the control criteria

Do not test against broad goals such as “the agent is secure” or “the agent follows policy.” Turn each requirement into an observable condition.

For example:

Requirement: High-impact actions require approval.

Criterion: Before execution, the trusted action service verifies an approval
record that contains the authenticated approver, decision, policy version, and
issuing authority. It rechecks current approver eligibility and revocation,
then atomically claims the unused, unexpired approval and creates a durable
execution record bound to the initiating principal, agent, tenant, run,
canonical operation, target, material parameters, and immutable operation ID.
Any mismatch, duplicate, or concurrent replay is denied.

The criterion tells the auditor what to inspect and what failure looks like. It also prevents a generic approval screenshot from passing a control that requires exact action binding.

Define complete populations

A sample means little when the population is incomplete. Build populations from systems that record actual use, not only the approved agent registry.

Relevant populations may include:

• Active and retired agents
• Agent workflows and environments
• Identities, credentials, access grants, and delivery routes
• Context bundles, Skill reads, and Memory writes
• Releases, configuration changes, and package updates
• Runs, tool calls, high-impact actions, and external effects
• Approvals, denials, exceptions, and policy bypass attempts
• Delegated tasks, child agents, queues, schedules, and callbacks
• Alerts, incidents, suspensions, recoveries, and retirements

Reconcile registry records against deployment systems, identity providers, tool gateways, MCP configurations, queues, schedulers, cloud accounts, and spend records. Record unresolved differences as exceptions instead of silently removing them from scope.

Copyable AI Agent Audit Workpaper

Use one record for each procedure and sample:

Audit objective:
Control criterion:
Scope and period:
Population source and count:
Selection method:
Sample ID:
Evidence references:
Examine steps and result:
Interview steps and result:
Test steps and result:
Result: Pass | Fail | Not tested | Not applicable
Exception and impact:
Finding severity:
Owner and due date:
Auditor and review date:
Retest evidence and result:

Not tested and Not applicable mean different things. A procedure is not tested when evidence, access, time, or scope prevents the work. It is not applicable only when the capability or condition does not exist, and the workpaper should identify the evidence and reviewer behind that conclusion.

1. Audit Inventory and Ownership

Objective: determine whether the organization knows which agents operate in scope and whether accountable humans own each workflow and Resource.

Evidence to inspect:

• Agent and workflow registry
• Deployment, identity, integration, queue, schedule, and spend inventories
• Business and technical ownership records
• Direct active human Owner access grants and active-owner enforcement
• Archive, restore, and ownership-transfer history

Test the registry against independent system sources. Select unregistered deployments that match the scoped workflow, environment, or fleet boundary as well as registered agents. For each sample, verify the purpose, environment, users, affected people, data, tools, integrations, autonomy, and current owners.

Pass when the registry population is reconciled, differences are resolved or reported, and each active agent and governed Resource has the human ownership required by policy. A team name, creator field, agent identity, or inactive user should not satisfy a requirement for a current accountable human.

2. Audit Risk and Obligations

Objective: determine whether the control set follows the agent’s actual reach and applicable duties.

Evidence to inspect:

• Approved use case and prohibited uses
• Risk and autonomy classification
• Data and human-impact assessment
• Legal, privacy, security, contractual, and sector obligation mapping
• Residual risk decisions and exceptions
• Material-change triggers

Compare the classification with the production workflow. Test whether its data, destinations, tools, external effects, reversibility, persistence, scheduling, delegation, and affected people match the approved record.

Pass when the classification is current, supported by production facts, approved by the named authority, and connected to specific control criteria. Report a finding when a low-risk label excludes a capability the agent actually uses.

3. Audit Identity, Authorization, and Delegation

Objective: determine whether every action is attributable and stays inside current, bounded authority.

Evidence to inspect:

• Agent and initiating-user identities
• Credential issuance, storage, rotation, revocation, and expiry records
• Authorization policy and enforcement configuration
• Positive and denied action traces
• Delegated task grants and child-agent records
• Cross-tenant and replay test results

Select ordinary actions, high-impact actions, denied attempts, expired authority, and delegated work. Reperform checks for operation, resource, tenant, environment, data class, time, spend, and target ownership. Test whether revocation blocks a queued or delayed action at execution time.

For delegation, verify that authority can only narrow. A task grant should bind issuer, subject, audience, tenant, run, allowed operations and resources, expiry, delegation depth, nonce, and revocation state. Shared parent credentials or prompt-supplied tenant claims should fail the procedure.

Pass when trusted credentials identify the principal and tenant, each consequential action rechecks current authority, delegation stays within the full authority intersection, and raw credentials remain outside prompts, context, Memory, tool results, and model output.

NIST’s software and AI agent identity and authorization concept paper describes agent access to data, tools, and applications as a reason to apply identification and authorization controls. The exact grant fields above are an implementation profile for making those boundaries testable.

4. Audit Context, Skills, Memory, Permissions, and Routes

Objective: determine whether the right versioned agent inputs reached the right session under authorized permission and delivery decisions.

Evidence to inspect:

• Context inventory, owners, trust labels, and version history
• Review and release records for stable instructions and Skill packages
• Memory versions, writers, and retention rules
• Repository roles and independent delivery routes
• Context assembly, host insertion, Skill availability, and Skill invocation records
• Conflict, injection, stale-write, and cross-tenant test results
• Filtered-Resource evidence-store, audit API, log, and export access tests

Sample context deliveries across agents, users, Groups, workflows, and environments. Include route additions and removals, mandatory delivery, optional delivery, permission changes, Skill package reads, Memory writes, and denied access.

Test repository permission and delivery as separate decisions. A route may deliver approved context without granting repository discovery or edit rights. Conversely, repository access should not create a delivery route. Reperform eligibility and tenant checks at bundle assembly.

Follow the evidence through each delivery stage. A server record can prove which released instructions, current Memory, and Skill metadata and digest were assembled. Trusted request, host, or session evidence should bind that bundle to the final serialized model request and protected instruction channel. Record the agent and session, message role, order, trust class, precedence, delimiters or escaping, truncation decision, and any host transformation. A package read or sync should prove that the advertised Skill package was available.

A published Skill package read should require repository access, a current Always route rechecked with tenant eligibility at read time, or an unforgeable, short-lived delivery grant bound to the authenticated initiating principal, receiving agent, audience, tenant, task or run, session, route or delivery event, Resource, published digest, expiry, and nonce. Reads of latest, pinned, exact-version, or draft content still require the applicable repository role. Derive identity and tenant from authenticated scope, not request parameters. Test unrelated agents, removed routes, revoked grants, changed identifiers, cross-principal and cross-session replay, expiry, and cross-tenant requests. If policy requires the Skill for the task, test invocation separately.

Run adversarial injection tests through retrieved documents, web pages, files, tool results, user messages, and agent-to-agent messages. Define unauthorized actions, disclosures, approval changes, route changes, cross-tenant access, and persistent writes as failures. Enforce high-impact policy at trusted boundaries outside the model.

Test whether filtered Resource identifiers, permission results, scope mismatches, counts, or reason classes appear in ordinary audit APIs, tenant-visible logs, agent output, or exports available to the requesting principal. Use repeated, adaptive requests that vary one possible Resource or scope at a time. Detailed and aggregate exclusion evidence should remain in an independently controlled, tenant-scoped store, and cross-tenant or ineligible requests should reveal no protected Resource existence or access topology.

For a positive delivery sample, pass when the governed Knowledge, Skill, or Memory has an owner, source, trust level, immutable or current version, authorized route, eligible recipient, and point-in-time delivery evidence. For a negative sample, pass when the system denies the read, write, route, or delivery for the expected reason without revealing protected Resource details. Required Resources cannot be removed by an unauthorized principal.

Transient inputs such as user messages, retrieved documents, web pages, files, and tool results need source and trust labels, isolation from privileged instructions, and enforced authorization before any sensitive action, disclosure, route change, or persistent write. Memory writes must recheck current write access and the exact latest version inside the transaction.

The point-in-time agent audit guide explains how to reconstruct one delivery and action chain. The AI agent audit logs guide lists the event fields needed to support that work.

5. Audit Tools, Approvals, and Outcomes

Objective: determine whether tool use stays inside policy and whether the evidence proves what happened outside the agent runtime.

Evidence to inspect:

• Tool registry, schemas, versions, and risk classifications
• Operation, parameter, destination, and data limits
• Approval policy and eligible approvers
• Approval and denial records
• Tool request, result, retry, and error traces
• System-of-record outcomes
• Sandbox, egress, and cleanup tests

Select reads, writes, high-impact actions, denied calls, errors, retries, changed targets, and approval replays. Reperform the policy decision using the sampled actor, tenant, run, operation, target, and parameters.

For approved actions, change one material field and verify that the approval no longer works. Send concurrent requests with the same approval and immutable operation ID, then confirm that the trusted action boundary atomically consumes the approval and executes at most once. Verify the approver’s authenticated identity, approval decision, policy version, current eligibility, revocation state, and any separation-of-duties rule at action time.

Test failures after the approval claim but before dispatch, during the target commit, and after the target commits but before the action service receives or records success. The approval claim and durable execution record should commit in one local transaction. The target should bind the canonical request and immutable operation ID to its idempotency record atomically with the external effect, then retain that record through the full retry and replay horizon.

For a claim known never to have been dispatched, recheck approval expiry, approver eligibility, initiating authority, policy version, suspension state, tenant, and target scope immediately before first dispatch. Cancel the claim and require fresh approval if any condition changed. If dispatch may have occurred, perform read-only outcome reconciliation before deciding whether a retry is safe.

Verify the result through a system-of-record event, audit API, or separate read identity. An agent response that says “completed” and an adapter response that says 200 do not prove that the intended external state changed correctly.

Pass when tool access is default-deny, high-impact approval comes from a currently eligible approver and binds to the exact action and immutable operation ID, the approval claim and execution record are atomic, target-side idempotency prevents duplicate external effects across the retry horizon, undispatched claims are reauthorized before dispatch, unknown outcomes use read-only reconciliation before retry, changed requests fail, external outcomes are checked independently, and risky execution is isolated from the host and unrelated credentials.

6. Audit Data, Privacy, Secrets, and Retention

Objective: determine whether agent data use matches approved purpose and whether sensitive material stays protected across the full workflow.

Evidence to inspect:

• Data inventory and flow map
• Purpose, collection, access, retention, residency, and deletion rules
• Encryption, key ownership, and tenant-isolation controls
• Prompt, output, cache, trace, backup, and training-data settings
• Redaction and secret-scanning results
• Data rights or deletion tests where applicable

Trace sampled data through prompts, retrieval, context, Memory, tool inputs and results, model providers, logs, caches, exports, and backups. Inspect both successful and failed requests because errors can disclose material that normal output filters remove.

Test whether secrets can enter model-visible input through environment variables, files, tool output, error messages, or generated artifacts. Confirm that a trusted service holds usable credentials. A trusted service should mint each model-visible handle as an opaque, high-entropy server-side reference or protect its claims with authenticated encryption or a signature. Reject altered or unverified claims before resolution.

The handle should bind the current authenticated principal, agent, tenant, session, resolver or tool audience, method, operation, target, Resource, material parameters, expiry, and nonce. The resolver should recheck current authorization and revocation, enforce single use when the action is not safely repeatable, and reject cross-tool, cross-destination, cross-principal, cross-session, post-revocation, claim-tampering, and replay attempts. Prefer non-exportable capabilities where the runtime supports them.

Pass when each use has an approved purpose and classification, access follows tenant and role, retention applies to every copy, verified deletion reaches required stores, evidence minimizes protected data, raw credentials remain outside model-visible surfaces, and any model-visible handle is integrity-protected, audience-bound, narrowly scoped, short-lived, revocable, and replay-resistant.

7. Audit Releases, Changes, and Supply Chain

Objective: determine whether production runs match reviewed artifacts and whether material changes reopen the right controls.

Evidence to inspect:

• Release manifest and approval
• Code, configuration, context, Skill, tool, model, runtime, and host-adapter identifiers
• Test results and acceptance limits
• Package inventory, source, signatures, digests, scans, and advisories
• Change assessments, staged rollout, rollback, and emergency-change records
• Revoked package and publisher tests

Select planned, emergency, failed, and rolled-back changes. Trace a production run to the complete reviewed artifact set. For live Memory, inspect the approved update policy and the version present during the run instead of treating Memory as fixed at release.

Match downloaded or synced Skill packages and other executable components to approved digests. Confirm that source, integrity, malware, vulnerability, and revocation policy ran after retrieval and before use.

Pass when a reviewer can identify the exact production release, all executable components meet admission policy, material changes trigger reassessment, and rollback restores a known configuration without reviving stale authority or pending work.

NIST SP 800-218 provides outcome-based secure software practices, including software provenance and supply-chain risk. Apply those practices to agent code, dependencies, Skills, MCP servers, plugins, models, containers, tool adapters, and integrations.

8. Audit Runtime Monitoring and Evidence

Objective: determine whether monitoring detects control failures and whether an independent reviewer can reconstruct important actions.

Evidence to inspect:

• Telemetry map and required event schema
• Alert rules, tests, acknowledgments, and resolution records
• Context delivery, authorization, approval, tool, outcome, and lifecycle events
• Append-only or tamper-evident evidence-store configuration
• Evidence access reviews, integrity tests, retention, and export results
• Control performance reviews and recurring-failure analysis

Select high-impact actions, normal runs, denied actions, failed tools, alerts, and changes across the audit period. Reconstruct the initiating principal, agent, tenant, task, run, context bundle, policy and Skill versions, Memory version, authority, approval, tool request and result, external outcome, and lifecycle state.

Inspect evidence-store administration. The agent and operators whose work is recorded should not be able to alter or silently delete their own evidence. Test an export and verify its stable identifiers, timestamps, hashes, digests, and source references.

Reconcile event counts and identifiers against independent systems of record, including the action service, target system, identity provider, queue, or scheduler where relevant. Test sequence gaps, duplicate IDs, ingestion lag, dropped-event alerts, and protected clock synchronization. Verify that high-impact execution fails closed when required evidence capture is unavailable.

Pass when each selected high-impact action can be reconstructed, event populations reconcile, ingestion failures and sequence gaps are detected, required alerts fire, reviewers act within policy, evidence integrity is protected by an independently controlled path, and the evidence contains only the protected data needed for assurance.

NIST’s AI RMF Playbook suggests maintaining histories and audit logs that support review, tracking policy exceptions and go or no-go decisions, and auditing whether risk-management mechanisms work. The Playbook is voluntary guidance, so use it as an input to your criteria rather than an automatic pass condition.

9. Audit Incidents, Suspension, and Recovery

Objective: determine whether the organization can detect agent failures, stop all related work, preserve evidence, contain effects, and recover from known state.

Evidence to inspect:

• Agent-specific incident plan and severity rules
• Escalation paths and current contacts
• Incident records and evidence holds
• Suspension and revocation controls
• Exercise results and measured stop times
• Recovery, reconciliation, and restart approvals

Select incidents and exercises across identity compromise, harmful output, unauthorized context, tool misuse, data exposure, persistent Memory corruption, and uncontrolled delegation where relevant.

Reperform the stop path in an authorized, isolated environment. Verify that suspension reaches active sessions, child agents, delegated grants, cached credentials, target-system tokens, routes, queues, schedules, callbacks, local context and Skill copies, and persistent work. Quarantine pending and in-flight work, invalidate pre-incident action approvals and grants, and require fresh authorization for each action that may resume.

Observe the system for the period required to expose delayed callbacks or target processing. Reconcile target systems for effects that completed before or after suspension and keep stale work quarantined until each outcome is known.

Before restart after suspected compromise, revoke and rotate or reissue affected credentials, signing keys, sessions, delegated grants, and target-system tokens. Run residual-access tests against the old authority and inspect for use after revocation.

Pass when staff can preserve evidence, suspend all related access and work within the required time, quarantine in-flight and pending work, identify and contain external effects, reconcile actions across the post-stop observation window, invalidate stale action authority, restore approved context, reissue trusted authority, prove that old authority no longer works, and obtain fresh approval for restart and any resumed action.

10. Audit Retirement and Residual Access

Objective: determine whether retired agents stop operating without destroying evidence the organization must keep.

Evidence to inspect:

• Retirement approval and effective date
• Identity, credential, role, route, tool, integration, queue, schedule, and callback removal
• Child-agent and delegated-work closure
• State, Memory, local copy, cache, and backup handling
• Ownership transfer and evidence-retention records
• Post-retirement activity and residual-access scans

Sample retired agents and search for activity after the effective time. Attempt authorized access checks against former credentials, routes, tools, queues, and integrations. Verify disposition of state and copies, then confirm that retained audit evidence remains protected and discoverable.

Pass when retired agents cannot receive context, start or resume work, use credentials, invoke tools, or leave unowned Resources. Retention should preserve required evidence without preserving live authority.

How to Sample AI Agent Activity

Use both risk-based and random selection because they answer different questions.

Risk-based samples target the events most likely to expose material failure:

• High-impact or hard-to-reverse actions
• Broad data access or cross-tenant boundaries
• Human-impact decisions
• Delegated, scheduled, queued, or long-running work
• New agents, tools, Skills, models, or integrations
• Emergency changes and approved exceptions
• Denials, retries, timeouts, and partial failures
• Alerts, incidents, suspensions, and recoveries

Random samples test whether ordinary operation matches the documented process. If the auditor only selects known problems, the results cannot support a claim about routine control performance.

Record the population source, total count, query or export time, selection logic, random seed when used, replacement rules, excluded records, and selected IDs. Expand the sample when an exception suggests a broader or systemic problem. Do not replace failed samples with clean ones.

Sample size should follow the assurance goal, control frequency, expected deviation, population size, risk, and audit method. A fixed number copied from another organization is not a defensible rule by itself.

Grade Evidence Before You Grade the Control

Strong evidence is:

• Attributable to a person, agent, service, and tenant
• Tied to the audit period and sampled event
• Complete enough to show the decision and outcome
• Stable through immutable versions, digests, hashes, or protected references
• Produced by the enforcing system or an independent source
• Protected from the subject of the audit
• Minimal enough to avoid a new privacy or secret exposure

A policy statement proves design intent. A configuration can prove implementation. A representative test can prove behavior under tested conditions. A protected runtime trace can prove that the control operated for one event. Use the evidence type that matches the audit claim.

Interview evidence helps explain a process, but it should not be the only support for a technical control that can be examined or tested. Screenshots also age quickly and often omit the actor, version, query, and time. Prefer exports, API responses, signed manifests, versioned records, and reproducible tests.

Report Findings and Track Remediation

Write each finding so another person can retest it:

Finding title:
Criterion:
Condition observed:
Affected scope and samples:
Evidence references:
Risk and likely impact:
Root cause:
Severity:
Required or proposed action:
Owner and due date:
Interim control or exception:
Retest procedure:
Closure evidence:

Define severity before testing. A useful model weighs impact, likelihood, effective reach, affected data or people, reversibility, detectability, exploitability, duration, and whether the failure is systemic.

Do not close a finding because a policy was rewritten. Retest the changed control with the same failure condition, inspect a new operating sample when the control is recurring, and confirm that the fix did not move the weakness to another path.

Treat audit scope limits as scope limits. Missing evidence, an unavailable system, or a blocked test should not become a passing result. State what the review could and could not support.

Audit Checklist, Governance Checklist, and Readiness Review

Practice	Primary user and output
Governance checklist	Owners confirm that required governance work and evidence exist for one workflow.
Audit checklist	An independent reviewer selects samples, tests control operation, reports findings, and verifies remediation.
Security assessment	Security specialists test misuse resistance and release controls for a defined workflow and release.
Readiness assessment	Release authorities decide whether one workflow may enter its next operating stage.
Point-in-time audit	A reviewer reconstructs the inputs, authority, actions, and outcome for one moment.

These practices can use the same evidence, but they answer different questions. The security assessment focuses on misuse resistance and a bounded release decision. The audit covers broader governance, security, data, operation, and lifecycle controls across a defined period. A completed governance checklist is an input to an audit, not proof that every listed control operated throughout the period.

How Alignbase Supports an AI Agent Audit

Alignbase is an AI context control plane that keeps governed Knowledge, Skills, and Memories in one context repository and routes them to agents.

Knowledge and Skills use review and publication. Memory is live, versioned, and audited. Repository permissions stay independent from Always delivery routes, so an auditor can test access and delivery as separate decisions. Point-in-time assembly records can identify the published Knowledge and Skill versions, current Memory version, route source, and intended agent. Linked trusted host or request-session evidence is needed to prove what entered the agent’s final request.

That structure supports the context procedure in this checklist. It does not replace the rest of the audit. Identity, action authorization, tool execution, approval, external outcomes, incident response, and retirement still need evidence from the systems that enforce those controls.

Start with the highest-authority production workflow, define the populations, and run all ten procedures. The Alignbase blog has deeper guides on governance, controls, audit logs, and context delivery when a finding needs a more specific remediation plan.