AI Auditing Needs More Than Model Evaluation — Designing audit-ready AI across decisions, execution, and outcomes

The AI made a recommendation. A person approved it. The system logged “success.”

Could an independent reviewer nevertheless establish that the operation was carried out under the conditions that were approved — and that the intended outcome actually occurred?

The target might have changed after approval. The information supporting the decision might have become outdated. “Success” might mean only that a command was transmitted. Model evaluation results and explanatory text alone cannot answer these questions.

Our argument is straightforward:

Effective AI auditing requires more than audit methods. It requires systems designed from the outset to make AI decisions, acceptance, execution, and outcomes verifiable.

This is not an argument against model evaluation. It is an argument about what must connect to model evaluation when AI is entrusted with consequential business decisions and actions.

1. What should an independent reviewer be able to verify?

Primary sources published in September 2026 make this question increasingly concrete.

In “We Must Pace the Frontier,” Dario Amodei sets out Anthropic’s commitment to give embedded third-party evaluators ongoing, employee-like access. Their scope extends beyond completed models to training and operational processes and adherence to safeguards. The proposed arrangement also gives reviewers the right to publish key findings without Anthropic’s editorial control, subject to limited information-protection exceptions. [1]

On September 16, OpenAI published a framework for tracking, investigating, and disclosing model misalignment across the model lifecycle, without requiring actual harm or a complete explanation before disclosure. Its six initial reports include instructions to conceal mistakes inserted into task summaries and unauthorized external file sharing. They are individual case reports, not statistics establishing how frequently such behavior occurs. [2]

Anthropic’s September 9 assessment examines four incidents of unauthorized access to real third-party systems from misconfigured cybersecurity evaluation environments. The evaluations ran without the cyber safeguards used in released products. The company also tested additional blocking monitors. For our purposes, the significant connection is between model behavior and the environment, monitoring, and controls surrounding execution. [3]

California’s September 18 executive order directs the development of recommendations on potential changes to state law concerning frontier AI, including onsite independent verification at large frontier developers and ongoing verification of model shutdown mechanisms. The task includes examining technical feasibility. The order does not, by itself, make those measures blanket implementation requirements for companies. [4]

A voluntary corporate commitment, a research report, and a process for considering legislative changes are different things. They should not be presented as one established standard.

Our reading is that, alongside explanations of safety, these materials give concrete attention to the access, records, and controls needed to check whether safeguards actually work. We now apply that question of verifiability to the design of consequential operations in organizations using AI.

2. “The AI can propose it” does not mean “the business may accept it”

Consider an illustrative, hypothetical logistics scenario. An AI proposes consolidating two shipments:

“Transporting them together would reduce shipping costs.”

Even a plausible recommendation is not sufficient authorization to dispatch a vehicle. Shipment readiness, delivery deadlines, receiving capacity, and shipper permissions must satisfy the relevant conditions.

When waiting to consolidate would make an urgent delivery late, reject that consolidation proposal. When the necessary permission cannot be established, hold the proposal. Where a separate shipment meets the delivery deadline and other conditions, select that alternative.

The requirement is not merely to report later that the AI’s recommendation was unsuitable.

Do not let a proposal proceed to execution when its acceptance conditions are not satisfied. Preserve verifiable reasons for rejection and deferral, as well as acceptance.

A change to the destination or departure time after approval also requires checking whether the original approval still applies. Confirming delivery must remain distinct from confirming that a dispatch instruction was sent.

The central issue is not just the intelligence of the model. It is a design that keeps recommendation separate from authority, approval separate from execution, and execution separate from outcome.

3. The Objective Is Verifiable Links, Not More Logs

For AI systems handling consequential operations, we treat the following chain as a basic unit of design:

AI proposal → acceptance checks → execution authorization → execution → outcome verification

Evidence should not be assembled only afterward into an explanatory report. It should be generated and retained at each stage, with the relationships between stages preserved. An independent reviewer should then be able to compare the records against the conditions that applied.

From an audit perspective, we organize the problem into five questions. This is our design framework, not a common standard established by the cited sources, and not a requirement to collect identical records for every use of AI.

• Decision context

  
  

  What supported the proposal at the time?

  
  

  Relevant evidence may include the versions of inputs and reference information, acquisition times, and applicable conditions.

• Acceptance grounds

  
  

  Which conditions were satisfied, and under whose authority was the proposal accepted or held?

  
  

  Relevant evidence may include criteria versions, check results, approvals, exceptions, and deferrals.

• Execution consistency

  
  

  Was the approved operation actually performed on the approved target?

  
  

  Approval and execution identifiers, operation details, and pre- and post-execution states should be linked.

• Outcome validity

  
  

  What was established beyond the fact that a command was accepted or transmitted?

  
  

  Depending on the operation, this may involve prior guarantees applicable to the execution together with measurements, receipts, or state records.

• Intervention effectiveness

  
  

  When intervention was required, what could actually be stopped or changed, and what state was reached?

  
  

  Shutdown, cancellation, and safe-transition tests, elapsed times, and resulting states may all be relevant.

A log saying “approved” does not establish what was approved. A log saying “stop request sent” does not establish that stopping was completed.

An unchanged record may also contain an incorrect observation from the outset. Record integrity, the trustworthiness of observations, and the link to the particular execution are separate questions.

Likewise, stopping a software process and bringing physical equipment to a safe state are not the same operation. Testing intervention effectiveness first requires defining what should stop, within what boundary and time, and what state should follow.

Reverification does not mean sending the payment again or repeating the physical operation. It means checking the retained grounds against the criteria that applied at the time. The necessary evidence, access to confidential information, and retention periods should therefore be designed together.

4. Verification infrastructure does not replace auditor independence

Institutional oversight and technical execution controls have different roles.

The IIA’s Third-Party Topical Requirement became effective on September 15, 2026. It addresses governance, risk management, and controls in applicable third-party assurance engagements. Applying it to an AI vendor depends on the relationship, risks, and engagement scope; it is not a new statutory audit obligation for every company using AI. [5]

Japan’s Financial Services Agency, in its financial administration priorities published on September 15, also calls for strengthening internal audit’s autonomous verification and checking functions, alongside improvements to financial administration that assume the use of generative AI. It does not prescribe a particular execution gate or evidence architecture. [6]

Our position is that the people who design and operate a control should be distinguished from those who independently assess its adequacy and effectiveness, with verifiable evidence connecting their work.

A gate can work exactly as configured while enforcing an inappropriate rule. Technology alone does not decide who sets the criteria, authorizes exceptions, or revises them.

The infrastructure should not claim to make audit judgments automatically correct. It should enable auditors to examine the grounds and challenge the conclusions.

5. This connection is the focus of GhostDrift’s research

At GhostDrift Mathematical Institute, our work on Responsibility OS and ADIC has explored structures linking decision grounds, responsibility records, execution, and evidence. Our public technical materials approach this problem from several directions. [7] [8] [9]

Responsibility OS Kernel: keep operations connected to responsibility and evidence.Built on the ADIC assurance structure, this Lean 4 formalization models how operations, audit traces, decision grounds, and responsibility records remain connected as processes are composed. Its concern is preserving relevant distinctions in responsibility and evidence within an observable operational view. [7]

ADIC Cyber Assurance Gateway: a generated action is not its own authorization.This formal model links authorization, approvals, pre- and post-execution records, and state relationships so that the grounds for accepting a protected operation can be checked again. It separates producing an action from having justified grounds to accept it. [8]

Physical AI Outcome Assurance: correctness within a model is not the same as a realized outcome.This formalization distinguishes model guarantees, the executions to which they apply, and the outcomes that available evidence supports. It does not assert that new post-execution observations are always necessary: what can be certified depends on the scope of prior guarantees and the outcomes the evidence can distinguish. [9]

Viewed through the lens of AI auditing, our position is clear:

We are not seeking to replace auditors. We are building technical foundations that enable them to examine the connection from AI decisions to execution outcomes.

The aim is a structure in which acceptance conditions and execution grounds can be checked without unconditional reliance on an AI’s explanation. This does not eliminate trust. It makes dependencies explicit and expands what can be verified.

We aim to carry this design approach into consequential cyber operations, privacy-sensitive external disclosures, and decisions and actions in logistics and manufacturing. The conditions differ across domains, but the starting point is the same: “can be generated,” “may be accepted,” “was executed,” and “the outcome occurred” are distinct claims.

The public formalizations establish mathematical cores or conditional results under stated assumptions. They do not prove the safety or legal compliance of an entire commercial system. Evidence authenticity, correspondence with the deployed system, and the inability to bypass an execution gate still require separate validation. Making the boundary between what has been proved and what has not been proved explicit is itself part of auditable design. [7] [8] [9]

6. Audit AI — and design the systems that make auditing possible

This approach does not require one company’s implementation. Existing access management, approval workflows, audit logging, and runtime monitoring could also be combined to build it. The meaningful comparison is not the label, but how far decisions, authority, execution, and outcomes can be linked through evidence.

We describe this design objective as audit-ready AI. Here, the phrase is neither a certification nor a claim that an audit has been passed. It means that the conditions and evidence required for verification are built into the execution of business operations.

The objective is neither simply to stop inappropriate actions nor to accumulate records. It is to let actions proceed when the relevant conditions are satisfied, route them to a hold or intervention when they are not, and preserve grounds that make the distinction independently checkable.

Choose one recent consequential automated action by an AI system. Can someone trace why it was accepted, what was executed, and how the outcome was established — without relying on the operator’s memory?

We need not only more ways to audit AI, but ways to use AI that make auditing possible. GhostDrift’s focus is the decision, execution, and evidence infrastructure needed to do that.

Primary sources and public technical materials

Descriptions of policies and publications reflect materials available on September 21, 2026. The external sources below do not endorse or certify GhostDrift or the technical architecture proposed in this article.

External primary sources

[1] Dario Amodei, “We Must Pace the Frontier” (September 2026).Relevant section: Embedded Evaluators, covering access, scope, and publication rights.

[2] OpenAI, “Our framework for reporting model misalignment” (September 16, 2026).Relevant sections: disclosure criteria, the initial six reports, and the disclosure process. This is the developer’s own reporting framework.

[3] Anthropic, “An alignment assessment of recent cybersecurity incidents” (September 9, 2026).Relevant sections: Introduction, Replication and monitoring, and the assessment of monitors, including their limitations and evaluation conditions.

[4] State of California, Executive Order N-9–26 (September 18, 2026; PDF).Relevant provision: section 3, particularly 3(a)–3(c), directing recommendations on potential legislative changes concerning independent verification and shutdown mechanisms.

[5] The Institute of Internal Auditors, “Third-Party Topical Requirement” (issued September 15, 2025; effective September 15, 2026).See pages 2–3 of the requirement document (PDF) for applicability and organizational accountability for third-party risks.

[6] Financial Services Agency, Japan, “2026事務年度金融行政方針（主なポイント）” (September 15, 2026; Japanese PDF).Key points of the FSA’s 2026 financial administration priorities, including internal audit and AI-enabled administrative reform. Publication page. English descriptions in this article are paraphrases.

GhostDrift’s public technical materials

[7] GhostDrift Mathematical Institute, “Responsibility OS Kernel”.A Lean 4 mathematical core linking operations to responsibility and evidence. Read the README’s Core claim and What is not proven together.

[8] GhostDrift Mathematical Institute, “ADIC Cyber Assurance Gateway”.A Lean 4 formal model for evidence-based reverification of operation acceptance. See What this proof establishes, Trust boundary, and Scope.

[9] GhostDrift Mathematical Institute, “Physical AI Outcome Assurance”.A Lean 4 formalization distinguishing model guarantees, their application to target executions, and outcome certification. See Main result and What the formalization does not claim.