The restricted-pilot pattern for risky AI systems

A restricted pilot is a formal disposition that authorises an AI system to operate within a precisely defined scope — a specific user population, a specific data class, a specific operational context — under named controls, with a defined path to full production clearance and a defined trigger for pilot hold.

The three defining characteristics that make a restricted pilot different from an informal pilot or a conditional approval:

• Explicitly bounded scope.The scope is defined in terms of who, what, and where — not just size. “Internal HR team (12 people), for non-regulated internal queries only, within the HR information system” is a bounded scope. “Limited deployment” is not.
• Pilot-specific controls. The restricted pilot may require controls that would not be required at full production scale — for example, a mandatory human review of all model outputs during the pilot phase that would be impractical at scale but is appropriate for the limited pilot population.
• Defined path forward. The pilot exists to gather evidence for a production clearance decision. The evidence requirements are specified in the pilot disposition — not determined after the pilot completes.

The restricted-pilot pattern is appropriate in two scenarios:

Scenario 1: Risks that require operational evidence. The threat model identifies residual risks that can only be properly evaluated at limited scale — for example, risks related to how users interact with the system, risks related to edge cases that only emerge in real use, or risks whose severity depends on user behaviour that cannot be simulated in a design-time review. The pilot provides the operational evidence that the full production clearance decision requires.

Scenario 2: Controls that cannot all be verified before deployment.Some controls cannot be fully verified until the system is operating. A monitoring control that detects anomalous usage patterns cannot be verified until there is usage to monitor. An escalation process cannot be verified until there is something to escalate. The pilot phase validates these controls at limited scale before they are required to function at production scale.

The restricted pilot is not a way to deploy a system before it is ready. It is a way to acquire the evidence needed to determine whether the system can be made ready — while maintaining accountability for the scope within which it operates in the meantime.

Scope definition is the most important element of the restricted-pilot disposition. The scope must be defined across four dimensions:

• User population:The specific users, roles, or user groups authorised to use the system during the pilot. Named teams, specific roles, or a maximum user count. Not “a small group of users.”
• Data classes: The specific categories of data the system is authorised to process during the pilot. Any data class not listed is explicitly excluded. If the system may encounter excluded data, there must be a control that prevents it from processing that data.
• Operational context: The specific use cases, systems, or business processes the system is authorised to support during the pilot. Adjacent use cases not listed are explicitly excluded.
• Explicit boundaries: What the system is explicitly prohibited from doing during the pilot — capabilities disabled, data classes excluded, user-facing outputs limited. These boundaries define what is out of scope regardless of what the system is technically capable of.

Pilot-phase controls are different from production controls in two ways: they may be more intensive (because the pilot is specifically designed to evaluate risks that cannot be assessed design-time), and they may include controls that are not scalable to full production.

Controls typically required specifically at pilot phase:

• Manual review of all or a sample of model outputs during the pilot period.
• Enhanced logging that captures inputs, outputs, and model decisions at a level of detail that would not be retained in production.
• Named pilot users who have been specifically briefed on the system's limitations and expected to report unexpected behaviour.
• A named pilot owner responsible for monitoring the pilot, collecting pilot-phase evidence, and escalating anomalies.
• A defined review interval — typically weekly — at which the pilot owner reports on pilot-phase observations to the governance committee.

The path to production defines the evidence the pilot must produce and the gates that must be cleared before the system can receive full production clearance.

The evidence requirements should be specified at the time of the pilot disposition — not determined retrospectively. Typical evidence requirements for a restricted pilot:

• Pilot-phase incident log — a complete record of anomalous events observed during the pilot, their severity, and their resolution.
• Control verification evidence — confirmation that the pilot-phase controls operated as designed throughout the pilot period.
• Pilot-phase performance data — evidence relevant to the residual risks identified in the threat model, showing whether they materialised and at what rate.
• Pilot-phase user feedback — structured feedback from pilot users regarding the system's behaviour in edge cases.

The gates to full production clearance are the governance committee's review of this evidence and confirmation that the evidence supports production clearance. The gates must be specified — not just “pilot review.” What is the minimum evidence package, and what would cause the committee to decline production clearance after reviewing it?

A pilot hold is the immediate suspension of the pilot in response to a specific event. The pilot hold trigger must be defined in the pilot disposition — not determined ad hoc when a problem occurs.

Standard pilot hold triggers:

• Any incident involving data outside the authorised data classes.
• Any model output that causes identifiable harm to a pilot user.
• Any breach of the pilot scope — users outside the authorised population, use cases outside the authorised context.
• Failure of any pilot-specific control to operate as designed.
• A vendor incident (for vendor AI) that affects the system during the pilot period.

A pilot hold requires governance committee review before the pilot can resume. This creates an automatic escalation mechanism for events that exceed the pilot owner's authority to manage independently.

The restricted-pilot disposition must produce a documented evidence record that includes:

• The full scope definition with explicit boundaries.
• The pilot-specific controls with owner and verification mechanism.
• The path to production evidence requirements.
• The pilot hold triggers and the response process.
• The named pilot owner and the pilot review interval.
• The pilot duration — the date by which the pilot must produce the required evidence, or the system returns to the review queue.

For the broader disposition record structure, see The anatomy of an AI evidence pack.