Design-time vs runtime AI security — where review belongs

Design-time security asks: should this system go to production?It evaluates the system as designed, against the deployment context it is being deployed into, and produces a judgment about whether the system’s risk profile is acceptable. The primary output is a clearance decision — a documented judgment that names the risks, the controls, the residual exposures, and the conditions under which the system may operate.

Runtime security asks: is this system behaving as it should? It observes the deployed system, compares observed behaviour to expected behaviour, and surfaces anomalies, policy violations, and unexpected output patterns.

Design-time review establishes what “normal” and “acceptable” look like before the system is deployed. Runtime monitoring detects deviations from that baseline after it is. Without the design-time baseline, runtime anomaly detection has no reference to work from.

Design-time security operates on the architecture, the threat model, the configuration, and the intent of the system — not on a deployed instance. It is the set of activities that happen before go-live, or before a significant change to a deployed system, to determine whether the system is safe to operate.

Design-time activities include:

• System boundary definition. What components are in scope, what are the data flows, and what is explicitly excluded (with a documented reason).
• Threat modelling. Identifying the threats specific to the system, its deployment context, and the model family in use. STRIDE, OWASP LLM Top 10, and OWASP Agentic Top 10 are common starting frameworks.
• Control plan development. For each identified threat, specifying the control required, the owner, the verification method, and the lifecycle gate at which it must be in place.
• Residual risk assessment. Identifying the risks that controls do not fully close, and obtaining explicit acceptance from a named, accountable person.
• Clearance decision. The formal judgment — proceed, conditional, restricted pilot, hold, or decline — that documents the outcome of all the above.

Design-time review is the pre-condition for an informed go-live. A system that goes to production without a design-time clearance has not had its risk profile assessed. Runtime tools deployed against it have no baseline and no agreed definition of what “anomalous” means.

Runtime security operates on a deployed system. It observes the system’s behaviour — the inputs it receives, the outputs it produces, the tool calls it makes — and compares that behaviour to a policy or a baseline. Its primary purpose is detection: identifying when the system is behaving in a way that diverges from its intended operation.

Runtime security activities include:

• Input policy enforcement.Filtering or flagging inputs that match known attack patterns — prompt injection signatures, PII patterns, off-topic inputs outside the system’s intended scope.
• Output policy enforcement. Reviewing model outputs before they are returned to users — blocking content that violates policy, flagging outputs that suggest the model is following injected instructions.
• Audit logging of model interactions. Capturing inputs, outputs, tool calls, retrieved documents, and session context at a level of detail that supports post-incident investigation.
• Anomaly detection. Identifying sessions or output patterns that deviate from the baseline established at design-time review — unusual tool call frequency, unexpected data access patterns, outputs that match known attack signatures.

Runtime tools are valuable, but they are not a substitute for design-time review. They can detect that something went wrong. They cannot tell you whether the system should have been deployed in the first place, whether the threat model is complete, or whether residual risks were properly accepted.

The evidence each discipline produces is different in kind, not just in source.

A governance framework that asks “do you have AI security?” should be asking both questions separately. “Do you have design-time clearance records for your AI systems?” and “do you have operational controls that detect policy violations in deployed systems?” are different questions that require different evidence.

Running only one of the two disciplines leaves specific, predictable gaps.

Design-time review without runtime security:

• You know the system was assessed before deployment. You do not know whether it is behaving as intended now that it is deployed.
• You cannot detect prompt injection attempts at scale, or identify that the model is producing a pattern of outputs it should not be.
• You lack the audit trail that post-incident investigation requires. If an incident occurs, you may have the design-time record showing what was intended, but not the operational record showing what actually happened.

Runtime security without design-time review:

• Your runtime tools have no baseline to work from. Anomaly detection without a documented normal produces false positives, alert fatigue, or missed detections because the threshold was miscalibrated.
• You can detect that something is wrong but cannot tell whether it reflects a design flaw (the system was never properly scoped) or an operational failure (the system was correctly designed but a control is not working).
• You have no defensible record for governance. An alert log showing suspicious inputs is not a governance record. It does not demonstrate that residual risks were accepted, controls were specified, or the system was cleared to operate.

The design-time review sets the parameters for the runtime programme. Specifically:

• The threat model from the design-time review identifies the attack patterns the runtime programme should be able to detect. If indirect prompt injection via retrieved documents is in the threat model, the runtime programme should have a detection strategy for that pattern.
• The control planspecifies the operational controls that must remain in place — and the verification methods for each. Runtime audit logs are often the verification method for controls marked as “ongoing.”
• The re-assessment triggers define the events that require a return to design-time review. When a runtime anomaly indicates that a control assumed to be in place is absent, that fires a re-assessment trigger.
• The residual risk acceptance defines what the runtime programme does not need to eliminate — it is accepted risk, with named conditions. If a runtime alert relates to an accepted residual risk, the response is to verify the condition still holds, not to treat it as an uncontrolled finding.

A runtime programme that operates without the design-time record is flying without instruments. The design-time record is the instrument panel.

The AI security review is the design-time discipline. It belongs at the start of an AI system’s lifecycle, before production, and again at each of the trigger points that indicate a material change to the assessed system.

In a complete AI security programme, the sequence is:

1. Design-time review produces the clearance decision, the control plan, and the baseline definition of normal operation. This is the go-live gate.
2. Runtime security is configured against the threat model and control plan from the review. Detection strategies are targeted at the risks the review identified. Operational controls from the control plan are verified against the audit logs the runtime programme captures.
3. Trigger events return the system to design-time review — model change, scope expansion, or incident. The updated review produces a new baseline. The runtime programme is reconfigured against it.

This is not a linear pipeline with one pass. It is a cycle. Design-time review and runtime security are complementary disciplines operating at different points in that cycle. Neither replaces the other, and a security programme that runs only one of them has a predictable gap at the layer it skipped.

Drel is a design-time platform. It produces the clearance record, the control plan, and the re-assessment triggers that the runtime programme needs. It does not replace runtime tools — it is the prerequisite that makes them effective.