Assessing the AI feature inside SaaS you already bought

The embedded AI problem is structural: the security review that approved the vendor was designed to assess how the vendor's software processes customer data, not how an LLM embedded in that software processes it. These are different things, and the difference is not captured by updating the vendor's SOC 2 status or confirming that the new feature is covered by the existing DPA.

Existing DPAs cover how the SaaS vendor processes customer data within its own infrastructure. When the vendor adds an AI feature powered by a third-party model provider — and most embedded AI in SaaS products is powered by a third-party model — the original DPA may not govern how that model provider processes the data it receives. The subprocessor chain has changed, and the legal basis for the new processing relationship may not have been established.

To conduct a supplemental assessment for an AI feature in an existing SaaS product, the first step is to characterise what changed when the AI feature was added. The relevant changes are in four areas:

Data flows.Customer data that previously stayed within the vendor's infrastructure now travels to a model inference API. The destination, the retention policy, and the access controls are different. In many cases, the model inference API belongs to a third-party provider the customer has no direct relationship with.

Processing logic. The outputs of an AI feature are probabilistic, not deterministic. The same input can produce different outputs at different times. The feature may produce outputs that are unexpected, incorrect, or — in the case of a customer-facing AI assistant — harmful.

Training risk. The model powering the AI feature may use inference data — including prompts containing customer data — to train or fine-tune the model. The DPA for the SaaS vendor likely does not address this; the data processing terms for the model provider need to be examined separately.

Capability surface.The AI feature may have capabilities that the original vendor assessment did not contemplate. A model that can summarise documents may also be able to produce outputs that are operationally significant in ways the feature's intended use case does not expect.

The supplemental AI security assessment for an embedded AI feature has a narrower scope than a full vendor assessment — it focuses specifically on the AI feature and its additions to the existing vendor relationship. The starting point is the original vendor assessment: what was established, what was approved, and what conditions were attached. The supplemental assessment adds to that record.

The supplemental assessment covers three areas: model disclosure (who provides the model, what are its properties), data processing addendum gaps (what the existing DPA does not cover), and opt-out controls (whether the AI feature can be disabled or scoped to exclude sensitive data classes).

A supplemental assessment is not a shortcut around a full review. It is the right scope for the right question: not “should we use this vendor?” but “does the AI feature this vendor added change our risk exposure in ways the original assessment did not address?”

The model disclosure section of the supplemental assessment asks the vendor to identify and describe the AI model that powers the feature. The specific questions:

• Which AI model powers this feature? Specify the provider, model family, and version.
• Is the model hosted by your organisation or by a third-party provider? If third-party, identify the provider.
• Does the model provider have access to the customer data sent during inference? On what terms?
• Does the model provider retain inference data? For how long?
• Does the model provider use inference data for model training or fine-tuning? Can customers opt out?
• What is the data processing agreement between your organisation and the model provider? Does it meet the requirements of the applicable data protection regulation?
• What model card or equivalent documentation is available for this model?

Vendors who cannot answer these questions specifically — or who answer them only at the level of the SaaS vendor without addressing the model provider's practices — have an incomplete picture of their own AI subprocessor chain.

The DPA gap analysis reviews the existing data processing agreement against the new data flows introduced by the AI feature. The specific areas to check:

• Subprocessor list: Is the model provider listed as a subprocessor in the DPA? If not, the data transfer to the model provider may not have a lawful basis under the applicable data protection regulation.
• Processing purpose:Does the DPA's description of processing purposes cover the AI feature's use of customer data? Many DPAs describe processing for “provision of the service” broadly enough to cover new features — but not all do, and AI processing may require a specific purpose.
• Data subject rights: Can data subjects exercise their rights — access, erasure, portability — in relation to data processed by the AI feature? Does the DPA address how such requests are handled when data has been sent to a model provider?
• International transfers: Does the model provider process data outside the EU or other applicable jurisdiction? If so, is the transfer mechanism established in the DPA?

Where gaps are identified, the options are: negotiate a DPA amendment with the vendor, disable the AI feature pending resolution, or scope the AI feature to exclude data classes that require the missing protections.

Opt-out controls are the mechanisms that allow an organisation to limit the AI feature's access to specific data classes, or to disable the feature entirely. Their existence and granularity materially affect the risk assessment.

The specific questions:

• Can the AI feature be disabled at the account or workspace level without disabling other product functionality?
• Can the AI feature be scoped to exclude specific data fields, record types, or user segments?
• Can training data use be opted out of independently of disabling the feature entirely?
• Where opt-out controls exist, how are they administered and what is the evidence that the opt-out is effective?

An AI feature with granular opt-out controls allows an organisation to use the feature selectively — enabling it where the risk is acceptable, excluding it where it is not. An AI feature with no opt-out controls presents a binary choice: accept all the AI feature's data flows, or stop using the product.

The supplemental AI security assessment for an embedded AI feature should produce the following outputs:

• A description of the AI feature, the model that powers it, and the third-party provider if applicable.
• An assessment of the existing DPA's coverage of the AI feature's data flows.
• A list of identified DPA gaps and the remediation options for each.
• An assessment of the opt-out controls available and the residual risk where those controls are insufficient.
• A disposition record that references the original vendor assessment and describes what has changed, what additional controls are required, and what re-assessment triggers apply.

The supplemental assessment should be added to the existing vendor assessment record — not created as a separate file. The original assessment continues to cover the vendor's infrastructure and general security posture. The supplement covers the AI-specific addition. Together, they constitute the complete security record for the vendor relationship.

For the contractual terms that should govern the AI vendor relationship — including model-change notification and re-assessment rights — see The security terms an AI vendor contract needs.