Confidential AI: How to Use Sensitive Data with AI

The more AI becomes part of core business operations, the more often it comes into contact with sensitive corporate data. But conventional security architectures leave a gap when that data is actively processed. Confidential AI addresses that gap, enabling organisations to extend AI to workloads that would otherwise remain difficult to secure.
‍

Recent data from Germany shows how AI usage is spreading in the EU’s largest economy. In May 2026, 54.5% of German companies were using AI in their business processes, according to the ifo Institute. A further 37.6% were planning to adopt it or were discussing its use. In other words, more than nine in ten companies are now actively engaging with AI.
‍
‍

At the same time, more companies are moving to infrastructure operated by third parties. According to Bitkom’s Cloud Report 2026, 86% of German companies use cloud services, while 42% obtain AI services from the cloud. For 87%, protection against unauthorised access by the cloud provider itself is a “must-have criterion” when selecting a provider.

‍

This raises a key security question for organisations: how can AI be used with data that must remain confidential? This includes customer information and internal documents as well as intellectual property and research data. High security requirements often slow down digital initiatives. In a Bitkom survey published in March 2026, 77% of German companies cited data protection as an obstacle to digitalisation, while 61% pointed to technical security.

‍

For AI projects, this creates a practical conflict. Business units want to combine powerful models with their own data, while security and compliance teams need to ensure who can see that data, where it is processed and how access can be verified.

‍

Confidential AI helps organisations address these questions and enables them to use sensitive data in AI workloads.

‍

‍

What is Confidential AI?

‍

Confidential AI applies the principles of Confidential Computing to AI workloads. At its core, Confidential Computing protects sensitive data while it is actively being processed.

‍

This matters especially for AI security. A model processes prompts, contextual information and documents to generate a result. Retrieval-augmented generation (RAG) systems introduce additional processing steps: they create chunks and embeddings, store vectors and assemble the relevant context for each request.

‍

Conventional encryption protects data at rest or in transit, but does not cover these processing stages when data is in use. Confidential Computing addresses this gap by protecting data during processing inside hardware-based Trusted Execution Environments (TEEs). A TEE isolates the protected workload from the rest of the infrastructure. Remote Attestation adds cryptographically verifiable evidence about the state of the execution environment. Systems can use this evidence, for example, as a condition for releasing encryption keys or confidential data.

‍

For AI, this creates a different trust model. Confidential AI uses technical controls to restrict privileged access by infrastructure operators and administrators. The scope of the protected environment varies by AI model and use case.

‍

‍

External frontier models or a confidential runtime?

‍

The right Confidential AI architecture is shaped by the task at hand. What information does the model need to perform that task? And does the model actually need – or, which is often more accurate, is it allowed – to see sensitive information in its original form?

‍

With external models, where inference remains with the provider, the answer is often no. One option is to replace sensitive information with consistent pseudonyms inside a TEE before sending it to the external model. The model then receives only the abstracted information relevant for its task. The model output returns to the protected environment, where the application combines the pseudonymised information with the original data before delivering the final result.

‍

Pseudonymisation does, however, have its limits. It can preserve many linguistic relationships and structures, but not every type of analysis tolerates the same degree of abstraction. In some scenarios, real values or identities are necessary to produce useful results.

‍

If the model needs access to this unmodified information, Confidential AI architecture requires a different approach: the model itself can run within a confidential execution environment and process the original data there. This depends on if the model can be deployed in a controllable runtime. That is typically the case for proprietary models operated by the organisation itself and for many open-weight and open-source models. For smaller models or less compute-intensive workloads, a CPU-based TEE may be sufficient. Large language models generally require GPU acceleration. In this case, a Confidential GPU extends the protected environment to the AI computation itself.

‍

Neither approach is inherently superior. External models often provide access to high model performance and rapid innovation. Confidential model runtimes, by contrast, give organisations greater control over data, the model and execution within their own trust boundary.

‍

Model inference, however, is only one part of the data path. In RAG workflows, several sensitive processing steps take place before the model is even invoked.

‍

‍

How can internal knowledge remain protected on its way to the model?

‍

In RAG workflows, protecting confidential data starts before any request reaches the model. Documents are ingested, divided into sections and converted into embeddings. The system then stores vectors and metadata for subsequent queries.

‍

A Confidential RAG architecture can protect this data path in several stages: Source documents enter the system through a secure upload process and are stored in encrypted form. Chunking and embedding take place within confidential execution environments, verified by Remote Attestation. Vectors and metadata are subsequently stored in an encrypted database with tenant isolation and separate collections and indexes. During retrieval, only attested RAG components with an authorised workload identity can access the intended datasets. Document-level access controls and audit trails add further safeguards.

‍

Confidential RAG therefore protects the path from an internal document to the context supplied to the model. What happens to that context afterwards depends on the model architecture. With an external model, it can be pseudonymised before transmission. If the model itself runs confidentially, inference also remains inside the protected environment.

‍

The next question is what these measures enable in practice.

‍

‍

How organisations benefit from Confidential AI

‍

Confidential AI does more than extend the security architecture of existing AI systems. It also changes which data, models and deployment options are practically available to organisations.

‍

• Sensitive information can be used with AI: Today, AI projects are often constrained less by the capabilities of the model than by the security requirements surrounding sensitive data. Confidential AI can enable use cases in regulated sectors such as healthcare, public administration and financial services that might otherwise fail to meet data protection or internal security requirements.
  ‍
• More flexibility in models and infrastructure: Confidential AI also makes it possible to align the technical architecture more closely with the needs of each workload. For some tasks, a powerful external model may be the best fit. Others require greater control over the model and the data and therefore run inside a confidential environment. The same applies to infrastructure: depending on the specific workload, organisations can deploy in private or public clouds, on premises or in hybrid environments. By substantially restricting infrastructure operator access, confidential architectures can also allow more sensitive workloads to run on external cloud and managed infrastructure.
  ‍
• Governance and compliance: Confidential AI provides technical guardrails for sensitive data in AI workflows. Access controls, pseudonymisation and traceable processing can support data protection and AI governance requirements. The GDPR, for example, explicitly identifies pseudonymisation as a measure that can contribute to data protection by design. The EU AI Act is also increasing the need for controlled and documentable AI processes. Confidential AI does not in itself ensure legal compliance, but it can provide technical controls and evidence on which governance and compliance processes can build on.
  ‍
• Concrete use cases: The range of use cases extends far beyond individual AI chat tools. Customer service assistants can work with real case histories while personal data is pseudonymised before external model calls. Confidential RAG gives internal knowledge assistants controlled access to contracts, policies or case files. Confidential model runtimes make it possible to process highly sensitive financial or healthcare data, or proprietary source code for software development. Organisations can also provide employees with secure, validated access to AI models, creating a controlled alternative to shadow AI.
  ‍

Secure employee access is worth examining in more detail because it illustrates how Confidential AI can affect organisational processes as well as technical architecture.
‍

‍

Shadow AI: Secure alternatives rather than bans
‍

AI services have very low entry barriers. In many cases, all an employee needs is a web browser. As a result, these tools can become part of everyday work faster than organisations can establish clear guardrails. ISACA’s 2026 European AI Pulse Poll illustrates this: While 82% of organisations explicitly permit AI use and 74% permit generative AI, only 42% have a formal, comprehensive AI policy. At the same time, 87% of respondents are concerned about employees using AI without authorisation.
‍

Both outrights bans and the lack of rules are problematic, since freely available AI services remain accessible in just a few clicks and, particularly under time pressure, may become the most convenient option for employees. This leads to shadow AI operating outside controlled data flows. The challenge for organisations is therefore to provide secure alternatives that interfere as little as possible with usability in day-to-day work.
‍

Confidential AI allows organisations and their employees to use different types of models – public and private – while addressing one of the main drivers of shadow AI: the absence of approved access to capable models for workflows involving sensitive information. Organisations should first define which models may work with which categories of data and implement the appropriate confidential safeguards. These controls then operate directly within the data flow and remain largely invisible to employees, while logging and audit trails make usage traceable.
‍

The result goes beyond defining which tools are permitted or prohibited. Governance policies can now be enforced through technical controls. At the same time, Confidential AI enables organisations to offer employees AI workflows that are both useful and appropriate for confidential business data.
‍

‍

Conclusion
‍

Using AI models is increasingly becoming routine. The more difficult question is which data organisations can make available to AI applications when high security requirements apply.
‍

Confidential AI pushes the technical boundary. It protects sensitive information during critical processing stages and reduces reliance on trust in infrastructure and model providers by introducing technical controls. Depending on the architecture, those controls may apply to the pseudonymisation of sensitive data, a RAG pipeline or the entire model inference process.
‍

This also gives organisations a stronger technical foundation for addressing data protection and compliance requirements under frameworks such as the GDPR and the EU AI Act. Confidential AI alone, however, does not guarantee compliance. Nor does Confidential AI resolve every security issue associated with artificial intelligence. Attacks such as prompt injection or manipulated input data require separate measures.
‍

The main value of Confidential AI lies in expanding the range of options available to organisations. Companies in regulated sectors no longer need to exclude sensitive data from AI projects by default. Instead, they can determine more precisely which information is made available to which model, and under what technical conditions it may be processed. In turn, this can unlock new use cases. Many of them have been technically possible for some time, but have been held back because organisations lacked a sufficiently secure way to use sensitive data. Confidential AI addresses precisely that constraint.

‍