AI in GMP: From Governance to Continuous Validation

1.0. The AI in GMP Assurance Challenge

In Part 1, “EMA AI Roadmap: 61 Use Cases for Pharma GxP Compliance,” we examined how the European Medicines Regulatory Network is moving beyond AI experimentation toward a more structured approach built around use cases, AI tools, governance, validated prompts, structured data, and continuous monitoring. A central question emerged: how can organizations demonstrate that AI remains fit for purpose after deployment, as models, prompts, data, retrieval sources, vendors, and workflows change?

In Part 2, “EMA Regulatory Data: PMS, eAF and Compliant Submissions,” we extended that principle beyond AI itself. The EMA’s digital regulatory transformation shows how PMS, eAF, RIM, eCTD, ePI, and related regulatory systems are becoming an increasingly connected data ecosystem. As regulatory processes become more data-driven, validation must address not only individual applications, but also the data, interfaces, configurations, and processes that support regulatory outcomes over time.

Part 3 brings these two perspectives into the GMP environment.

Consider an AI application that reviews environmental-monitoring data, identifies a possible excursion, retrieves relevant batch and deviation records, drafts an investigation summary, and recommends escalation.

Several months later, a Quality reviewer or inspector asks, Which model and configuration produced the output? What data and knowledge sources influenced it? What did the AI recommend, and what did the human reviewer change? Was the system operating within its approved Context of Use? Did a model, API, prompt, data source, or cloud service change after validation? Can the organization demonstrate that the AI remains fit for its intended purpose today?

These questions bring together the themes from the first two parts, connected systems require connected evidence, and AI systems require lifecycle assurance.

The conversation is therefore moving beyond “Can AI be used in a regulated GMP environment?” toward a more operational question:

“How can an organization continuously demonstrate that an AI-enabled process remains controlled and fit for its intended purpose under GMP?”

This shift brings governance, validation, AI risk management, human oversight, performance monitoring, supplier management, change control, and evidence into one connected lifecycle.

2.0. The GMP Question Is Changing

For years, discussions about AI in pharmaceutical manufacturing focused on trust, model validation, explainability, and policy. Those questions remain relevant, but they do not fully address what happens after deployment.

A model may perform well during qualification and still create assurance challenges later. Its prompts may change. Retrieval sources may be updated. A supplier may modify an API. A cloud provider may change infrastructure. Users may apply the tool in workflows that were not included in the original Context of Use. Data quality may deteriorate, or the operating environment may shift.

The issue is therefore not simply whether an AI model can produce an accurate output during testing. The more important issue is whether the complete AI-enabled process remains controlled in routine operation.

That process includes the model, application, data, interfaces, users, procedures, suppliers, human decisions, and records created or influenced by the system. It also includes the evidence needed to reconstruct what happened when a decision is questioned.

This is where AI governance in pharma must become operational. Governance defines expectations, responsibilities, permitted uses, risk tolerance, and escalation paths. Operational controls and evidence demonstrate whether those expectations are working in practice.

The transition is clear,

From “Can we use AI?”

To “Can we control AI throughout its lifecycle?”

3.0. AI in GMP and the EU-GMP Annex 22 Discussion

The EMA’s 30 June–1 July 2026 multistakeholder workshop on AI guidance development for EU-GMP Annex 22 matters because it reflects this changing assurance question. The workshop was organized to gather expert opinions and evidence to inform the development of EU guidance on the use of artificial intelligence in medicines manufacturing.

The first day was an open session in which experts presented opinions and evidence. The second day was a closed session involving the Annex 22 drafting group. The workshop did not finalize Annex 22, create a new legal requirement, or establish a blanket permission for generative AI, large language models, dynamic systems, adaptive systems, or probabilistic models in critical GMP applications.

Its significance is that it signals an evolving regulatory discussion.

Earlier draft discussions and industry commentary had raised concerns about whether dynamic, adaptive, probabilistic, generative, and large-language-model systems could be adequately assured in critical GMP applications. Stakeholder discussions now suggest that risk-based control strategies, guardrails, lifecycle monitoring, and human oversight are receiving attention alongside those concerns.

The final regulatory position remains subject to Annex 22 drafting and the formal regulatory process.

A probabilistic model is not automatically unacceptable because its behavior differs from deterministic software. However, probabilistic behavior changes the validation and control challenge. Organizations need to understand the intended use, Context of Use, potential failure modes, impact of incorrect outputs, uncertainty, accountability, monitoring, and change detection.

This connects directly to established pharmaceutical quality principles. ICH Q9(R1) Quality Risk Management provides a foundation for assessing and controlling risk. Lifecycle management, data integrity, computerized-system expectations, outsourced activities, supplier and service-provider management, risk-based validation, performance monitoring, and change control remain relevant.

AI in GMP does not necessarily require abandoning those principles. It requires applying them in a way that reflects AI’s behavior, dependencies, and rate of change.

4.0. Context of Use Is the Anchor

AI validation in pharmaceutical manufacturing should begin with intended use and Context of Use, not with the model name, vendor, or benchmark score.

The same model can have very different regulatory significance depending on how it is used. An AI tool that produces an internal administrative summary may have limited GMP impact. An assistant that summarizes controlled procedures presents a greater concern because it may influence how personnel interpret approved information.

An AI system that recommends a product-quality decision, interprets a critical process signal, generates a GMP record, supports batch release, or controls a manufacturing process presents a substantially higher assurance challenge.

The difference is determined by the consequence of error, potential impact on product quality and patient safety, data-integrity implications, data sensitivity, system connectivity, degree of autonomy, level of human oversight, and ability to create, modify, or influence regulated records.

A practical GxP AI assurance model connects Intended Use to Context of Use, then to Risk, Controls, Performance Evidence, Human Oversight, Monitoring, and Change Management. These elements should not be treated as separate documents or isolated activities. They form a connected lifecycle.

If the model, prompts, retrieval sources, data, users, workflow, interface, supplier, or operating environment changes, the relationship between these elements may change as well.

That is why a benchmark score cannot establish fitness for every use. Performance evidence must be relevant to the actual decision, workflow, data, and risk.

A check at only one point in the chain cannot provide assurance across the whole process.

5.0. Guardrails and Human Oversight

Guardrails are layered controls, not a single technical feature.

Preventive controls can restrict inputs, users, data, tools, permissions, and operating boundaries so that an AI application does not operate outside its approved Context of Use. In a deviation-management workflow, for example, the system may be limited to approved procedures, validated data sources, defined user roles, and authorized investigation templates.

Detective controls identify unsupported outputs, uncertainty, abnormal behavior, degraded performance, data-quality problems, prompt failures, unexpected tool use, or changes in the operating environment. Monitoring may reveal that an environmental-monitoring classifier performs differently after a change in sampling patterns or facility conditions.

Containment controls limit the impact of an unacceptable result through escalation, fallback procedures, human intervention, workflow suspension, controlled shutdown, or use of an alternative approved process.

Together, these controls create a prevent–detect–contain model. No single prompt, model setting, validation test, or user review should be expected to eliminate every failure mode. The objective is not to make AI perfect. It is to make the complete AI-enabled process controlled and demonstrably fit for its intended purpose.

Human oversight must be designed with the same discipline.

Placing a person at the end of an automated workflow, or adding a “reviewed by a qualified person” checkbox, does not automatically create effective control. The organization should define what the reviewer must verify, what supporting evidence must be visible, what constitutes an exception, when the output must be rejected, how rejection and escalation are handled, how reviewer performance is assessed, and how the review is recorded.

The reviewer also needs sufficient context, training, authority, time, and access to source information. Human review does not automatically make an AI system compliant. It must be risk-appropriate, meaningful, documented, and supported by evidence.

6.0. Continuous Validation in Pharmaceutical Manufacturing

A static validation package may not provide sufficient assurance for every AI-enabled GMP system.

Models, prompts, system instructions, retrieval sources, reference data, APIs, cloud services, user behavior, application interfaces, external dependencies, and operating conditions may change. Even when the underlying model remains unchanged, performance may shift because the data, workflow, user population, or operating environment has changed.

Continuous validation does not mean fully revalidating every AI system after every minor change. It means establishing a rational, risk-based mechanism for determining what changed, what risk the change introduces, which intended uses and Contexts of Use are affected, and which controls, test cases, performance criteria, or human-oversight procedures may need reassessment.

The resulting decision may involve monitoring, targeted testing, approval, escalation, or revalidation.

For an AI system that identifies a possible environmental-monitoring excursion, the organization should be able to reconstruct the event using the approved use, relevant model and configuration, data and knowledge sources, generated output, available uncertainty indicators, reviewer actions, final decision, exceptions, and subsequent changes.

The same principle applies to an AI assistant supporting a deviation investigation. If it retrieves historical records and drafts an investigation summary, the evidence should show which sources were used, what the system generated, what the investigator accepted or changed, and why the final conclusion remained within the approved workflow.

Continuous validation may include performance tracking, stress and robustness testing, independent datasets, uncertainty monitoring, drift detection, periodic review, and risk-based revalidation triggers. The purpose is not surveillance for its own sake. It is to maintain a defensible connection between system behavior and intended use.

7.0. Where Continuous Intelligent Validation (cIV) Fits

Continuous Intelligent Validation (cIV), should be understood as an operational evidence capability rather than another static validation document or isolated testing tool.

Its purpose is to connect requirements to risks, risks to controls, controls to test scenarios, test scenarios to execution evidence, and execution evidence to approvals. It can connect changes to affected validation activities and performance signals to revalidation decisions.

This turns validation evidence from a point-in-time deliverable into a continuously maintained system of evidence.

For CSV and CSA professionals, that means extending familiar traceability into the operational lifecycle. For Quality leaders, it can support review, inspection readiness, change-impact assessment, supplier-change evaluation, deviation investigations, and lifecycle assurance. For digital and AI teams, it provides a structured way to connect technical changes with GxP consequences.

The central question becomes: “What evidence demonstrates that this AI system remains fit for its intended purpose today?”

That question is more useful than asking only, “Was this AI system validated?”

cIV does not replace Quality, validation professionals, regulatory judgment, or human accountability. The human remains responsible for quality and compliance decisions. cIV helps teams generate, connect, review, trace, and maintain the evidence needed to support those decisions.

8.0. The Future of AI Control in GMP

This is the third part of the AI, GxP & Regulatory Transformation Series.

In Part 1, we examined the EMA AI Roadmap and the emergence of a more structured approach to AI across the pharmaceutical regulatory ecosystem. The key lesson was that AI cannot be treated as a one-time deployment. Use cases, prompts, models, data, governance, and monitoring must be considered across the lifecycle.

In Part 2, we followed that lifecycle thinking into the EMA’s broader digital regulatory transformation, exploring how PMS, eAF, RIM, eCTD, ePI, and connected regulatory data are changing where compliance risk appears. The focus shifted from validating individual applications to maintaining control over the connected data, interfaces, configurations, and processes that support regulatory outcomes.

This part applies those ideas to AI in GMP manufacturing and continuous validation.

Continuity is the common theme. Regulatory data, AI models, validation evidence, quality decisions, manufacturing systems, suppliers, and human oversight form connected operational ecosystems.

The future compliance challenge will not simply be validating individual technologies. It will be maintaining continuous, traceable evidence that the connected AI-enabled system remains controlled.

Validation is moving from an event to an operating capability.

The future of AI in GMP will not be determined only by the number of policies an organization publishes or the number of validation documents it creates. It will depend on whether the organization can continuously demonstrate what the AI was intended to do, what it actually did, what controls operated, what humans reviewed, what changed, what performance signals showed, and why the system remained within its approved Context of Use.

The question is no longer simply, “Has the AI been validated?”

It is, “Can we continuously demonstrate that the AI remains fit for its intended purpose?”

That is the foundation of Continuous Intelligent Validation (cIV).

9.0. References

1. EMA — AI Guidance Development for EU-GMP Annex 22
2. PDA — EMA June Meetings: AI in GMPs
3. ICH Q9(R1) — Quality Risk Management