By Prof. Eric Bouteiller, Adjunct Professor of Management, CEIBS

Artificial intelligence is rapidly moving from experimental use cases to real-world deployment across healthcare systems worldwide, reshaping how care is delivered, how decisions are made, and how value is created across the healthcare value chain. Yet while technological progress continues to accelerate, the more difficult question is no longer what AI can do, but how healthcare systems can adopt it safely, responsibly, and effectively at scale.

Drawing on field research and two panel discussions recently held on the CEIBS Shanghai campus, CEIBS Adjunct Professor of Management Eric Bouteiller explores how artificial intelligence is reshaping healthcare systems beyond technological innovation by identifying the key governance, accountability, trust, value creation, and ecosystem challenges that determine whether AI can be effectively and responsibly embedded into healthcare practice at scale.

Artificial intelligence is rapidly reshaping healthcare, but the debate is still too often framed as a question of technological capability. Can AI discover new molecules faster? Can it improve diagnosis? Can it reduce administrative burden?

These questions matter, but they are no longer sufficient. For AI in healthcare, the more difficult question is in fact managerial: under what conditions can AI be safely, responsibly, and effectively embedded into healthcare systems?

How AI Is Embedding Itself Across the Healthcare Value Chain

AI adoption is now visible across the healthcare value chain: from drug discovery, molecular design, clinical trial management, and pharmacovigilance, to medical imaging, patient triage, hospital workflows, and care delivery. This is no longer a distant scenario. The question is therefore shifting from whether AI will affect healthcare to how quickly, where, and under which governance conditions.

AI-discovered drugs are reaching human trials

Recent examples illustrate the scale of the change. An AI-driven biotech company, Insilico Medicine, has announced that an experimental lung-disease drug showed encouraging early results in human testing for a serious lung-scarring disease called idiopathic pulmonary fibrosis (IPF); the drug was originally discovered using AI, and the company’s AI-based discovery process was featured in a 2024 Nature Biotechnology paper. The strategic implication of this is not limited to one asset. The promise is higher R&D efficiency, faster cycle times, and lower development cost.

More recently, Hengrui and Bristol Myers Squibb have announced a strategic collaboration that signals a broader structural shift in the generalisation of AI in healthcare. This partnership covers multiple early-stage programmes and reflects a broader movement: global pharmaceutical companies are increasingly engaging with Chinese R&D ecosystems as a source of speed, cost efficiency, and pipeline optionality.

AI is becoming the first filter in the patient journey

The disruption is even more profound on the patient side. As highlighted during the recent discussions at CEIBS, AI may progressively become the first filter in the patient journey. For decades, patients in China began their journey through hospital visits and, more recently, via online search engines. Increasingly, AI-driven diagnostic tools may become the first point of orientation.
In a few years, this will change the logic of patient access. If a therapy is not embedded within the diagnostic or triage logic of AI-enabled systems, it may be excluded from the treatment pathway before the physician even sees the patient.

Despite this momentum, however, execution remains difficult. The recurring issue is not a lack of ideas, but the difficulty of translating AI capability into safe, trusted, measurable, and accepted healthcare practice. In that sense, AI in healthcare should be understood less as a technology project than as an organisational transformation.

Six Core Managerial Challenges for AI Adoption    

The following six challenges summarise the key takeaways from the panel discussions. They are presented as a practical framework for healthcare organisations, pharmaceutical companies, insurers, hospitals, regulators, and technology providers.

Accountability must remain with qualified professionals

AI systems can improve efficiency and reduce costs, but they also introduce new risks: hallucinations, opaque recommendations, biased outputs, and misleading confidence. The central issue is not whether AI can be made perfect. It cannot. The real issue is how responsibility is allocated when AI contributes to a clinical or operational error.

Healthcare cannot outsource responsibility to an algorithm. Even when AI supports diagnosis, triage, treatment selection, workflow prioritisation, or patient communication, final clinical responsibility must remain clearly assigned to qualified professionals and accountable institutions.

The “Wei Zexi incident” in China (in which Wei Zexi, a 22-year-old Chinese university student who died after pursuing an experimental cancer treatment promoted through misleading online search advertising in 2016) remains a powerful warning: when misleading platform incentives, weak medical oversight, and opaque information flows combine, patient safety and public trust can be severely damaged.

Practical responses include:

• Maintaining human oversight for clinical decisions. AI should support clinical judgment, not replace it. The more direct the impact on patient care, the stronger the requirement for human review.
• Define accountability in advance. Responsibilities should be allocated among AI developers, clinicians, hospital management, data owners, and regulators before adoption, not after. This includes model validation, clinical use, monitoring, error management, and patient communication.
• Position AI as augmented intelligence, not an attempt to replace clinicians. The aim should be to enhance clinical capability, improve consistency, and reduce low-value tasks while preserving medical judgment, ethical responsibility, and patient trust.

Data governance must come before deployment

AI performance depends on data quality, representativeness, and lawful access. Healthcare data is highly sensitive, fragmented, and often incomplete across systems. Clinical trial data are designed around the purpose of the clinical trial, but they may not represent real-world populations. Patient consent and secondary use rules prevent the consolidation of data.

Speed without data governance diminishes value. AI in healthcare requires not only more data, but better-controlled data: traceable, representative, secure, and ethically usable.

Practical responses include:

Building data governance before launching projects. Consent, anonymisation, encryption, access rights, retention rules, and secondary-use policies should be clarified from the outset.
Using federated learning where appropriate. Hospitals, research centres, insurers, and pharmaceutical companies may be able to collaborate on model training while keeping patient-level data within secure institutional environments.
Introducing third-party audits and breach disclosure mechanisms. Independent audits can strengthen confidence in model development, while clear disclosure rules can protect patients, regulators, and partners in case of misuse, leakage, or security failure.

Concentration must be countered with shared infrastructure

While the current ecosystem looks fragmented, the underlying economics of healthcare AI will ultimately push the market toward concentration. IT infrastructures, large datasets, and computing power require huge amounts of capital and thus may favor platform dominance.

This creates a structural tension. In the short term, the market looks dispersed and exploratory. In the long term, the scale requirements of healthcare AI may produce oligopolistic structures, with medical knowledge, digital infrastructure, and decision-support capabilities concentrated in the hands of a limited number of actors.

Practical responses include:

• Developing trusted health data spaces and shared infrastructure. Secure and governed data-access environments can allow hospitals, researchers, insurers, and industry partners to collaborate without surrendering control to a few dominant platforms.
• Promoting interoperability, open standards, and model registries. Common documentation and technical standards can make AI tools easier to compare, validate, integrate, and monitor across healthcare systems.
• Requiring independent validation and explainability reports. External validation and clear reporting on model logic, data sources, limitations, and performance can improve transparency for clinicians, patients, regulators, and payers.

Value must be defined before the tool is selected

AI can create value through cost reduction, productivity gains, improved outcomes, faster R&D, stronger safety monitoring, and new care models. But these benefits are heterogeneous and difficult to measure. Many AI projects fail because the business problem they are supposed to solve is poorly defined. The technology is deployed before the workflow, user needs, or value metrics are clarified.

AI projects should therefore begin with a precise definition of the business or clinical problem they are meant to solve. The relevant workflow should be assessed end-to-end before the tool is selected. This helps avoid technology-led experimentation and forces the organisation to define measurable outcomes.

Three categories of value can be distinguished:

• Operational AI. Short-term efficiency and productivity gains, including workflow automation, resource optimisation, administrative simplification, scheduling, triage support, and faster decision support.
• Clinical AI. Improved patient outcomes and reduced clinical risk, including earlier diagnosis, treatment personalisation, safety monitoring, reduced variability, and more consistent care delivery.
• R&D AI. Higher probability of technical success, faster clinical trials, improved target and molecule selection, better portfolio decisions, and stronger pipeline value.

Adoption must be designed around clinicians

AI adoption is constrained not only by technical capability, but also by human factors. Healthcare organisations and professionals may perceive AI as a form of surveillance, an additional workload, a legal liability, a loss of autonomy, or even a partial replacement. At the same time, there is a shortage of talent combining clinical knowledge, AI literacy, and change-management experience.

The adoption challenge is therefore not only to convince professionals that AI is useful, but also to make AI relevant to their daily work, safe to use, and aligned with professional identity.

Practical responses include:

• Co-designing solutions with end users from the outset. Clinicians, nurses, pharmacists, administrators, and patients should be involved early so that AI tools address real needs and fit naturally into workflows.
• Focusing on existing pain points in workflows. Adoption is more likely when AI reduces visible burdens such as documentation, triage complexity, scheduling inefficiencies, repetitive administrative tasks, or alert fatigue.
• Providing training and AI literacy programs across stakeholder groups. Users need practical training on how to use AI tools, how to interpret outputs, and where the limits and risks are.
• Introducing AI as optional and measurable before scaling. Pilots should allow professionals to test AI in controlled conditions, assess its impact, and build confidence before broader deployment.
• Framing AI as augmentation, not replacement. AI should be presented as a tool that supports professional judgment, removes low-value tasks, and creates more time for patient care.

Governance must not wait for regulation to settle

Healthcare AI regulation is still being built and is constantly evolving. Regulators, like everyone else, are still learning what AI is and how to use it. Waiting for perfect regulatory clarity is unrealistic. But launching AI initiatives without proper governance is risky.

The practical answer is not to wait. It is to create internal governance that is strong enough to support responsible experimentation and flexible enough to adapt as regulation evolves.

Practical responses include:

• Establishing cross-functional AI governance boards. Clinical, legal, IT, compliance, cybersecurity, data protection, and business teams should review AI projects before deployment.
• Classifying use cases by risk level. Each use case should be assessed according to its potential impact on patients, clinical decisions, privacy, safety, and business continuity.
• Prioritising low-risk, high-value pilots. Organisations should start with use cases that create visible benefits while limiting clinical, regulatory, and reputational exposure.
• Defining KPIs from the outset. Performance should be measured through safety, accuracy, time saved, user adoption, patient impact, operational efficiency, and cost-effectiveness.
• Monitoring model drift continuously. AI systems should be reviewed after deployment to ensure that performance does not deteriorate as data, clinical practices, patient populations, and operating conditions evolve.

AI Adoption Is a Transformation of Healthcare Operating Models

The key insight from these panels is that AI adoption in healthcare is not limited by imagination, but by execution. The decisive capability is the ability to translate technological promise into accountable, trusted, measurable, and accepted healthcare practice.

The biopharma and healthcare industries are already complex and knowledge-intensive sectors. AI adds another layer of complexity, but the underlying challenges are familiar to general management: governance, incentives, accountability, adoption, value measurement, market structure, and institutional trust.

Organisations that treat AI as a purely technical project will struggle to scale it. Organisations that treat AI as a transformation of the healthcare operating model will be better positioned to create sustainable value.

Ultimately, the aim should not be to deploy AI for its own sake, but to improve patient outcomes, strengthen professional decision-making, increase system efficiency, and preserve trust. In healthcare, responsible AI is not a constraint on innovation. It is the condition for innovation to endure.

Prof. Eric Bouteiller is an Adjunct Professor of Management at CEIBS. His research mainly focuses on corporate strategy and public policies for pharmaceutical and healthcare industries, cross-border investment and strategy, economic development and Asian business. 

Frequently Asked Questions

Do the lessons from China apply to other healthcare systems?

The lessons extend beyond China, but they should be applied with context in mind. The discussions behind this article took place in Shanghai, where the speed and scale of healthcare innovation make China a useful observation point. Accountability, data governance, adoption and value creation remain relevant elsewhere, even when healthcare systems and regulations differ.

What makes AI in healthcare harder than AI in other industries?

AI in healthcare operates in an environment where errors can affect patient care, data are sensitive and fragmented, and professional responsibility cannot simply be transferred to an algorithm. Adoption also touches clinicians’ workload, autonomy and liability, while regulation continues to develop. These factors make managerial governance as important as technological capability.

Where should a healthcare organisation start?

They should start by defining the business or clinical problem before selecting an AI tool. The next step is to co-design the solution with the professionals who will use it. From there, organisations can test low-risk, high-value applications through pilots, with clear KPIs established in advance to assess whether they create meaningful value.