By Mai Ke

On May 25, 2026, the Vatican released Magnifica Humanitas (“The Magnificence of Humanity”), the first encyclical of Pope Leo XIV. Focusing on human dignity in the age of artificial intelligence, the document reflected growing global concerns about the social and ethical implications of increasingly powerful technologies. Standing beside the Pope during the announcement was Chris Olah, co-founder of Anthropic and one of the architects behind the Claude family of AI models.

While Pope Leo XIV warned that AI could become an instrument of domination, exclusion, and even destruction, Olah highlighted a challenge that extends beyond technological capability: AI safety cannot be left solely to the organisations developing these systems.

Together, their remarks point toward a pair of questions that are becoming increasingly urgent: what does it mean to be human in the age of AI? And, as artificial intelligence becomes increasingly capable of replicating aspects of human cognition, what role remains for human judgment, responsibility, and moral accountability?

In looking for answers, we can turn to two and a half years ago, to the debate surrounding OpenAI in late 2023.

The OpenAI episode and the limits of corporate governance

In November 2023, OpenAI's Board of Directors temporarily removed CEO Sam Altman from his position, citing concerns that he was not “consistently candid in his communications”. The decision triggered an extraordinary response. Hundreds of employees signed a letter expressing their willingness to leave the company unless Altman was reinstated. Within days, Altman returned as CEO, while the board members who had dismissed him stepped down.

What appeared to be a corporate power struggle was, in fact, something more revealing: a demonstration of how power operates in modern organisations.

The central issue was not whether Altman was trustworthy.

In organisational life, trust is never absolute; it is continuously negotiated—between words and actions, between institutional narratives and decisions made under pressure. The OpenAI controversy highlighted the vulnerability of governance models that were designed for a different era.

OpenAI was originally established as a nonprofit organisation dedicated to ensuring that artificial general intelligence would benefit humanity as a whole. The nonprofit board was intended to serve as a safeguard against purely commercial incentives and to represent long-term societal interests. From a governance perspective, the board's intervention reflected precisely the role it had been created to perform.

Yet, in practice, it failed. The reason was not necessarily a failure of judgment, but a mismatch between formal authority and actual power. Investors, employees, and industry partners—an ecosystem built around OpenAI’s success—held much greater power than the formal authority of the board. While the board possessed legal authority, Sam Altman held the actual leverage.

This imbalance is not unique to OpenAI. Similar dynamics have emerged repeatedly throughout the history of innovation-driven organisations.

In organisations built around a singular vision and a charismatic leader, formal accountability gradually erodes. Employees don’t merely work for a company; they work for a mission. Over time, that mission becomes inseparable from the individual who leads and embodies it.

Organisational scholars describe the following phase of an organisation’s development as a collapse of psychological safety. This process rarely occurs abruptly; rather, it unfolds quietly through a series of subtle adjustments. Questions remain unasked. Concerns are raised but not revisited. Certain topics become implicitly off-limits. In fast-moving environments where speed is celebrated and skepticism is viewed as an obstacle, these dynamics can emerge with remarkable speed.

Ironically, organisations that publicly champion openness and psychological safety are often those most vulnerable to losing it. Indeed, the language of openness can become part of an organisation's identity even as the conditions that sustain it gradually disappear.

The consequences of suppressed dissent are rarely immediate. They emerge gradually.

What are the costs? Decisions receive less scrutiny, risks remain insufficiently examined, and information reaching senior leadership becomes increasingly filtered. By the time the resulting problems become visible, they have often become deeply entrenched over several years.

For a software company, these dynamics are extremely significant. For organisations developing technologies that may fundamentally reshape economies, institutions, and societies, the stakes are considerably higher.

This challenge raises a broader governance question. Are existing institutions—including boards, regulators, professional norms, and market mechanisms—equipped to oversee organisations operating at unprecedented scales of influence and speed? Increasingly, the answer appears uncertain.

There is, however, a more optimistic interpretation of the OpenAI story.

Sam Altman is widely regarded as a highly capable leader: a visionary able to articulate complex ideas with exceptional clarity. OpenAI’s technological achievements are remarkable, and the employees who supported him are among the most talented in the industry.

However, ability and accountability are fundamentally distinct.

History repeatedly shows that highly capable leaders often remain trusted long after they should have been more rigorously questioned—not because they act in bad faith, but because the systems surrounding them are not designed to challenge them effectively.

OpenAI’s board attempted to do so. It did not survive the attempt.

The more troubling question concerns what follows.

How can we govern organisations that have outgrown the structures intended to regulate them?

Answering this requires confronting an assumption we often take for granted: that those most capable of building the future are also best positioned to determine its direction.

Moral boundaries under pressure

This issue extends beyond any single leader.

My own research has found that highly creative individuals are more likely to justify ethically ambiguous decisions—not through deception, but by reframing their actions as innovation or the pushing of boundaries. When organisations encourage creativity while imposing strong performance pressure, this tendency can intensify. Employees may engage in “creative misconduct,” finding novel ways to violate ethical norms. Moreover, other scholars have identified that when individuals perceive their creative contributions as rare or indispensable, a sense of entitlement may emerge, further increasing the risk of unethical behaviour.

Together, these findings point toward a common pattern. People engaged in highly creative or transformative work may gradually become vulnerable to ethical blind spots. Once a sense of exceptionalism emerges, whether expressed through beliefs about personal talent, organisational rules, or historical significance, ethical boundaries can begin to shift in subtle but consequential ways.

For technology companies at the forefront of AI development, these dynamics are amplified. The more significant the mission is, the easier it becomes to rationalise exceptions. The greater the freedom required for innovation, the more fluid ethical boundaries become.

This challenge becomes even more complex when AI itself is increasingly deployed in contexts involving ethical judgment. At their core, these efforts can appear fundamentally at odds with the freedom that innovation requires. Society’s development depends on individuals willing to challenge conventions and push technological boundaries. Yet it also requires safeguards to ensure that, in the process of breaking old rules, they do not inadvertently dismantle the ethical boundaries on which society depends.

The risks of outsourcing moral judgment to AI

Recent research published in Proceedings of the National Academy of Sciences highlights an important warning. Comparing AI-generated moral judgments with responses from more than 90,000 participants across 48 countries, researchers found that leading large language models such as ChatGPT systematically overrepresented moral concerns common in Western societies while underrepresenting values prevalent in many non-Western cultures. The pattern appeared consistently across multiple AI models and persisted even when questions were asked in local languages.

These biases stem from the data on which such models are trained. Originating from Western sources, AI models often interpret underrepresented cultures through assumptions derived from Western perspectives. Rather than eliminating human prejudice, AI reproduces and amplifies it at scale while presenting their outputs.

In other words, if we use AI to guide ethical decision-making, whether in content moderation, policy recommendations, or corporate ethics decisions, we should be careful not to mistake its reflections for an accurate or universal representation of ourselves.

Accountability as essential infrastructure

While companies are deploying AI at unprecedented speed in human resources, healthcare, legal systems, and more, accountability mechanisms are lagging far behind.

Current approaches rely heavily on broad ethical principles, voluntary commitments, and reactive regulatory responses. While these efforts represent important first steps, they remain insufficient for technologies whose capabilities and applications continue to evolve at a rapid pace.

Effective AI governance requires progress in several areas.

First, organisations must actively cultivate environments in which dissent is encouraged rather than discouraged. Psychological safety should not be treated as a cultural aspiration alone; it must be supported through institutional mechanisms such as independent ethics reviews, anonymous reporting channels, and effective whistleblower systems for those who raise concerns.

Second, governance structures must reflect contemporary realities of power. The OpenAI case demonstrated that traditional corporate governance frameworks can become ineffective when informal influence is distributed through networks of investors, employees, partners, and public stakeholders. We need to rethink who is qualified and capable of holding organisations truly accountable in the AI era.

Third, moral responsibility cannot be delegated to algorithms. AI can assist decisions, but they cannot replace human judgment, especially across cultures and value systems. Outsourcing moral judgment to systems that already carry cultural biases is not an improvement in efficiency, but a transfer of responsibility—one that is often opaque and difficult to trace.

Finally, public discussions of AI require greater nuance. Today’s AI narratives often swing between technological optimism and existential fear. Neither helps us make practical, everyday decisions. What we need is a more balanced understanding of both benefits and risks.

Conclusion

The questions raised by the OpenAI episode extend far beyond a single company or a single leader to the relationship between trust, power and accountability in an era when organisations can shape societies at a pace that exceeds the capacity of traditional institutions to respond.

This brings us back to remarks made by Chris Olah following the release of Magnifica Humanitas. In a particularly revealing reflection, he observed that many of the world's leading engineers are increasingly looking beyond computer science—to philosophy, ethics, religion, and other enduring traditions of human inquiry—for guidance on AI alignment. That shift is telling in itself. Ultimately, regardless of how algorithms evolve or how sophisticated our tools become, it is people—not machines—who determine who we are, what we value, and the direction in which we are headed.

Artificial intelligence may be capable of simulating human intelligence with extraordinary effectiveness. It cannot, however, substitute for conscience. For this reason, the most important safeguard in the AI era is not technological, but human. The long-term success of AI will depend not only on advances in engineering, but also on our ability to strengthen the institutions, norms, and forms of accountability that allow innovation to serve broader societal interests.

Ultimately, this is not just a story about a CEO. It’s about all of us. The central challenge is not whether AI will become more intelligent, but whether human beings will remain sufficiently responsible to govern the technologies they create.

Mai Ke (Michael) is Associate Professor of Organisational Behaviour at CEIBS. His research examines the intersection of business ethics, technology, and human creativity inside organisations. He has published in Nature Human Behaviour, Nature Mental Health, the Academy of Management Journal, the Journal of Applied Psychology, Organizational Behavior and Human Decision Processes, Personnel Psychology, the Journal of Management Studies, and Harvard Business Review, among others.

Reference：

Company Announcement (2025.5.5) Anthropic co-founder Chris Olah's remarks on Pope Leo XIV's encyclical "Magnifica humanitas."

Mai, K. M., Ellis, A. P., & Welsh, D. T. (2015). The gray side of creativity: Exploring the role of activation in the link between creative personality and unethical behavior. Journal of Experimental Social Psychology, 60, 76-85.

Mai, K. M., Welsh, D. T., Wang, F., Bush, J., & Jiang, K. (2022). Supporting Creativity or Creative Unethicality? Empowering Leadership and the Role of Performance Pressure: KM Mai et al. Journal of Business Ethics, 179(1), 111-131.

Vincent, L. C., & Kouchaki, M. (2016). Creative, rare, entitled, and dishonest: How commonality of creativity in one’s group decreases an individual’s entitlement and dishonesty. Academy of management journal, 59(4), 1451-1473.

Zewail, A., Figueroa, A., Graham, J., & Atari, M. (2026). Moral stereotyping in large language models. Proceedings of the National Academy of Sciences, 123(10), e2519941123.