5. The Makers and Their Myths

What are we accusing one another of?

“You are playing God” can be a warning, an insult, an expression of wonder, or a refusal to explain an objection. Its force comes partly from its uncertainty. Before answering the charge, we should ask which meaning is at work.

Sometimes the phrase means that humans are changing something once beyond their reach. Under that meaning, many acts of healing, building, and discovery would qualify. The accusation alone cannot show that the change is wrong. We need to know what is being changed, for whose benefit, at whose risk, and under what conditions.

Sometimes it means that a person is acting as though knowledge were complete. That is a different criticism. We can reasonably object to a decision made with unwarranted confidence even when its aim is good. The wrong lies in the handling of uncertainty, not necessarily in the wish to achieve something new.

Sometimes it means that someone is claiming a right over other people’s lives without a sufficient reason. Here the charge approaches the central argument of this book. Possessing an extraordinary ability does not settle whether one is entitled to use it on someone else.

And sometimes the phrase expresses a metaphysical idea: that human creativity participates in a reality larger than the separate self. This is another discussion again. It may change how a person experiences responsibility, but it does not serve as evidence that a particular invention will work or that a particular leader should be obeyed.

These meanings can slide into one another during a single conversation. An engineer describes a new capacity. A critic hears an assertion of supreme authority. A promoter replies that all progress once looked impossible. A spiritual speaker says that separation is an illusion. Everyone appears to be answering the same question, while each addresses a different one.

The result is a debate full of temperature and short of illumination.

Consider a hypothetical proposal to use AI to redesign a town’s relationship with a nearby river. The system can compare possible channels, homes, farms, and flood defenses. One resident calls the plan an attempt to play God. Another says it is the only responsible use of the tools available. Neither statement yet tells us whether the design is sound.

The resident may be warning that the model cannot capture the river’s full behavior. She may be defending a home that will be removed without consent. She may think the landscape should never be deliberately changed. Those are three different objections, requiring three different answers. It would be careless to dismiss all of them as fear of progress. It would be equally careless to accept all of them as proof that the plan must stop.

The first question is what we know. The second is who gets to decide. The third is what we have reason to value. Powerful technology makes their separation more necessary because it can make an answer to the first look like an answer to the other two.

The language of gods is useful when it draws attention to a dangerous assumption. It is less useful when it prevents anyone from identifying the assumption. If the phrase begins an inquiry, keep it. If it ends the inquiry before reasons are offered, ask for more.

The strongest case for making

A serious defense of technological ambition begins with the lives that might improve. A parent hoping that a child will suffer less does not need a fantasy of omnipotence. A scientist seeking a better way to understand disease does not need to despise nature. A person using assistance to speak, learn, or create may be expressing a basic wish to participate.

It would be morally strange to treat those wishes as evidence of arrogance simply because fulfilling them requires something new. There can be arrogance in intervention, but there can also be arrogance in telling other people that their present suffering is a necessary lesson in humility.

The strongest advocate of AI can therefore make a demanding argument. Human limits are not neutral. They include limitations that cost lives, close opportunities, and leave people dependent on the accident of where they were born. If a technology can help reduce those burdens, refusing to develop it also has consequences. Ethical judgment must examine the costs of inaction alongside the risks of action.

This argument deserves to stand in its strongest form. An inquiry into limits should not become a celebration of preventable helplessness. Nor should respect for human dignity require humans to perform every task unaided. Assistance can make agency possible. Someone who cannot undertake a project alone may become its author with the right support.

But the argument for making does not settle every proposed method of making. “We could help people” is a reason to investigate a possibility. It is not a blank authorization to expose people to whatever risk the investigation creates. The intended benefit and the means of pursuing it require separate judgment.

Imagine two teams with the same promising goal. One asks affected people which difficulties matter most, explains uncertainty, and builds ways to detect failure. The other decides that consultation will slow the work and that users can adapt after deployment. The difference is not a contest between courage and timidity. It concerns the relationship between the people building the system and the people whose lives will absorb its errors.

Speed can be a moral value. So can reliability. So can the ability to withdraw. The difficulty is not resolved by declaring one of them supreme in advance. A fast response may be needed when delay itself threatens harm. A slower process may be justified when a decision would bind people for years. Responsible ambition requires the ability to distinguish the situations.

There is also an unfairness in who gets called ambitious. A wealthy institution’s desire to expand may be described as innovation, while a poor community’s desire to protect its own priorities is described as resistance. Conversely, established organizations may use the language of caution to preserve their position against newcomers. Neither progress nor prudence should be accepted as a moral credential without examining what it protects.

The better defense of innovation is therefore a defense of possibilities for people, including people outside the institutions that make the technology. It welcomes challenges that reveal a missed need or an avoidable burden. It does not interpret every limit as humiliation. A bridge is not made less ambitious by a serious account of what it must carry.

We should want a civilization capable of making remarkable things. We should also want its makers to be able to explain whom those things serve. The capacity to answer that question is part of the achievement.

Harari, Žižek, and the danger in the story

In his conversation about the next fifty years, Yuval Noah Harari warns against treating AI as a passive instrument when it may act with considerable autonomy. He also contrasts problem-solving ability with the wisdom required to choose worthwhile goals. His image of language freeing itself from human dependence is speculative and metaphorical. It should not be mistaken for a demonstrated account of what present AI systems are.¹

The distinction between ability and purpose is valuable even without the strongest predictions. A system might become better at pursuing goals while leaving us less clear about which goals deserve pursuit. If the means grow faster than our ability to examine ends, more power can deepen an existing confusion.

Yet we should be careful with the opposing image of an uncontrollable god. It can warn us about exaggerated promises of control. It can also make design choices sound like destiny. Autonomy comes in forms and degrees. The practical question is what decisions a system can make, what resources it can reach, how its behavior is evaluated, and what happens when it exceeds its assigned role. A mythic description should not erase those distinctions.

The familiar claim that we are creating a superior intelligence that will serve us contains a real tension. Service requires limits on what a system may do. Superior competence may make those limits harder to maintain in some settings. But tension is not a proof that every useful delegation must fail. It is a reason to investigate the relationship between capability, objectives, access, and control with care.

In a debate between Harari and Slavoj Žižek, another concern comes forward. Attempts to improve humanity may favor the qualities institutions find useful while neglecting qualities important to the people being improved. The discussion also warns that apparently generous plans can conceal an expanded claim to manage everything. These are philosophical warnings about the shape of power, not evidence that a specific future must occur.²

We can develop the point through a hypothetical training system. Its designers want users to become more decisive. They measure hesitation and reward faster commitments. Many users benefit. But one user hesitates because she notices people whom the official goal has left out. Her delay may be a failure of confidence. It may also be an act of moral attention.

If the system treats every hesitation as a defect, it cannot recognize the difference. Improving the measured trait may weaken the person’s ability to resist a bad demand. The difficulty is not that decisiveness is undesirable. It is that a human quality has meaning within a larger life. Extracting it from that life can change what improvement means.

The same problem can arise with calmness, obedience, productivity, resilience, or even empathy. Empathy cultivated to understand another person differs from emotional sensitivity cultivated to sell that person something. Resilience that helps someone recover differs from training that makes unacceptable conditions easier for an institution to maintain.

The strongest reply is that these problems predate AI. Families, schools, employers, and states have long shaped human behavior around chosen ideals. That is true as a general observation. It is also why technological novelty cannot be our only moral test. What matters is whether a new capacity expands the scale, precision, secrecy, or difficulty of escape from a familiar form of power.

A technology can make an old mistake easier to repeat, but it can also make the mistake easier to expose. An intelligent assistant could help a person name the pressures operating on her, compare alternatives, and recover a neglected purpose. The argument should remain open to that possibility. Otherwise our criticism repeats the very narrowing it condemns by allowing technology only one role in the story.

The lesson from these debates is to examine the picture of the human being inside a promise of improvement. Who is the proposed better person better for? What can this person do that mattered to them before the intervention? What can they still refuse afterward? A credible account of progress must answer at least as carefully as a credible account of danger.

Watts and the limits of self-creation

An excerpt attributed to Alan Watts invites a radically different perspective. Its central image presents separate lives as expressions of a larger reality playing through many forms. It encourages a shift from experiencing oneself only as a helpless fragment toward identifying with the wider process of life.³

This is metaphysical reflection. It is not an experimental result about the universe, and it is not evidence that human intention can determine external events. Reading it well requires neither dismissing its imaginative power nor converting it into a claim it cannot support.

The image can loosen a cramped understanding of the self. A person is sustained by circumstances beyond voluntary control: other people, languages, histories, environments, and bodily processes. Seeing oneself as connected to a wider world can encourage care for that world. It can also soften the fantasy that strength means needing nobody.

The danger appears when this perspective becomes a claim that a person chose everything that happens to them. A victim of abuse did not authorize the abuse by belonging to the same universe as the abuser. A community harmed by another’s decision did not consent through some metaphysical unity. Whatever one believes about the ultimate nature of reality, differences in power, knowledge, and action remain morally relevant within human life.

This is where a language of self-creation needs boundaries. People can often influence the interpretation of an event, their response to it, and the direction they take afterward. That power can be profound. It does not imply that they created every condition to which they must respond.

Imagine a person learning a new skill after a long period of discouragement. A change in outlook may help her persist. An AI tutor may give her patient feedback she could not otherwise afford. Her commitment matters. So do the tool, the available time, and the conditions that allow her to keep practicing. Recognizing those supports does not diminish her achievement. It makes the account more accurate and may help us extend similar opportunities to someone else.

Now imagine telling a person without those supports that failure proves insufficient belief. The language of empowerment has become a device for ignoring circumstances. It protects the observer from having to ask what help is owed. That is a small but consequential form of playing God: the speaker claims knowledge of another person’s inward failure while overlooking the conditions of their life.

The opposite mistake is to reduce everyone to circumstances so completely that agency disappears. If every action is described only as the outcome of forces, people may be offered sympathy while being denied a serious place in change. A good account should preserve both truths: we act from conditions we did not choose, and our actions can still help alter what comes next.

AI enters this tension as a possible aid to agency and a possible replacement for it. A system can help people understand their situation, rehearse difficult choices, and create work beyond their previous reach. It can also encourage the belief that every unease has a technical solution and every aspect of the self should be continuously adjusted.

The promise to become anyone can become a demand never to be satisfied with being someone. At that point, self-creation ceases to feel like freedom. It becomes an endless audition before an imagined superior version of oneself.

A more humane ambition leaves space for gratitude, attachment, and limits freely recognized. A life can be open to change without being treated as a defective prototype. Participation in a world does not require omnipotence over it. Sometimes a person exercises the most important power available by deciding that a relationship deserves care more than another round of personal optimization.

Makers who can be corrected

Every account of making carries a story about the maker. The heroic story emphasizes vision: one person sees what others cannot. The managerial story emphasizes coordination: expertise organizes a world that would otherwise waste its possibilities. The spiritual story emphasizes participation: creation expresses something larger than the isolated self. The democratic story emphasizes shared authorship: those who live with a decision deserve a place in shaping it.

Each story can reveal something. Each can conceal something. Vision can become exemption from criticism. Management can turn people into entries on a plan. Spiritual participation can blur responsibility. Shared authorship can become a slogan while a few participants quietly control the agenda.

We should therefore ask what allows correction inside the story. Can the visionary admit that another person noticed a missing fact? Can the expert acknowledge that an affected person values something absent from the model? Can the believer distinguish a conviction from evidence others can examine? Can the group recognize when its procedure has excluded someone it claims to represent?

The ability to be corrected is not a minor personality virtue. It determines whether intelligence stays connected to the world it seeks to improve. A system that absorbs every objection as evidence of its own greatness has become insulated from learning, however many facts it contains.

Return to the hypothetical river project. Suppose the model identifies a technically excellent solution. During consultation, residents explain that a proposed relocation would separate older people from the neighbors on whom they rely. That discovery is not necessarily a reason to abandon the entire project. It is a reason to improve the definition of the problem.

A maker committed to being right may hear the objection as interference. A maker committed to making something worthwhile can treat it as information about success. The difference lies in what the project is for. If success means vindicating the original plan, criticism is an enemy. If success means helping people live better with the river, criticism may become part of the design.

This outlook permits firm decisions. Some objections will be mistaken. Some requests will conflict. No practical project can make every consequence reversible or obtain universal agreement. Humility cannot mean waiting until disagreement disappears, because it may never disappear. It means making the grounds of a decision visible and preserving a serious route for evidence to change what happens next.

The same discipline should apply to grand accounts of humanity’s future. We can welcome the possibility of better health, wider knowledge, and forms of creativity beyond our present experience. We can take risks for those goods. What we should resist is the claim that the greatness of the destination cancels the standing of people asked to supply the sacrifice.

No story of progress should require us to stop noticing the person in front of us. No story of limitation should require us to stop imagining what might help that person. The difficult work lies in keeping both forms of attention alive.

Who gets to play God? If the phrase means who may discover, create, repair, and enlarge the possibilities of life, the answer should be as generous as justice allows. If it means who may become unanswerable because of what they can make, the answer should be no one.

The first answer keeps ambition alive. The second keeps ambition humanly intelligible. Together they allow us to build without pretending that the act of building places the builder beyond responsibility.

Sources and notes