Skip to reading
Forecaster
Complete research draft15 chapters plus introduction, conclusion and reader guide published
Contents Introduction

Introduction

Introduction. Before the Answer Arrives

A useful assistant can help a person speak, while an institution still gives them nowhere to be heard. The introduction asks what turns technical ability into legitimate authority.

By Kevin L. Michel11 minute read

Imagine a father at a kitchen table trying to understand a letter from his daughter’s school. He speaks the language in which it is written, but the formal terms are unfamiliar. A deadline appears near the bottom. He asks a program to explain the letter in ordinary words, then uses it to prepare questions for the meeting. When he arrives, he is ready to take part in a conversation that might otherwise have left him silent.

This is an imagined situation. It shows a modest use of powerful technology: a person gains the ability to understand an institution and speak to it. There is no need to call the program wise or conscious to recognize the value of what has happened.

At the meeting, however, he learns that a prediction has already placed his daughter on a restricted educational track. Her earlier absences count against her. There is no field in the record explaining that the family had been moving between temporary homes. The staff can show him the recommendation. They are less certain who may change it.

The father now has the words to raise a question, but the institution has not provided a clear place for that question to go.

The whole scene is hypothetical, including the school’s use of prediction. It presents a conflict worth examining before it becomes someone’s ordinary experience. Technology can increase a person’s ability to participate while an institution uses technology in a way that reduces the practical value of participation. Access to an intelligent assistant does not, on its own, amount to access to a fair decision.

We need to know more than how intelligent the software is. We need to know what the institution has asked it to do, what evidence supports that use, who stands behind the result, and what happens when the person affected says something is wrong.

The machine may be the newest part of the story. It is not the whole story.

The question hidden inside the title

To ask who gets to play God is to enter a crowded room. Some people hear a warning against human pride. Others hear an excuse to stop research. Some think of a doctor saving a life. Others think of an official deciding which life deserves to be saved. The phrase can express religious conviction, moral alarm, scientific ambition, or irritation with anyone who claims the right to decide for others.

It can also end a conversation before the difficult thinking begins. If every effort to alter the conditions of life is treated as an offence against the natural order, there is little room left for medicine or public health. If every advance is treated as its own moral permission, there is little room left for consent or restraint. Neither habit gives us a satisfactory way to live with powerful tools.

This book takes the title as a question about authority. Who may turn an ability into an action that governs someone else’s life? On what grounds? Within what limits? With what obligation to answer for the result?

The title is not a claim that human beings are becoming divine. It does not settle whether God exists, what God intends, or whether any machine could ever possess an inner life. Those questions matter to many readers, but the immediate problem can be stated without resolving them. People can acquire powers over other people that exceed the consent, knowledge, and institutions available to govern those powers.

That is the gap this book explores.

The word “gets” deserves attention. Sometimes it means has the capacity. Sometimes it means owns the equipment. Sometimes it means receives permission. Sometimes it means wins a contest and therefore takes control. Those are different routes to power. A society that treats them as interchangeable may discover that the people who can act fastest have quietly become the people who decide what everyone else must accept.

Consider an engineer who can improve a system used to allocate a scarce resource. Her skill gives her good reason to be heard about what the system can reliably do. It does not, by itself, give her sole authority to decide which claims deserve priority. Consider a minister who has the legal power to approve its use. His office gives him a defined role. It does not make his judgment infallible or eliminate the need for evidence. Consider a religious leader who insists that every person has dignity. That claim can illuminate the decision. It does not provide all the technical answers or settle every disagreement in a plural society.

Each may contribute something necessary. None automatically possesses everything required.

One of the central arguments of this book is that we should stop looking for a single kind of greatness that entitles its holder to rule the whole problem. Intelligence matters. Expertise matters. Experience matters. Democratic authorization matters. Moral imagination matters. So does the testimony of the person who has been misclassified, excluded, or ignored. Responsible institutions make these forms of knowledge meet without pretending that they are the same.

Why begin with a pope?

Pope Leo XIV provides a point of entry because his public interventions ask what the technological age is doing to the human person. His remarks belong to a religious tradition with its own account of creation, dignity, work, community, and the common good. The first chapters examine that account and the historical connection he draws between the present transformation and the industrial questions associated with Leo XIII. They also examine what readers outside that tradition can accept, challenge, or restate in their own terms.

A book that takes the Pope seriously need not treat him as beyond criticism. Respectful attention is different from obedience. A reader may agree with his concern about exclusion while rejecting his theology. Another may share his faith but disagree about an institutional response. A third may think religious institutions should examine their own uses of power before advising others. Those positions belong in the conversation.

The inquiry is stronger when it permits them.

The same rule applies to technology leaders and public intellectuals. A builder may understand possibilities that critics underestimate. A critic may notice forms of dependency that builders have incentives to overlook. A philosopher may reveal confusion in the words we use while offering little practical help with a procurement contract. A scientific achievement may transform what we can know without answering who should own its benefits.

We should listen closely without surrendering judgment.

The materials behind this book bring together voices that do not fit neatly into opposing camps. They include arguments about evolution, mortality, consciousness, industrial ambition, religious meaning, and the future of political power. Their value lies partly in their friction. Taken seriously, they make it harder to pretend that the only choice is between uncritical enthusiasm and permanent alarm.

But a book cannot merely arrange impressive people around a table and call that an argument. The chapters that follow make distinctions, test claims, and offer proposals of their own. Where they move beyond a source, that movement is the responsibility of this inquiry. A recommendation in these pages should not be mistaken for a statement by the Pope, a research finding, or an existing legal requirement.

The price of saying yes, and of saying no

There is a moral cost to refusing useful technology. A preventable delay can be an injury. An inaccessible service can remain inaccessible while committees admire the complexity of the problem. A teacher can continue to lack resources, a scientist can continue to face a bottleneck, and a person with a disability can continue to be denied tools that expand independence.

It is easy to describe these losses as the absence of progress. For the person bearing them, they are part of the present.

There is also a moral cost to treating promised benefits as sufficient evidence. A demonstration can be impressive while a deployment is unreliable. A system can improve an average while making a smaller group worse off. An institution can use the language of access to justify a cheaper service from which meaningful human attention has been removed. A new capability can arrive together with a contract that leaves its users unable to leave.

The task is to make these costs visible enough to compare. That means asking about the real alternative. Sometimes the alternative to a proposed system is an excellent human service. Sometimes it is a badly functioning process. Sometimes it is no service at all. The assessment changes with the facts. Invoking an imaginary perfect human institution makes technology seem worse than it is. Invoking an imaginary perfect machine makes it seem better than it is.

We owe people a comparison with the world they actually inhabit and with improvements that can realistically be made.

Even that comparison has limits. A more efficient process does not automatically justify every method used to achieve it. A service could become faster by removing a group’s right to challenge mistakes. A workplace could become more productive by making private life continuously observable. In such cases, the decision involves more than a net total of benefits. It concerns the terms on which people are being asked to live.

This is why dignity, consent, and accountability are not ornamental words added after the calculations. They identify parts of the problem that a calculation can otherwise conceal.

The argument will sometimes support faster experimentation. At other times it will support a firm boundary. Often it will support a narrower use, a better comparison, an alternative route, or a requirement that someone with real authority be available to correct an error. The appropriate response depends on the decision and its stakes. A general posture of excitement or fear cannot do this work for us.

Following power through ordinary life

Artificial intelligence is a broad label for different systems and practices. The software that produces a fluent answer, the model that classifies an image, and the system that helps control a physical process need not have the same design or limitations. Throughout this book, the relevant question is what a particular system is doing in a particular setting. Treating AI as one creature with one intention would obscure the choices made by designers, owners, users, and institutions.

Nor does the book assume that every harmful automated decision is produced by a recent generative model. Older statistical systems and simpler rules can exercise considerable power. The moral importance of a decision does not rise and fall with the sophistication of the technology making a recommendation.

The chapters begin with the person and the claim to authority. They then examine intelligence, creation, and mortality before turning to care, scientific discovery, and work. The inquiry moves from there to surveillance, influence, and war, where distance between the decision maker and the affected person can become especially dangerous. The final chapters ask what legitimate institutions might look like, including in smaller countries that are often expected to adopt technologies whose terms have been set elsewhere.

This movement matters. Discussions of AI often begin with exceptional futures and work backwards toward ordinary life. Here, even the largest philosophical questions must eventually face someone trying to obtain care, earn a living, learn, speak freely, or get a mistake corrected. An account of human dignity that cannot reach those settings is incomplete. So is an account of innovation that cannot explain who receives its gains.

Several distinctions will recur. Prediction is not judgment. Performance is not proof of consciousness. Consent is not merely the presence of an agreement. A human signature is not necessarily human control. And being legally permitted to do something does not exhaust the question of whether it is right.

These are tools for thinking, not answers to be applied without effort. A prediction may contribute to excellent judgment. A carefully designed authorization process may provide meaningful consent. A human reviewer may exercise real control when given sufficient time, information, and authority. The point is to examine the conditions under which the connection holds.

The chapters use imagined situations to make those conditions visible. Such examples are identified as hypothetical. They are not offered as substitutes for empirical evidence. A thought experiment can reveal a conflict between values or expose a hidden assumption; it cannot tell us how often a real system fails. Research can establish a pattern or test a claim; it cannot, without further argument, decide which tradeoffs a community should accept.

Good judgment needs both kinds of work and the discipline to keep them distinct.

A future we can still discuss

The most discouraging account of the future is not necessarily the darkest one. It may be the account in which no meaningful choice remains. Progress is said to have a fixed direction. Competition is said to excuse every decision. Complexity is said to make public understanding impossible. The only role left for ordinary people is to adapt.

That story should be questioned even when it arrives in confident language.

People will not control every consequence of the technologies they create. They never have. But incomplete control does not imply the absence of responsibility. We can choose the purposes of institutions, the evidence required for particular uses, the distribution of decision rights, and the means by which mistakes become visible. We can preserve options for later correction. We can refuse arrangements in which the benefits belong to one group while the burden of proving harm falls entirely on another.

Those choices are neither effortless nor guaranteed. They involve disagreement, money, patience, and the possibility of failure. They also offer a more useful basis for hope than the promise that a superior intelligence will resolve the difficulty of living together.

Greater intelligence may help us understand more of the world. It may make discoveries available that would otherwise remain beyond reach. It may relieve people of tasks that consume time without giving life much meaning. These are possibilities worth working for. The achievement will be larger if it also expands people’s ability to shape the conditions under which they live.

The question is therefore not simply whether humanity should exercise more power. We already face choices about how power will be exercised, withheld, distributed, and challenged. Refusing to discuss those choices does not leave them unmade. It leaves them to whoever is already in a position to act.

Who gets to play God?

No person or institution should receive unlimited authority merely because it possesses an extraordinary capability. What we may grant instead is a bounded responsibility: to do a particular thing, for defensible reasons, on evidence that can be examined, under terms that leave the people affected with a place in the decision.

That sounds less magnificent than omnipotence. It may be a much more demanding achievement.