Tomorrow Will Not Fit in One Number

A year appears on a screen. It is the year machines will supposedly pass some great boundary in intelligence. Beside it is a percentage: the chance of an extraordinary outcome, good or terrible.

The numbers look small enough to carry. A person can repeat them in an interview. A headline can hold them. Soon they begin to stand for the future itself.

But the future contains people who will make choices. It contains discoveries no one has made, failures no one has found and institutions no one has built. It contains the work we decide to fund tomorrow morning. A number can summarize a judgment about that world. It cannot release us from the responsibility of helping to shape it.

We need forecasts. A society that never looks ahead will arrive unprepared. We should make our forecasts clearer, expose them to criticism and learn which methods deserve trust.

We also need courage that does not depend on pretending to know the ending.

I reject the demand that we prove a wonderful future in advance before working toward it. No generation receives that proof. We have to act from the evidence available, improve what we can and preserve room to learn. AI gives us powerful reasons to try: the possibility of faster discovery, greater independence and useful help reaching people who have gone without it.

Serious dangers belong in this account. They should direct urgent research and strong protection. They should not be allowed to occupy the entire territory of the possible while every hope is dismissed for being uncertain.

If we can imagine how greater intelligence could bring us into trouble, we can also think carefully about how it could help us escape troubles we already know. The discipline required is the same: show the steps, examine the obstacles and explain what we should do now.

Four ways to speak about tomorrow

A forecast estimates what is likely to happen. It should name an outcome, a time and the conditions that matter. A forecast that a tool will pass a specified test by the end of next year is clearer than a claim that AI will soon change everything. The first can eventually meet an answer. The second can survive almost any event by changing the meaning of soon or everything.

A projection asks what would follow from assumptions. If computing demand grows at a stated pace and efficiency improves at another, a model can estimate future electricity use. The calculation may be sound even if the assumed path never occurs. To judge it fairly, we must keep the if attached. A conditional calculation becomes misleading when someone removes its conditions and presents the output as destiny.

A scenario explores a possible course of events. A planner might ask what happens if AI assistance becomes much cheaper while demand for the service remains weak. Another scenario might combine rapid adoption with a shortage of trained people to check the work. Scenarios help reveal vulnerabilities and opportunities. Their presence in a report does not tell us how likely they are.

A goal states what we mean to pursue. Affordable help for every learner is a goal. Reliable access to useful scientific tools is a goal. These statements should be judged partly by the quality of the plans attached to them. Calling them forecasts invites a false argument over whether success has already been guaranteed.

Confusing these forms of thought creates needless fear and false confidence. A worst-case scenario becomes the expected result. A company target becomes a delivery date. A projection based on widespread adoption becomes evidence that widespread adoption is inevitable.

The correction begins with a small question: what kind of statement am I being asked to believe?

Consider a claim that a model could perform much of the work in an occupation. The claim might concern technical ability under test conditions. A forecast of actual job loss must also consider costs, demand, the remaining work and the choices of employers and governments. Those links require evidence of their own.

The same discipline applies to a warning. A model’s dangerous behavior in a test can be an important finding. The chance of a particular real-world disaster also depends on access, exposure, defenses and what people do in response. We should investigate those links with urgency. Skipping them produces a vivid story without showing how strongly each step is supported.

A useful forecast lets us see where its confidence comes from. A weak one asks us to be impressed by the confidence itself.

The people behind the percentage

Expert judgment is valuable, especially where direct evidence is limited. It remains judgment.

In a survey conducted in 2023, Katja Grace and colleagues gathered responses from 2,778 researchers who had published in leading AI venues. Expectations varied widely. The study also found that asking about dates at fixed probabilities produced different timelines from asking about probabilities at fixed dates. Not every respondent received every question.¹

Those findings are evidence about researchers’ beliefs and the sensitivity of their answers to question design. They are not measured rates at which future worlds occur.

The distinction matters when a percentage leaves the survey and enters a speech. A share of respondents who believe an event is plausible differs from their average estimate of its chance. Either differs from the frequency of an event observed in repeated trials. Each number answers a different question.

This is no reason to sneer at expert concern. Researchers may recognize mechanisms that outsiders overlook. Their worries can identify important work long before a danger becomes common. The right response is to examine the reasons for the judgment and the actions they support.

We should ask the same of confident optimism. A researcher may understand a promising technical path very well while knowing less about hospitals, public budgets or the work of making a service widely available. The technical expectation can be informed without settling the whole social outcome.

Combining many opinions does not automatically remove these limits. A large group can bring varied knowledge. It can also share assumptions, professional incentives and sources of information. If everyone has missed the same obstacle, averaging their answers will not reveal it.

The most useful disagreement therefore concerns reasons. Which development would change your mind? What is the hardest step in the path you expect? Which observation would make your timeline less likely? Where are you relying on a judgment that cannot yet be checked?

These questions turn a prediction into an invitation to inquiry.

They also protect against a common abuse of authority. A forecast can inform a decision without granting its author the right to make that decision for everyone else. A person may be skilled at estimating a hazard and still owe the public an account of the policy they recommend. The costs, alternatives and distribution of power do not disappear inside the percentage.

Our task is to make the expert’s knowledge useful while keeping the public’s judgment alive.

What makes a prediction bad

A forecast can be wrong without having been foolish. Something unlikely can happen. Something likely can fail to happen. If we judge every prediction only by whether its favored outcome occurred, we will reward people who conceal uncertainty and punish those who describe it honestly.

Imagine a forecaster who repeatedly assigns an eight-in-ten chance to rain. Over many comparable forecasts, rain should occur roughly eight times out of ten. A single dry day does not expose a fraud. A long record in which rain occurs only half the time suggests that the confidence needs repair.

This relationship between stated confidence and observed frequency is called calibration. It gives probability a practical meaning. But a forecaster who repeats the same broad average every day may be well calibrated while telling us little about which particular day to carry an umbrella. We also want predictions that distinguish different situations well.

Research shows that forecasting practice can improve. In a two-year geopolitical forecasting tournament, Barbara Mellers and colleagues found benefits from probability training, collaboration and selecting strong performers for teams. Their study examined the accuracy of forecasts that could be scored against later events.² It does not establish that the same methods can reliably price every unprecedented AI catastrophe.

The useful habit is accountability to outcomes. Write down the prediction before the event. Keep the original wording. Record revisions and the evidence that prompted them. Compare performance with a sensible baseline.

A forecaster who updates after new evidence has arrived may be doing exactly the right thing. The question is whether the update is visible and reasoned. Quietly replacing an old prediction with a new one makes learning much harder.

Bad predictions often escape judgment because their terms move. A promised breakthrough becomes a demonstration, then a prototype, then a claim that the direction was correct all along. A predicted disaster becomes any event that can be described as troubling. There may be something worth learning from the revised claim, but the original should remain available for inspection.

Public institutions should pay for this record-keeping. Important forecasts ought to have a named owner, a date, clear terms and a review schedule. If they influence large decisions, the grounds for that influence should be available too. Prestige should not make a prediction immune to checking.

We must also distinguish updating from chasing a headline. A new model release may contain evidence that changes a forecast. Excitement around the release is not itself that evidence. A security incident may reveal a serious weakness. Its significance depends on what failed and whether the failure applies to the systems or activities being forecast.

The aim is neither stubbornness nor constant revision. It is a belief that changes for a reason someone else can examine.

Some questions will remain difficult to score for a long time. A claim about the distant future may depend on a unique event, a disputed definition or choices that change in response to the forecast. We should not discard it automatically. We should give it the weight its evidence and limits warrant, and seek nearer questions that can teach us something about the path.

A grand claim may contain several testable smaller claims. Work on those. Find out which steps survive contact with the world.

Choose work that survives surprise

Suppose the most advanced AI systems improve much faster than expected. Suppose instead that progress becomes uneven, with striking strengths and stubborn weaknesses. Or suppose useful capabilities continue to grow while the cost of fitting them into institutions remains the main obstacle.

A serious public program should be examined under all of those possibilities. It should not depend on our favorite timeline being exactly right.

Robert Lempert, Steven Popper and Steven Bankes developed methods for this kind of planning in their 2003 RAND study of long-term decisions. Their approach examines strategies across many plausible futures and looks for choices that can adapt as information improves.³ The point is to make present decisions less fragile, rather than to discover one perfect account of the coming century.

For AI, I would start with investments whose value is not tied to a single dramatic arrival date.

Stronger public research institutions can investigate useful applications, expose failure and train people able to judge claims. If capabilities leap ahead, that capacity becomes urgent. If they improve slowly, the same institutions can help separate genuine gains from expensive distraction. The work changes with the evidence; the ability to do it retains value.

Accessible education gives people more room to use new tools and to question them. Good records and clear responsibilities make public services easier to improve, whether the next advance comes from AI or from a simpler reform. Security teams that can detect unusual behavior and recover from incidents have work to do under many possible futures.

These investments cost money. Calling them useful across futures does not mean that every possible project is worth funding. A research center can become wasteful. Training can miss the skills people need. The program should be judged by the work it actually enables and repaired when it fails.

Still, the direction is clear. Uncertainty strengthens the case for the capacity to learn and act. It gives us little reason to remain institutionally helpless while waiting for a forecast to resolve.

Some choices are more fragile. A public agency might sign a long contract whose benefits depend on a model becoming capable of work it cannot yet perform. A company might remove essential staff on the assumption that a future system will fill the gap. A government might make its entire response depend on one capability threshold remaining a reliable guide to danger.

These decisions can lock in a costly bet before the evidence arrives.

A better commitment leaves options open where doing so has value. Contracts can separate work already demonstrated from promised additions. Training can build skills that transfer across tools. Public systems can preserve a workable route for changing suppliers. Resources for independent testing can grow as the systems being tested become more demanding.

This is active preparation. It requires building things now.

We should also agree in advance on evidence that would change a decision. If a tool begins failing in a consequential way, who investigates and who can limit the affected use? If repeated independent tests show that a restriction no longer serves its stated purpose, who must review it? A revisable policy needs a route for loosening a bad constraint as well as tightening a weak protection.

Without both routes, temporary rules can become permanent habits. The public may keep paying for yesterday’s fear or yesterday’s optimism long after the facts have changed.

There is a limit to learning through use. Some exposures could be too severe or difficult to reverse for ordinary trials. In those cases, investigate under tighter conditions and demand stronger evidence before expanding access. We do not need a precise civilization-wide disaster probability to take a credible, specific failure seriously.

We do need to distinguish that failure from the whole field of inquiry. Research on understanding and containing a danger should advance. So should useful work outside the affected activity. A broad freeze can remove options just when we need more ability to discover which options are sound.

The program I defend moves capability and protection forward together, with different decisions for different exposures. Its strength comes from being able to learn without making the entire future depend on one guess.

Hope deserves a serious plan

There is an odd imbalance in some discussions of AI. A long chain of imagined disasters is treated as a sober view of the future. A long chain of possible improvements is dismissed as fantasy because each improvement still requires work.

Both chains deserve examination. Neither gets to skip the difficult links.

But hope has a further role. It helps us decide which links are worth building.

Imagine a world in which a person can obtain skilled help in the language they speak, rather than the language a distant company found most profitable. Imagine researchers able to pursue neglected questions because tools and funding are no longer reserved for the richest institutions. Imagine more people recovering time from needless work and having a secure claim on the wealth their society creates.

Follow the possibilities beyond the first convenience. Someone who can understand a complex form may gain access to a service. Someone with better access may gain time to study. A person who can study while supporting a family may help solve a problem that would otherwise remain outside the reach of their life.

Progress can enlarge who gets to become a contributor.

That is a future worth imagining in detail. Its realization would require capable tools, dependable services and choices about ownership and power. Those are reasons to build a program, not reasons to be embarrassed by the ambition.

We should spend at least as much moral effort asking how a good future could become real as we spend describing how a bad one could arrive. What research would help? What institution is missing? Which people need resources now? Which bargain would allow gains to spread instead of collecting at the top?

The answers will not fit in one forecast. They will fit into budgets, experiments, laws, classrooms and the daily work of people willing to improve what exists.

A vision becomes serious when it creates obligations. If we say AI should help everyone, we owe work on access. If we say it could accelerate discovery, we owe investment in the science and the checking. If we fear concentrated power, we owe institutions that distribute power. Saying the future is uncertain does not pay any of those debts.

Broad pauses offer no escape from this responsibility. A world that develops more slowly can still be unjust, insecure and badly governed. A world held back can still deny people care and opportunity. Stopping a line on a chart does not, by itself, repair the life beneath it.

I want forecasts that make our judgment sharper. I want warnings that lead to effective protection. I want hope strong enough to become work.

We cannot promise the ending. We can choose to become more capable of reaching the future we have reason to want, and more answerable for how we try. That is enough to begin. It is also too important to keep postponing.

Sources and notes