Reading the Evidence

A report says uncertainty has reduced investment. The shop owner puts it beside the machine quotation. It sounds like an explanation of her week. She has delayed a purchase, and she has been unsure about policy. The sentence seems to join those facts.

But she has also lost a possible customer. Her lender wants a larger deposit. The machine supplier has changed its delivery date. Which fact did the report measure? Which change did its authors try to separate from the others? Does “investment” mean the machine order, the payment, or the day the machine begins work?

These questions need not be hostile. They are how we let good research do its proper work. A study can be persuasive about one part of a problem and quiet about another. The danger comes when we borrow its authority for a claim it did not test.

It helps to begin with a plain account of the result. Who was studied, during which years, compared with whom, using what measure? Then add the size and limits of the difference. If those details disappear when a finding leaves the paper, the sentence may grow more confident just as its meaning becomes less clear.

What has been measured?

Scott Baker, Nicholas Bloom, and Steven Davis built a widely used US policy-uncertainty index from scaled article frequencies in ten newspapers. Their measure searches for economy, policy, and uncertainty terms; human audits help check what it captures. It is an index, not a probability of a rule change or a dollar loss. Their 2016 paper links the measure to weaker activity, while cautioning that the economy-wide relationships need not be causal.¹

Consider why the unit matters. If a bathroom scale rises, we know what “more” means because the instrument has a defined unit and a clear object. A news measure requires more interpretation. We need to know which kinds of articles enter, how the total amount of coverage is handled, and what comparison period sets the scale. A larger number cannot be read as a household having twice as much doubt.

Nor are doubt and disagreement interchangeable. Imagine two managers. One is sure a fee will rise; the other is sure it will remain unchanged. They disagree, but neither reports much personal uncertainty. Now imagine both managers assign similar chances to several possible rates. They agree with each other and remain unsure about the outcome. A measure of differences between their forecasts would tell a different story from a measure of the range within each forecast.

A fall in sales is different again. It is an outcome that has happened. Before the fall, a manager might have been very sure it was coming. A bad result can arrive with little surprise. If we label every unpleasant development “uncertainty,” the word stops identifying anything that can be tested.

One reason to use several measures is to see whether the account survives these differences. Yet agreement among measures is strongest when each brings fresh information. Two indexes built from much the same news are not two independent witnesses. A researcher, journalist, or reader should ask where the underlying observations came from before counting confirmations.

A better comparison needs a reason

Huseyin Gulen and Mihai Ion’s 2016 study relates a news-based policy-uncertainty index to company capital investment. The negative relationship is stronger where investment is harder to reverse and where firms rely more on government spending. That pattern fits an exposure-and-waiting mechanism. It does not establish that uncertainty alone explains a particular company’s cutback, or that all firms face the same effect.²

The appeal of company data is easy to see in the shop. We can look beyond a national total and ask whether firms with more reason to care behave differently. But a comparison has to earn our trust. Businesses that sell to public agencies may also differ in size, credit access, or order cycles. These differences could matter even in a year with little policy news.

A sound design tries to account for competing explanations. It may compare a firm with its own past, allow for events affecting everyone in a year, and examine whether the groups were already moving apart. Each step answers a particular concern. None acts as a magic seal that turns every remaining difference into the effect we hoped to find.

Take an invented comparison between two workshops. One makes ordinary replacement parts. The other makes custom parts for a new public project. The second delays its machine purchase after an election announcement. That is consistent with policy doubt. Yet perhaps the public project was already behind schedule. If we ignore the project’s timing, we may credit the announcement with a delay that was already coming.

What would strengthen the claim? Evidence that the workshops had similar prior trends would help. So would a clear account of when the unexpected news reached each firm. We would want to know whether the exposed workshop changed its plans before its orders fell, and whether other exposed firms behaved similarly. Each fact would make some competing stories less likely. We should say which stories remain.

This is also why choosing the comparison after seeing the result is dangerous. If we search long enough for a group that did well, nearly any struggling group can be made to look harmed by a chosen event. The comparison should follow from the reason some firms faced the change more directly.

Elections offer dates, not random assignment

Brandon Julio and Youngsuk Yook studied listed companies around 248 national elections in 48 countries during 1980–2005. With their controls, investment expenditures were about 4.8 percent lower in election years than in nonelection years. That is a relative percentage, not a fall of 4.8 percentage points in an investment rate. Their 2012 paper provides evidence consistent with political uncertainty affecting investment; its observational design does not make an election a randomized treatment.³

The calendar helps because it gives researchers a common event around which to compare decisions. Still, an event date can carry many changes. Parties can propose different spending plans. Banks can revise their outlook. Customers can wait too. If an election changes both the likely rule and the doubt around it, we cannot simply name the whole measured response an effect of uncertainty alone.

There is also a question about the people included. A study of listed firms can tell us much about those firms. It does not automatically describe the self-employed repairer who borrows against a home, or the new business that never opens and therefore never appears in company accounts. Missing businesses may matter greatly to the question we care about.

That does not make the observed firms the wrong ones to study. It limits the trip we can take with the result. We can use it as evidence about an important class of investment decisions. We should hesitate before multiplying it across every kind of business or turning it into next year’s forecast.

A reported average also leaves room for different paths. Some firms may delay, some may accelerate, and some may do nothing. Even if an average estimate is measured well, it will not tell the shop owner which path is hers. That requires information about her customers, exposure, and ability to reverse the purchase.

Models make their choices visible

Dario Caldara and coauthors combine trade-policy measures from news, company calls, and tariff data with firm and aggregate analysis. Their Federal Reserve research version appeared in September 2019; the journal paper followed in 2020. A two-country model separates higher expected tariffs from greater uncertainty about them. Its results depend on its structure and assumptions, so the model cannot turn every movement in a trade-policy index into a pure uncertainty shock.⁴

A model gives us a controlled place to ask a question that real events often mix together. We can hold an expected cost fixed while widening its range, or raise the expected cost while leaving its range unchanged. Outside that controlled setting, one announcement may do both. The model’s clarity is valuable precisely because the actual event is less tidy.

But every such exercise needs a bridge back to the world. How easily can firms change suppliers? Can prices adjust quickly? Can a buyer borrow through a bad period? Assumptions about these details affect what happens after the modeled shock. They are part of the answer, not fine print to be discarded when a result reaches a speech.

Imagine we build a model of the shop that allows instant resale of the new machine at its purchase price. The owner can try it and reverse the decision without loss. A model in which the machine has little resale value poses a much harder choice. If the two models predict different responses to the same uncertain rule, averaging their answers would hide the central question: which resale condition resembles the actual shop?

The same discipline applies to a counterfactual, an estimate of what would have happened under a different path. We observe one history. A model supplies the comparison history under stated assumptions. Calling the gap an “estimated effect” may be fair. Calling both histories observations would not be.

Sensitivity checks are useful when they show which assumptions can change a decision. If a modest change in resale value reverses a conclusion, we should find better resale information before recommending the purchase. If a broad range of plausible values leads to the same choice, confidence can rise for a stated reason. The point is to learn where the conclusion is fragile, rather than merely produce a large stack of model runs.

What managers planned in spring 2025

The Survey of Business Uncertainty asked nearly 1,000 executives about plans during April 14–25, 2025. In the Atlanta Fed report published on May 15, 40 percent planned to reduce hiring and 45 percent to reduce capital investment because of policy uncertainty over the next six months. These are respondents’ reported plans and reasons, not verified falls in completed spending. The question covered several policy areas, so these shares should not be relabeled as the effect of tariffs alone.⁵

An intention belongs to a date. Suppose the owner answers a survey on Monday and receives a firm customer order on Friday. Her new plan can be different without the Monday answer having been dishonest. A forecast is a statement made with the information then available. We should record that information date beside the period being forecast.

There are at least three clocks to keep apart here. The survey is collected during a fieldwork period. Its results are released later. The spending question refers to a future period. A report published in June might describe answers gathered in May about the coming twelve months. Describing it as “June spending” would quietly replace a plan with an outcome.

Even a direct question about the reason for delay cannot do all the work of a causal study. Managers know things outside observers do not. They can explain what they are waiting for. But they can also face several pressures at once, remember them imperfectly, or use the same word for different concerns. Their answers are evidence of reported reasons. We still need to check what happened.

Following the same respondents can help. Did a planned machine purchase become a signed order? Did the order become a delivery? Was the purchase reduced, canceled, or shifted into the next period? If firms leave the survey, we should ask whether their absence changes the picture. The businesses most under strain may be the hardest to reach.

None of this is a reason to throw away timely surveys until years of accounts arrive. A worker deciding whether to accept a temporary shift cannot wait for a final academic paper. The right response is to use the early evidence at the strength it has, and say what later evidence could confirm or change it.

For the shop owner, a disciplined reading ends with a bounded sentence. Evidence from several methods supports the view that unresolved policy can change investment timing. It does not supply a single discount to apply to every machine quotation. Her decision still needs its own facts.

For a public official, the next sentence should be a question about responsibility. Which decisions can the institution help people make now, and which uncertainty must remain because a legitimate choice has not yet been made? Research can expose costly delay. It cannot decide whose claims deserve to be heard by pretending that the economic estimate has already settled the moral question.

The report stays beside the quotation. It is more helpful now that its claim is smaller and clearer. The owner can use it to challenge an easy story about her business, ask for missing information, and explain why she is waiting. She does not need it to know the future. She needs it to improve the decision she can actually make.

Source notes