Limits of Predictability in Civil Litigation

Abstract

Lawyers, insurers, and litigation funders forecast how civil lawsuits will end, and legal theory distinguishes easy cases, in which the law determines the result, from hard cases, in which it does not. Yet there is little evidence on how accurately the outcome of a lawsuit can be forecast from the court record while the case is pending. We provide such evidence at scale. Using 835,190 filings and orders from 102,721 federal and state civil cases in 12 U.S. states, we estimate at each filing the probability that a case ends in a plaintiff win, a plaintiff loss, or a settlement, using only the record available then. The forecasts draw on structured information extracted from each filing, on comparable earlier cases and how they were resolved, a comparison practitioners make from experience and language models now make systematic, and on case histories of judges and law firms. We measure how predictable a case is by how concentrated its forecast is on a single outcome. Predictability varies across disputes: tort cases are the most predictable, and state-court cases are more predictable than federal ones, whereas business and corporate, intellectual-property, and property disputes and cases involving corporate parties are the least. Overall, 61 percent of forecasts are correct, rising to 95-97 percent when a win or a loss is given more than 90 percent probability. Richer information on facts, judges, and counsel improves forecasts for predictable cases but adds little for unpredictable ones. Holding case length fixed, forecasts do not sharpen as the record grows, and longer cases are harder to predict from the outset. Settlement is most frequent at intermediate predictability. The results provide a benchmark for how predictable civil litigation is, lay the groundwork for an empirical approach to legal indeterminacy, and show where forecasts could narrow information gaps between litigants.