Ideal Points from Supreme Court Agreement

The test

Everything here comes from the Supreme Court Database, begun by the political scientist Harold Spaeth and maintained at Washington University in St. Louis, where anyone can download it: every justice’s vote in every case the Court has decided since 1946.¹ This brief reads the version with one row per justice per case (the database also comes with one row per case), 2024 release 01. The agreement counts and the scaling use the 2020 to 2023 terms — a term is the Court’s working year, which opens on the first Monday in October and is named for the year it opens — the window with the current nine justices in it.

The Court keeps the same kind of record Congress does, one flight up, and the roll-call chapter sets out what that kind of record permits. Every decision says how each named justice voted. Nobody self-reported it and nobody could opt out, because the vote is the act. Ideal-point scores are then manufactured from the pattern of agreement, exactly as they are for Congress, and a court of nine is small enough that the machinery can be checked by hand.

Choosing this file rather than an easier one is most of the brief’s argument. Martin-Quinn scores are published, updated every term, and would answer where does each justice sit in a single download.² They are not used here, for one reason: a published score is the answer, and this brief is about how the answer gets made.

The data

The file’s grain is one justice’s vote in one case, and its most important column is not an observation. direction records whether a vote came out in the conservative or the liberal direction. That is a coder’s reading of a written opinion against a codebook, a rule specifying issue area by issue area which outcome counts as which. In a criminal case a vote for the defendant is liberal, and in an economic case a vote for the government usually is.³ Where the codebook supplies no rule the field is left blank.

Read one case through the rule. Borden v. United States, decided in the 2020 term by 5 votes to 4, asked whether a crime committed recklessly counts as a “violent felony” under a federal sentencing law; the Court said it does not, and Borden’s longer sentence fell with it.⁴ The database files the case under criminal procedure, where the codebook’s rule is that the side of the accused is the liberal one. So the 5 votes for Borden — Thomas, Kagan, Gorsuch, Breyer, Sotomayor — each enter the file as liberal, and the 4 votes against as conservative. The majority is the side that won and the dissent the side that lost, and the rule reads the outcome, not the justice: Thomas and Gorsuch, who sit at the conservative end of every ordering below, each pick up a liberal vote here.

direction is not the exception in this file. Issue area, the constitutional provision at stake, which party prevailed, the reason the Court took the case: each is likewise a person’s reading of a document, entered against a rule somebody wrote. The Supreme Court publishes opinions. It publishes no data. Everything countable about it was made countable by somebody outside it, decades after some of the cases were decided.

The cleaning is short and consequential. Only the chosen terms are kept, and every case whose direction is blank is dropped from the agreement counts, because “voted the same way” has no meaning when there is no direction to compare.

Four small tables come out of it. justices.csv gives each justice a cases count, a pct_conservative and a pct_in_majority. agreement.csv has one row per pair of justices, with a and b, the number of shared cases, the number of times they agree, and the pct. close_cases.csv lists the one-vote decisions with the majority_bloc named. by_term.csv runs pct_conservative back to 1946.

The unit of analysis — the thing one row stands for — moved twice to reach that pair table: from a justice-vote, to a case, to a pair of justices. Nothing in the source file is a pair. The pairs were built by taking every justice against every other, finding the cases in which both cast a directional vote, and counting the matches. Keeping cases beside agree, rather than reducing the two to a percentage on the spot, is the one build decision that does real work below.

10 justices sat in this window, and 45 pairs can be made from them. One of the 45 is not a number. Jackson and Breyer never sat on a case together, because she replaced him, so their agreement rate is undefined rather than high or low. No amount of additional data will fix that, because the data that would fix it was never generated.

Before you read on, commit to a count. The scaling below cannot take a hole, so the missing pair gets filled with the mean agreement across every other pair: an invented number sitting in the middle of the matrix the whole ordering comes out of. Now take it away. Drop both justices and re-run the scaling on the 8 who overlap completely. How many of those 8 end up somewhere other than where the filled version put them? Commit to a number, and to one more answer. Whichever way the individual ranks fall, do the two blocs survive intact?

What it says about democracy

The raw material for any ideal-point score is embarrassingly simple. How often does each pair of people vote the same way?

Kavanaugh and Roberts agreed 96.2% of the time. At the other end, Thomas and Sotomayor agreed 53.0%, barely better than a coin. That is the whole dataset: 45 numbers, one per pair. No ideology has been supplied to it. Nobody has told this table who is liberal, what any of the cases were about, or even that the people in it are judges.

Hover any cell for the pair and the number. The second button re-sorts the same 45 numbers into the order recovered further down.

Figure 1. All 45 agreement rates, in the order the file is in and in the order recovered from them. A matrix fits this question because there is no summary to plot yet: the data is every pair against every other pair, and the point is whether structure is already visible in it. Every cell is one pair, darker where the two voted the same way more often, on an absolute scale from 53.0% to 96.2%. The red-outlined cell is the pair with no rate at all. Re-sorting changes nothing about the numbers.

Two things in that picture were not put there. There is a block structure, a set of justices who agree with each other far more than with anybody else. And there is one cell with nothing in it.

Now the move that makes every ideal-point score work. Treat agreement as closeness and disagreement as distance, then ask what single line would arrange these 10 people so that people who agree sit near each other. Follow it for three of them. Roberts and Kavanaugh agree 96.2% of the time, so the distance between them is one minus that, 0.038. Roberts and Sotomayor agree 65.4%, a distance of 0.346, so on the line Sotomayor has to sit roughly 9 times further from Roberts than Kavanaugh does, on one side or the other. The arithmetic that finishes the job is called multidimensional scaling: it looks for the positions on a single line that reproduce all 45 distances as nearly as they can be reproduced, and the column below is where those positions fall.

Read the order. Thomas, Alito, Gorsuch, Barrett, Roberts, Kavanaugh, then a gap, then Jackson, Kagan, Breyer, Sotomayor. That is the ideological ordering of the Roberts Court as any court reporter in America would give it, and it came out of 45 agreement rates and nothing else.

One warning governs every number in that column. The method recovers distances, not coordinates. Multiply every position by −1 and every pairwise distance is unchanged, so the fit is unchanged. The order and the spacing mean something; the direction does not. That is equally true of Martin-Quinn, of NOMINATE, and of every ideal-point estimate anybody will ever quote at you.

Set the recovered position beside each justice’s share of conservative votes, a column the scaling never touched, and the two line up almost exactly. The correlation — a number between −1 and 1 that says how closely two columns move together, with 1 meaning perfectly in step — is 0.975 in size, and its size is the finding. Its sign, by the paragraph above, is not. Nor is the line evenly filled: the 6 justices on one side span 0.145, while the single step across the middle is 0.224, larger than that entire bloc. This is two clusters with a chasm between them, not a spectrum.

Now the number you wrote down.

2 adjacent pairs change places: Thomas and Alito; and Roberts and Kavanaugh. Nothing else moves at all. Most readers commit to one of the two extremes, that removing an invented cell changes nothing, or that it brings the whole ordering down. Both are wrong, and 4 positions move out of 8, every one of them between neighbours.

Figure 2. The same Court, scaled with the guess and without it. A dumbbell fits this comparison because the question is asked of each person separately. One row per justice joins the position the filled scaling gave them to the position they get once the invented cell is removed. Jackson and Breyer are absent from the second run, because refusing to guess means dropping them. Each run sets its own scale, so read the order down the rows rather than the length of any one rule. The blocs and the chasm sit in the same places in both.

The result cuts two ways at once. The recovered order is not a fact about the justices at fine resolution. Adjacent positions moved because of a decision about a cell that contains no data. Anybody reporting which of Thomas and Alito sits further out is reporting a fill choice as much as the Court’s behaviour.

And the blocs do not move. The correlation with conservative voting comes out at 0.990 in size on the 8 justices with no guess in the matrix at all, slightly stronger than the 0.975 obtained with one. The coarse structure is in the data. The fine structure is in the assumptions.

Published scores make dozens of choices of this kind. Martin-Quinn makes distributional assumptions and lets positions drift over time. NOMINATE fixes a scale and a functional form. The difference is not that this one has assumptions and those do not. It is that this one is visible, and reversible in a single line.

One last word the line does not mean. Kavanaugh is in the majority 95.5% of the time, and the lowest figure is Sotomayor at 70.7%. The justices nearest the middle of the recovered line win most often and those at either end lose most often. “Moderate” here is not a description of anybody’s opinions. It is a description of how often you are on the winning side, and it follows mechanically from where you sit.

What this brief cannot tell you

Agreement is not agreement about reasons. Two justices can reach the same result from incompatible doctrines, and the reasons are most of what law professors argue about. Nothing in a vote count reaches them.

The recovered dimension may not be ideology. One dimension was chosen and found to fit well, which is not a discovery that judicial disagreement is one-dimensional. Something else that moves with ideology would fit just as well: deference to precedent, institutional caution, the identity of the litigants. Agreement rates cannot settle that.

“Conservative” is a coder’s judgment, applied case by case for 78 years by people following a codebook. A case coded conservative in 1962 may have almost nothing in common with one coded conservative in 2022. That is not a reason to discard the series. It is a reason to say what it is.

The scaling is also a snapshot, in which nobody can move. It cannot show a justice drifting, which is precisely what Martin-Quinn was designed to capture. And the Court chooses its own docket, so the cases these rates are computed over were selected by the nine people being measured.

What you should have learned

• A left–right ordering of the Supreme Court can be recovered from 44 agreement rates that carry no information about ideology, cases or appointing presidents.
• That ordering lines up with a column the method never saw, at a size of 0.975, and the size is the finding while the sign of the axis is an arbitrary convention.
• The Court is not a spectrum on this measure: the 6 justices on one side span 0.145, and the single step across the middle is 0.224.
• One of the 45 pairs never sat together, and the guess filled in for it moves 4 of 8 positions. The blocs survive the refusal; the fine ranking does not.
• Being in the majority 95.5% of the time is a fact about where you sit on that line, not a fact about anybody’s moderation.

Extensions

None of these is answered above, and every one can be answered from the tables the Sources section hands over.

• agreement.csv gives every pair a pct. Sort it and mark the blocs by hand. Do they match the labels the newspapers use, and which pairs sit in neither?
• justices.csv has pct_conservative and pct_in_majority. Plot one against the other. Is being on the winning side the same thing as being in the middle?
• close_cases.csv lists the divided decisions with the majority_bloc named. Count how often each justice’s name appears in that column. Who actually decides the close ones, and is it the same person the all-cases figure names?
• by_term.csv runs pct_conservative back decades. Follow it. Does the Court move gradually, or in steps when its membership changes?

Two stretch ideas point past the spreadsheet. Martin-Quinn scores are published for every term since 1937; find the latest release and ask whether its ordering of the current Court matches the one recovered here. And Segal and Cover scored justices from editorials written before they cast a vote, so set those against the ordering here and ask which disagreements are the interesting ones.

Sources

Data

• The Supreme Court Database, Washington University in St. Louis, 2024 Release 01, justice-centered by citation — every justice’s vote in every case since 1946. http://scdb.wustl.edu/data.php (the site serves over http only). Agreement and scaling use the 2020–2023 terms; cases where the database records no ideological direction are excluded, because “voted the same way” is undefined without a direction to compare. The project’s account of itself is at http://scdb.wustl.edu/about.php, and the rule that codes each vote’s direction at http://scdb.wustl.edu/documentation.php?var=decisionDirection.
• Borden v. United States, 593 U.S. ___ (2021), the case read through that rule above. https://www.law.cornell.edu/supremecourt/text/19-5410

Academic literature

• Martin, A. D. and Quinn, K. M. (2002). “Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the U.S. Supreme Court, 1953–1999.” Political Analysis 10(2): 134–153. https://doi.org/10.1093/pan/10.2.134
• Segal, J. A. and Cover, A. D. (1989). “Ideological Values and the Votes of U.S. Supreme Court Justices.” American Political Science Review 83(2): 557–565. https://www.jstor.org/stable/1962405
• Epstein, L., Landes, W. M. and Posner, R. A. (2013). The Behavior of Federal Judges: A Theoretical and Empirical Study of Rational Choice. Harvard University Press.
• Poole, K. T. and Rosenthal, H. (1997). Congress: A Political-Economic History of Roll Call Voting. Oxford University Press. The same scaling idea, applied to a legislature.

The data itself

Every figure above is drawn from these tables. They are plain CSV, they open in a spreadsheet, and they are yours to take further.

agreement.csv, by_term.csv, close_cases.csv, justices.csv

I want to rebuild a public dataset from its original source, without any
starting files. Please do the following and show your work.

THE SOURCE. The Supreme Court Database, maintained at Washington
University in St. Louis — every case the Court has decided since 1946,
coded by researchers from the written opinions, because the Court
itself publishes opinions and no data. Release 2024_01, the
justice-centered file organised by citation:
http://scdb.wustl.edu/_brickFiles/2024_01/SCDB_2024_01_justiceCentered_Citation.csv.zip
About 1.7 MB zipped. Each row is one justice's vote in one case — more
than 80,000 rows. Two coded columns matter most: direction
(1 conservative, 2 liberal, blank where the coders could not say) and
majority (1 dissent, 2 majority).

HOW TO DOWNLOAD IT. The site serves over plain http only; an https
address fails outright. That is the source, not a problem with your
tooling — say so and proceed. The file is UTF-8; read it as such, or
the curly apostrophes in case names silently corrupt.

WHAT TO BUILD.
1. Per term, 1946 through 2023: how many cases were decided and what
   share went in the conservative direction.
2. For the 2020 through 2023 terms: every pair of justices and how
   often the two voted the same direction, with the number of cases
   they shared — and, per justice, how often each was in the majority.

CHECK YOUR WORK. The copy this book used was downloaded in August 2026.
If your rebuild matches it, you should find:
- 78 terms, 1946 through 2023
- ten justices sat in the 2020-2023 window, making 45 pairs; the
  busiest of them each heard 234 cases there
- Kavanaugh is in the majority 95.5% of the time and Roberts 94.3%,
  the two highest; Sotomayor, at 70.7%, is lowest
A new release lands roughly yearly at a new address; this one stays
put, so a difference means you downloaded a different release, not that
you made an error.