Two empirical studies

This post discusses two studies of appellate-brief writing that reached different conclusions and call for different explanations.

The first study scored nearly every merits brief submitted to the U.S. Supreme Court from 1969 to 2004 using four readability-assessment tools,[1] two of which are described here.

The Flesch Reading Ease Scale:

• uses sentence and word length to assess readability and assigns a score: zero to 30 is “very difficult,” while 90 to 100 is “very easy,” and 60 is “plain English.”[2]

The Flesch-Kincaid Grade Level:

• reports the number of years of formal education a reader needs in order to understand the text: 12 means a high-school graduate, 16 means a college graduate, and 19 means a law-school graduate.

For the time period assessed, U.S. Supreme Court briefs averaged a Flesch Reading Ease score of 35 (difficult) and a Flesch-Kincaid Grade Level of 14 (sophomore in college).

Also, during that time period, the grade level of Facts sections moved from 15 to 12—becoming simpler. The grade level of Argument sections moved from 14 to 13—again, becoming simpler. And the readability score for Argument sections moved from 33 to 39—becoming more readable.[3]

What does this trend to simpler, more-readable writing mean? Does it represent “the dumbing of America”? Should we conclude that even Supreme Court advocates are incapable of writing complex, sophisticated prose?

No. Given the high caliber of attorneys writing briefs to the Supreme Court, I draw a different conclusion. These advocates understand that a readable brief, written as simply as possible given the complex subject matter, will be more persuasive and engaging. It’s a good reminder for all legal writers.

The second study assessed briefs to the New York Court of Appeals from 1969 to 2008, and it relied on the tools already described: the Flesch Reading Ease Scale and the Flesch-Kincaid Grade Level.[4] But this study didn’t have good news.

Based on scores by decade (1969-78, 1979-88, 1989-98, 1999-2008), the average Flesch Kincaid Grade level moved from 13 to 14—becoming less simple, and the average Flesch Reading Ease Score moved from 42 to 32—becoming less readable.[5] The author was dismayed by these results in part because the time period, 1969-2008, roughly corresponds with a trend toward more and better legal-writing instruction in U.S. law schools.

Does this mean that legal-writing teachers, like me, aren’t making writing better—we’re making it worse? I took that pretty hard. I wanted a different explanation, and I think I found one.

Granted, when viewing the numbers by decade, you see a trend to less simple, less readable legal writing. But if you view the numbers by year, you see that there probably isn’t a clear trend.

In fact, when viewed by year, the numbers fluctuate widely and almost annually. For example, the grade level for 1990 is 8, but for 1996 it’s 16. From legal writing an 8th grader could understand to writing that would require a college graduate—in 6 years? It’s hard to explain.[6]

The readability scores rise and fall in a similar way. The score for 1981 is 52 (a high score for legal writing) but drops to 40 (difficult) in 1982. It then moves back up to 52 in 1990 but drops to 30 (very difficult) by 1993.[7]

In fact, the author’s charts that report scores by year look like up-and-down zig zags without (to me) a clearly discernible trend.

So here’s my explanation: briefs to the U.S. Supreme Court are produced by a fairly small and highly sophisticated group of appellate attorneys, and their writing gets better—simpler and more readable—over time. Briefs to the New York Court of Appeals come from a wider range of attorneys with a wider range of experience and skill, and their writing shows no definite pattern of improvement.

Now I feel better.

_____

[1] Brady Coleman & Quy Phung, The Language of Supreme Court Briefs: A Large-Scale Quantitative Investigation, 11 J. App. Prac. & Process 75, 76 (2010).

[2] Rudolf Flesch, How to Write Plain English 25 (1979).

[3] Coleman & Phung at 98, 99 (numbers rounded).

[4] Ian Gallacher, When Numbers Get Serious: A Study of Plain English Usage in Briefs Filed Before the New York Court of Appeals, 46 Suffolk U.L. Rev. 451 (2013).

[5] Id. at 465, 472.

[6] Id. at 472.

[7] Id. at 466.