July 2020 bar test-takers down significantly from recent years, but much smaller drop in those who passed

I wondered what happened from the July 2020 administration of the bar exam compared to previous administrations, in terms of test-takers and overall pass rates. I looked at results from 10 states: Colorado, Iowa, Mississippi, Missouri, Montana, North Carolina, North Dakota, Oklahoma, South Carolina, and West Virginia. These were states that administered a traditional in-person July 2020 bar exam and did not offer some later fall test. I thought this would be the cleanest way to compare recent trends—non-traditional bar exams (like an online option or a new state test), or those that offered another administration later in the fall, might skew the test-taking pool more.

I then looked at the last three years of July test-takers and overall passers. I thought this might even out any irregularities if I looked at just the July 2019 test alone. But, of course, law school graduating classes differ from year to year, both in size and test-taker ability, so it’s only a crude portrait.

In the July 2017, 2018, and 2019 administrations of the bar exam, these 10 jurisdictions averaged 3627 test-takers and 2576 who passed, a 71.0% pass rate.

In July 2020, test-takers were down significantly, 13.9%, down to 3124 (a drop of 503 test-takers). But passers were down just 3.3%, 2492 who passed the exam (a drop of 84 passers over the three-year average).

Complicating this analysis is that North Carolina temporarily reduced its cut score, which might, on the one hand, been designed to offset any disadvantages to preparing for the bar exam during a pandemic, but, on the other hand, may have increased newly-licensed attorneys if bar preparation during the pandemic was not so adversely affected among remaining test-takers. But North Carolina saw a 4.1% increase in passers over its three-year average, one of only two states to see an increase.

A driving factor appears to be that the most likely to fail the bar exam were the most likely to opt out of this administration. That’s particularly the case for those who previously failed a bar exam—repeaters as a percentage of overall test-takers were down significantly according to the NCBE.

For those worried about the pipeline of attorneys more generally and the availability of clients in search of access to justice, the news is at least modestly promising, that while there was a substantial decline in test-takers, the decline in licensed attorneys was much more modest. (This is not to defend other problems with adminstering the bar exam in a pandemic, as I’ve described extensively elsewhere.) It does vary from state to state, however, so it’s worth looking at specific jurisdictions, too. In Mississippi, for instance, passers declined by more than 22% over the three-year average; in contrast, West Virginia and North Carolina saw modest increases. I’ve included the tables below.

	2017-2019 avg		2020		2020 v. avg		2020 v. avg pct
	Takers	Passers	Takers	Passers	Takers	Passers	Takers	Passers
Colorado	740	530	642	499	-98	-31	-13.3%	-5.8%
Iowa	187	151	162	134	-25	-17	-13.2%	-11.1%
Mississippi	169	99	110	77	-59	-22	-35.0%	-22.5%
Missouri	660	525	602	507	-58	-18	-8.8%	-3.4%
Montana	97	80	83	71	-14	-9	-14.7%	-11.3%
North Carolina	833	532	668	554	-165	22	-19.8%	4.1%
North Dakota	72	48	58	44	-14	-4	-19.4%	-8.3%
Oklahoma	288	223	268	215	-20	-8	-6.8%	-3.7%
South Carolina	417	279	385	276	-32	-3	-7.7%	-1.1%
West Virginia	163	109	146	115	-17	6	-10.4%	5.8%

Would you rather take the bar exam, or work 6000 hours as an apprentice?

Emergency “diploma privilege” has been a hot topic around the bar exam. Modified versions of the privilege have been cropping up. Utah’s, for instance, limits it to certain would-be attorneys, along with a condition of 360 hours’ supervised practice. I won’t rehash a lot of the debate for now.

The District of Columbia recently adopted a version of this. But the version is hardly “diploma privilege”—that is, upon receiving a law school diploma, you are eligible to practice law. Instead, DC’s rule requires recent graduates to have three years of supervised practice in the District of Columbia. Assuming 50, 40-hour weeks (or a 2000-ish-billable year), that’s 6000 hours of work. Yes, you can get paid, but there are strings attached to the practice, and it’s limited to practice within DC.

We’ve moved far afield from “diploma privilege” into, essentially, other alternatives to the bar exam. A three-year apprenticeship might be a good thing, but it’s also a very different kind of requirement from true diploma privilege. Indeed, tacking a three-year apprenticeship at the end of a three-year law degree feels onerous. Advocates for “diploma privilege” in DC have recognized this and pushed back against this new requirement.

To me, the bar exam would be a superior path for most law school graduates than this model. But it’s also worth considering the broader list of costs and benefits with all non-bar exam alternatives. Supervised practice might be better for some cohort—but it does offer a particular delay to the full practice of law. Maybe at the end of it, those under supervised practices are subject to less career discipline, or are more “competent” attorneys. All good things to measure. But, and I just note it here briefly, that it’s hardly the case that all replacements for the bar exam should be deemed “diploma privilege,” and it’s not sure that all alternatives are better than the bar exam.

What we don't know about the July 2020 bar exam and Covid-19: a lot

My post last week offering the tentative conclusion that it appears no one contracted the coronavirus during the July 2020 administration of the bar exam languished in obscurity for most of the week until something spurred attention to it, offering voluminous feedback, in some ways the usual assortment of decidedly-helpful to decidedly-unhelpful responses.

There were a couple of tranches of responses I wanted to highlight. The first were a few people encouraged by the results, in particular thinking about how test-takers and bar exam administrators adopted some useful practices ahead of the exam. Test-takers were unusually cautious in self-isolation before the exam to avoid getting turned away from the exam; basic standards like mask or physical distancing requirements go a long way for a relatively short period of time in a room; test-takers traveled by driving in lieu of mass transit to avoid illness. In short, maybe these results shouldn’t be much of a surprise. (They would have been a surprise to me in March, when my robust “might” was all I could muster for an in-person exam!)

The second—and there were much more of these—were those critical of my conclusions. I want to focus on two of the more helpful points to highlight the weaknesses of my post.

The first bucket are questions about whether I asked the right question. As I wrote:

I reached out to several bar licensing authorities to see, as I put it, “whether any person contracted Covid-19 as a result of the administration of the July 2020 bar examination, or if any person contracted it as a result of their traveling to or from or lodging around the time of taking the bar exam.” (Should I have asked a different question? Well, I tried….)

Lots of people suggested I asked the wrong question. For instance, “as a result of” put too much of a causal emphasis, which bar licensing authorities would not necessarily be able to answer—or that made it easier to answer “no.” A positive test shared after the fact could still be reported as one not “as a result of” the exam. Fair critique! My goal was a narrow and specific concern about what we know. But my inquiry could of course have been broader. Additionally, as I noted, “some jurisdictions did emphasize they asked test-takers to disclose if they contracted the coronavirus within 14 days after the exam, and none did so. (That would sweep more broadly than contracting it during the bar exam, of course.)” For a follow-up inquiry (or, if you’re thinking about you, your next inquiry!), I might ask something more broadly, without the causal aspect.

Second, and relatedly about the scope of the question, was whether jurisdictions had contact-tracing protocols in place. Again, some did volunteer the post-exam 14-day reporting period, which, again, suggested to me some contact-tracing protocols in place. But, would it have been more useful to ask—before, during, after the exam—for bar licensing authorities to specify what, if any, contact-tracing protocol they had? Or what their negotiations with the local health officials looked like to do so? Absolutely.

Much of my encouragement came from Colorado's process, because (1) it appears people in the room were all made aware (although I cannot tell how much was word of mouth as opposed to formal notification), and (2) Colorado bar authorities apparently knew the negative results of the test-takers in the room who later took a test as a matter of peace of mind. (That was despite the fact that Colorado's local health authorities said that testing of everyone in the room wouldn't be required because of existing social distancing protocol.)

In short, are there more helpful questions? Absolutely. Is there more evidence we could gather? Definitely. Should we (err, should someone) do so? Totally.

We don’t know the results in a lot of jurisdictions. We don’t know who opted out of the exams. We don’t know how online administrations or administrations split across months look. We don’t know the psychological effect it had on test-takers and how it compares to previous administrations of the exam. We don’t know a lot. Indeed, I only got to seven jurisdictions in my original post! There are a lot of important questions still to ask. (Some I’m working on in the months ahead!)

So… do it! Ask good questions (or, questions better than mine!); engage in good faith with bar licensing authorities and public health officials; determine what we can do going forward!

The most helpful piece, I think, is from Professor Paul Horwitz over at PrawfsBlawg, with this bit I’d like to emphasize:

Voicing ex ante concerns about risks is perfectly understandable. One can hardly wait until after the event to express worries about future risks. And the outcome doesn't mean the concerns about risk were unwarranted. But it seems to me that, at least for those whose arguments were based on academic expertise, or invoked that expertise and appealed to past and ongoing empirical study of the issue, or otherwise invoked a kind of academic or data-driven or scientistic authority in making various arguments, there is an arguable duty to follow up and see what the data ultimately revealed about the accuracy of those warnings.

Claims about the future are often too easy to make. I really like to go back and see how often those (often dire) predictions come true. Sometimes these questions are hard to figure out. This question in particular seems very hard to figure out—hence, a post that opens with “appears,” turns exclusively on the causal questions, and includes a lot of caveats (go back and read all of them!). And, of course, even if true, it doesn’t answer the much bigger questions about the right approach in each jurisdiction (see, e.g., the caveats). It only helps us move marginally forward on one piece of a very complicated puzzle.

It appears the July 2020 bar exam did not spread Covid-19 among any test-takers

There was much concern ahead of the July 2020 bar exam that administering the exam would be unsafe for test-takers. Indeed, it’s a reason why many states postponed their exams, shifted to an online exam, or offered diploma privilege to some cohort of would-be test-takers.

Several states, however, administered the July 2020 bar exam in person, on schedule. And it appears that there were zero instances of the spread of Covid-19 related to the administration of the exam in any of jurisdictions.

(Let me offer a few throat-clearing caveats here. Past performance is no indicator of future success in later administrations. Some set of test-takers had difficulty studying for the bar exam ahead of the July 2020 exam; some may have had difficulty taking it in masked environments. We’ll see what the results of these administration look like, of course—it might be that pass rates decline by some degree among some because of these factors. But, then again, they might increase if people had little to do or few places to travel and more time to study! Some people also probably withdrew from the administration of the July 2020 bar exam due to family commitments or high concern of the risk of contracting Covid-19 during the administration of the bar exam, which is not a trivial cost. Some set also may well have opted into these particular tests in UBE jurisdictions, as the score could transfer to other jurisdictions that had postponed administrations. In short, there are a lot of open coronavirus-related questions that this post is not about, but additional concerns to think about in evaluating costs and benefits.)

That said, if we are concerned about the risk of spreading Covid-19 during the bar exam, this should be encouraging. There are a couple of ways to think about Covid-19 spread, too. The first is during the administration of the exam itself—several hours indoors for a couple of days. The risk of transmission seems particularly low here if adequate precautions are put in place. The second is the broader concern of spreading Covid-19 during travel to and from the bar exam or lodging in another city during the administration of the bar exam itself. That’s a longer time period, a greater likelihood of interacting with others, and a different kind of risk much more difficult for a licensing authority to control, so I don’t want to minimize the risk of spread to “only” the several hours of test-taking.

My instinct is that if someone did contract the coronavirus during the administration of the bar exam, we’d probably know by now. That is, contact tracing would require contacting dozens of test-takers, and that news would presumably make it to some media outlet. Of course, someone might privately have become ill and never disclosed it to the state bar. Or there could have been a great deal of discretion in how these inquiries proceeded. Or maybe contact tracing simply never took place. There are, admittedly, limits to silence.

I reached out to several bar licensing authorities to see, as I put it, “whether any person contracted Covid-19 as a result of the administration of the July 2020 bar examination, or if any person contracted it as a result of their traveling to or from or lodging around the time of taking the bar exam.” (Should I have asked a different question? Well, I tried….) Not everyone got back to me, and I didn’t reach out to everyone, so it’s also limited in this respect, too.

So far, representatives in Colorado, Iowa, Mississippi, Montana, North Carolina, North Dakota, and West Virginia confirmed that, to their knowledge, no one contracted the coronavirus on account of the administration of the bar exam. (I’ll update if I have updates.) Some additionally confirmed that no proctors or staff contracted it, either.

Proving a negative is tough, but some jurisdictions did emphasize they asked test-takers to disclose if they contracted the coronavirus within 14 days after the exam, and none did so. (That would sweep more broadly than contracting it during the bar exam, of course.)

Colorado was of particular interest, as one test-taker learned immediately after taking the bar exam she tested positive for the coronavirus. No one reported contracting the coronavirus as a result of the Colorado exam. Indeed, according to a representative, about half of the test-takers in the room with this test-taker ultimately took a Covid-19 test, and all came back negative.

This should all be encouraging news for bar licensing authorities. Yes, there remain costs, as noted in the parenthetical above, about administering a bar exam during the pandemic. But, I think, if bar licensing authorities have the space to distance test-takers, it provides ample opportunity to continue administrations in February 2021 (or later this fall in jurisdictions that postponed the exam), it may well be that administering the test as scheduled rather than postponing was the better option. Jurisdictions like North Carolina and Colorado each nearly 700 test-takers, which shows that the administration doesn’t have to happen exclusively on a very small scale. (North Carolina and Colorado had the 14th- and 15th-most test-takers in the July 2019 administration of the bar exam, and neither is offering an additional administration later this fall.) That said, scaling to the 10,000 or more test-takers in New York could present very different challenges!

And, of course, we’ll wait for more data to roll in—whether pass rates change, whether the demographics of test-takers differed, and so on. Some states have preemptively and temporarily lowered their cut scores to account for some coronavirus-induced changes, for instance.

North Carolina is a great example—their total test-takers declined from 783 in July 2019 to 668 this July. But the cut score was lowered slightly, and the pass rate rose from 73% to 83% (perhaps also due in part to some non-random self-selection out of the exam). As of September 1, results are out for hundreds of new law school graduates—and many other jurisdictions languish about their plans.

I don’t purport, as I’ve said since March (!), to have a single answer for every jurisdiction on the bar exam, or what the best steps are for each jurisdiction. Some version of diploma privilege may make sense in some places, modified versions of the exam may look better elsewhere. But on this one little piece of information, we should be encouraged about the health and safety of test-takers for in-person administration of the exam.

It's not bad news: MBE scores rise for July 2020 bar exam

What a tepid title I can offer…. As longtime readers of this blog know, I track performance on the bar exam fairly closely. The July 2020 administration, however, offers only a limited window into what’s happening in legal education. Only some jurisdictions held a July 2020 bar exam; even in those jurisdictions, self-selection among test-takers this time around is probably not random (perhaps skewed toward better test-takers—those with ample resources to study and prepare, for instance).

So the NCBE’s update on performance on the July 2020 bar exam is of limited value. Scores went up. They went up a lot. In fact, they reached an all-time record high! It’s great news for those who took the July 2020 bar exam, of course! And pass rates will probably rise significantly in these jurisdictions that administered the exam.

View fullsize

Now, I won’t get overly excited about this chart—at least, I usually use this chart to make an overall evaluation of the state of legal education, and it really isn’t great to do that here. About 5700 test-takers took the July 2020 MBE. That’s compared to 45,000 or so in a typical July! (Removing New York alone, of course, accounts for 10,000 or so tests.) 5700 test-takers is a big sample, but it’s relatively a small cohort that tells us little about the overall state of legal education.

The MBE scores were up 5 points over July 2020, but they were up only 3.5 points over 2019 in the 23 jurisdictions that administered the July 2020 bar exam. That means if we added in other jurisdictions, we may well see a smaller spike.

Even within that cohort of states that administered a July 2020 exam, first-time test-takers were about 75% of all test-takers, about 6 points [I think] higher than the previous exam. So the pool of test-takers were stronger as it skewed toward first-time test-takers, which likely accounts for at least some of the improvement in scores. (It also suggests that those who failed the bar exam previously increasingly opted out of this administration of the exam.)

In short, this is terrific news for the crop of test-takers who did take the exam. (Not to mention those in the four states with some version of “emergency” diploma privilege this fall.) Maybe the absence of, say, weddings and night life meant limited travel and greater focus on studying for the exam as an additional contributing factor to success. One could only guess. But it probably doesn’t tell us a whole lot about the overall state of legal education and the cohort of recent law school graduates.

A survey of the broad scope of the proposed New York diploma privilege law

A10846 is a diploma privilege bill working its way through the New York legislature. After unsuccessful postponements and questions about how an online version would work, there’s been renewed urgency to help license attorneys in one of the largest and most significant legal jurisdictions in the United States.

I’m sympathetic to some calls for diploma privilege in some jurisdictions in the current environment—but I do think some tailoring is appropriate, as I’ve written. While Washington’s diploma privilege struck me as potentially overbroad, the current text of A10846 far exceeds it, and it’s worth looking at its scope—and what might be amended out of the bill later.

New York does not have a particularly difficult bar exam. And one could look at its July 2019 statistics as a window into what rules for a temporary diploma privilege might look like. Among first-time test-takers from ABA-accredited schools, 86% passed—4748 out of 5517 test-takers. That’s a high pass rate. Of course, it means 769 didn’t pass, and a diploma privilege for all first-time test-takers from ABA-accredited schools would sweep them in, too. As emphasized in my previous writing, however, maybe that’s a tradeoff New York is willing to make, and maybe one could have more robust follow-up of this cohort to try to minimize misconduct or malpractice.

If one took a broader view, more like Washington, to all test-takers, first-timers or not from ABA-accredited schools, it would bring in another 990 test-takers from the July 2019 bar exam. Only 28% (272) of those repeaters passed, so it would add 718 graduates who failed the bar exam multiple times. Again, maybe a tradeoff worth taking in the larger picture.

New York’s bill goes further still. It would extend to “any person who has graduated or will graduate from an American Bar Association accredited law school or who would otherwise be eligible to take the New York state bar examination.” (Emphasis added.) That last clause is significant because New York has a significant cohort of foreign-educated attorneys that take its bar exam each year.

In the July 2019 exam alone, for instance, 2398 foreign-educated test-takers took the bar exam for the first time, but just 1266 passed, a 53% pass rate. Another 1161 repeated the bar exam, and 250 passed, a 22% pass rate. That means over 2000 foreign-educated test-takers failed the July 2019 bar exam. All would be admitted under this rule.

I’ve pointed out in the past how non-JD legal education is on the dramatic rise in American law schools, and how foreign-educated LLM degrees are on the rise. But bar passage rates remain low, likely in part due to language barriers. These test-takers remain a significant cohort of overall test-takers.

Unlike other states that have had temporary “emergency” diploma privilege rules, New York’s is not limited to those who previously registered for the July 2020 bar exam. It extends to all prospective attorneys, through September 30, 2021 (or until the end of the Covid-19 disaster, whichever is later), who satisfy the other requirements, like character and fitness requirements.

That could potentially sweep in many more attorneys who’d otherwise be eligible but have failed the bar exam in the past—they might be in New York (the law is limited to those “who intend[] to primarily practice in the state of New York”) and practice. If they complete “100 hours under the supervision of a permanently admitted attorney,” they would be eligible for permanent admission to the bar. (One hundred hours is two and a half 40-hour weeks.) That’s much lower than, say, Utah’s 350-hour supervised practice requirement.

In short, if enacted as written, it’s possible to see thousands of new attorneys in New York—perhaps much more than the equivalent of the 3500 who failed the July 2019 bar exam if one looks at the scope of the rule that might extend to other would-be attorneys who could otherwise meet the rule in the next year.

It’ll be worth seeing if New York enacts the law as written or amends it. Maybe it won’t be enacted at all. But if it is, it’ll be an even more significant experiment in attorney licensing than Washington’s rule, and emphatically one to watch.

Diploma privilege 2.0

While there’s a renewed effort afoot for diploma privilege (either on a one-time emergency basis or on a long-term basis) in lieu of the traditional written bar exam, it’s worth considering that this new version of “diploma privilege” looks nothing like the traditional “diploma privilege” of places like Wisconsin, or as recently abolished in states like Montana. For lack of creativity, I’ll call the new movement “diploma privilege 2.0.”

The bar exam predates law school, tracing back in the United States to 1763. But recent historical “diploma privilege,” or admission to the bar upon securing a diploma from a recognized law school (with other conditions, of course, like passing the character and fitness examination) had a couple of hallmarks. First, it was expressly designated to benefit in-state schools. And second, it was designed around a model of education in which the juris doctor diploma was granted on the condition of learning a certain body of in-state law.

It’s worth going back to recent history abolishing diploma privilege (err, “diploma privilege 1.0”) in some jurisdictions. Consider this justification in Montana in 1980: In the Matter of Proposed Amendments Concerning the Bar Examination and Admission to Practice of Law in the State of Montana, 187 Mont. 159 (1980).

There is no substantial or acceptable argument for retention of the diploma privilege. Its primary purpose has long since ceased to exist i. e., incentive to attract students to a small law school as it struggles to gain recognition in the legal community or the common argument that the last quarter of law school must be devoted to preparing the student for the bar examination which is a detriment to their normal course of study.

There is, in fact, a double standard created by the diploma privilege and the Bar examination as it relates to admission to the Bar in Montana. This standard goes beyond the courses offered in the law school and given on the Bar examination. It is the fact that the diploma privileged person enters the job market in June, whereas a Montana resident forced to attend an out-of-state law school must wait until October to take the examination, and in some cases does not pass fairly enough, many people have elected to attend schools outside the State of Montana. It is also noteworthy that the University of Montana School of Law is no longer struggling. It is turning away many, many students who are Montana residents and who would like to remain here to go to school. Also, we should be encouraging our young people to go to other schools outside the State of Montana for the diversity of educational background and the intellectual exposure which is essential in a free society. There is no doubt that the University of Montana School of Law is very good, but concentrating Montana graduates into the Montana Bar becomes dangerously parochial.

The effect of a diploma privilege on the student and on the faculty of a law school that extends the privilege is subtle but sometimes harmful. There exists the possibility of abuse and the standards of the law school may be affected by the fact that nobody really does his best until he has to. Knowing that their students are not to be examined, some professors may be prone not to put forth their best efforts, or at least a better effort than they did the previous year teaching the same course. Under some circumstance, the curriculum can be adjusted to teach the students what they want the students to know, and there is nothing in the world to prevent this.

E. Marshall Thomas, the former chairman of the National Conference of Bar Examiners, makes the point that even though all subjects were the same on the school curriculum and on the bar examination, it would still not be an idle act to require that they take the examination since it serves a real additional purpose. The fact that the law student knows he must face the Bar examination after graduation and before admission to practice is a healthy, educational stimulant. Mr. Thomas further contends that it is also a stimulant to the law school faculty to maintain high standards of legal education because the faculty knows that their students will be examined by state authorities. He says that the Bar examination serves an additional function in that the Bar examination has one essential difference from the law school examination it is a comprehensive examination covering the entire field of several years of law study.

Further, the American Bar Association has taken a positive, clear and very hard stand against the diploma privilege in connection with the standards of legal education and A.B.A. approval of law schools. Further, there are very few jurisdictions left which permit this kind of privilege. The A.B.A. Section of Legal Education and Admission to the Bar is very strong in its opinion that graduation from a law school should not confer the right of admission to the Bar and that every candidate should be subject to an examination by a public authority to determine his/her fitness.

The University of Montana School of Law has stressed that the Supreme Court members, or rather two of them, are on the Board of Visitors and can oversee and control the curriculum of the school. The A.B.A. says that this is not right, that there is no public officer or officers or departments who control the curriculum of any school. Their job is to see that the school turns out properly educated people who can adequately serve the public. Any attempt to control curriculum content would be an unfortunate limitation on the educational freedom of the school and could not be tolerated in the name of the diploma privilege.

It follows then that to reach our ultimate goals, the diploma privilege must be eliminated. However, at that time, those students who have applied and will be accepted or are accepted or are in the law school when these changes are made, will be given the benefit of all privileges held out to them, i. e., a grandfather-type concession will be afforded to those with any remote contractual right to be a beneficiary of the diploma privilege, as set forth above.

This Court orders the diploma privilege abolished in conjunction with the other changes to be made in our qualifications and admission to practice.

One can judge whether the reasons for or against the privilege are persuasive (it reflects, for instance, a distrust of law schools that I’ve highlighted elsewhere). But I wanted to highlight a few of the points: in Montana, for instance, the privilege was overtly designed to advantage the in-state school over out-of-state schools, and it ended up advantaging in-state enrollees in the job market, too. Another is that the Montana Supreme Court viewed it as inappropriate to try to control the specific curriculum of the law school to ensure that there would be the advantage of diploma privilege of in-state law school graduates.

Wisconsin adheres to this old tradition, if somewhat inconsistently—state bar controlling the curriculum (to a degree), an advantage for in-state graduates, a focus on state-specific law.

This in-state v. out-of-state makes sense in an old regime of thinking about law, in times with less interstate litigation and greater common law variation. Today, however, we have increased mobility, increased uniformity, increased specialization, and such a wide body of law that we expect attorneys to look it up rather than assume they’ve memorized it all—or, more to the point, memorized it all before practicing. Having a generalized knowledge of a single state’s law feels less typical than it might have been decades ago.

Recent developments in written bar exam have moved this way. The Uniform Bar Exam, put forth by the National Conference of Bar Examiners, has sought to make bar exam scores more easily transferred across jurisdictions. It’s much easier to become licensed in multiple states with the UBE. But the UBE, of course, is an emphasis on uniform. It’s hardly ensuring that recently graduates have a (memorized!) body of state-specific knowledge for a given jurisdiction, as the old bar exams might have emphasized. (But it still requires memorization and understanding of law, even if, for some areas, it’s a bit of a fictitious amalgam of common law!)

Indeed, a recent New York task force called for returning to more state-specific knowledge, pushing back against the NCBE’s UBE move. For the reasons I note above, I’m skeptical about this proposal—particularly as the report relies heavily on “anecdotal” evidence that new attorneys don’t know enough New York law to practice in the state. (It would be interesting to try to measure new lawyers’ abilities to handle certain tasks within months of being licensed and whether any disparities in ability exist; and whether those disparities, if they exist, disappear in a relatively short period of time. But, as far as I know, no one is trying to study or measure that.)

The recent push for “emergency diploma privilege” has led to Utah, Oregon, Washington, and Louisiana to offer forms of diploma privilege, but in ways that did not exclusively advantage in-state graduates. Utah and Oregon privileged both in-state graduates and a good chunk of out-of-state graduates; Washington and Louisiana privileged essentially all ABA-accredited graduates. And they did not condition admission to the bar on learning a body of state-specific knowledge—but there are conditions, like, say, in Utah, where admissions is also conditioned on a stretch of supervised practice, which may include some of the state-specific learning one may desire.

(Indeed—and this is an open question!—is this the first time in history that a state offered diploma privilege to out-of-state law graduates? I’m not sure. But I’d be interested to find out if there’s a law school historian out there!)

UPDATE: There’s plenty of interesting history out there! Oregon once had a kind of “reciprocal” diploma privilege, offering it to graduates of any school whose home state would offer them diploma privilege. Texas had an exemption that included out-of-state schools. So, there are other historical examples—just none in recent years.

Earlier, I tried to disaggregate the emergency diploma privilege and the longer-term diploma privilege proposals. I highlighted a point I raised a year ago, that bar exams are, I think, in part about distrust that state bar licensing authorities have with law schools—that is, that state bar licensing authorities think law schools do not have the admissions, retention, and graduation standards that would allow all JDs to become licensed attorneys. It could be other reasons, of course—one may think, like the New York task force, and contrary to my views, that learning some state-specific body of law is important, etc.

In that older post, I suggested some smaller states could learn to trust their in-state law schools again if they worked closely with one another—and not to control the curriculum with state-specific knowledge, but to ensure that all the graduates were capable of practicing law. But I think I would want to reflect on such a proposal to be broader—and that’s hard to do. That is, it shouldn’t just be a privilege for the one or two or three in-state schools that the state bar licensing authority trusts. It should be open to a broader set of schools. But how?

Proponents of this new diploma privilege put their cut-off at ABA accreditation, but I think that’s not sufficiently narrow—that is, sadly, the ABA accreditation process is not designed to ensure schools graduate cohorts competent to practice law, a reason its accreditation rules are tied to, say, a 75% ultimate bar passage rate (not 100%!), or recognition that many graduates never practice law. And it’s beholden to other interests, like the DOJ. ABA accredited schools face probation or, in recent years, closure for failing to maintain standards.

In short, there has to be a way for state bar licensing authorities to determine whether in-state and out-of-state law schools both have the admission, retention, and graduation standards that would entitle them to diploma privilege. That’s a much trickier concept to sort out. But I think it’s the better approach. Rather than favoring in-state schools with conditions of learning state-specific bodies of knowledge, a more generous “diploma privilege 2.0” would look at all schools nationwide without such conditions. But filtering out which is a sticking point for this too-long blog post! Utah and Oregon set the cut-off at schools with an 86% first-time bar passage rate in 2019. That won’t work if the bar goes away as a reference point. Maybe it’ll take a new licensing authority, something other than the ABA, to offer diploma privilege-certified schools. But that also feels like a lot of effort. In short, I don’t have great ideas—just identifying that it’s a different system than the old one and one that, I think, requires some different thinking.

High-level implications: California Supreme Court reduces bar exam cut score from 144 to 139

Whew! A long time coming with a lot of implications about the ever-present debate about the “right” cut score. Here’s a few (ed.: hardly!) thoughts that come to mind.

First, California had the second-highest cut score (144), behind Delaware’s 145. (Even accounting for non-scaled score jurisdictions, it’s still probably the second toughest bar in the country). Lowering it 139 puts it behind Alaska (140) and Virginia (140)… and that’s it. So… from the second-toughest to the fourth-toughest.

California has basically permanently had a cut score of 144—it’s modified its scoring over the year, but this has been approximately the same over iterations of the test. Which is to say, lowering to 139 is a big change after several decades! (And I should add, still higher than the range of 133 to 136 proposed by many of California’s law deans in 2017.)

A recent standard-setting study suggested the cut score should remain roughly the same, consistent with historical studies in California, but it left open the possibility of a range of possible scores within an acceptable range. At the lower end, approximately two standard errors below the median recommended cut score of 143.9 (i.e., almost the same as the 144 existing cut score!) from the standard setting study, was a cut score of 139, consistent with a proposal from the state bar. This is the figure the California Supreme Court adopted.

So this 139 is not out of nowhere—it’s consistent within the range (admittedly, at the bottom of the range!) of options from the 2017 study. And it’s possible that score could change as the bar continues to examine the cut score in the future—but this will be a “permanent” change (i.e., not for this administration of the bar, like Oregon). I suppose one could chalk this up to a coronavirus-related change, but it is far more than that.

Second, it’s worth noting that despite being a relatively high cut score, California test-takers are typically more able than most test-takers, and the reduction will translate into significantly more attorneys. On the July 2018 bar exam, for instance, the mean scaled score was 142.8 compared to the national average of 141.1. Given that scores tend to follow a bell curve, this reduction in the cut score will sweep in a significant number of passers. For instance, a simulated cut score for the July 2016 bar exam of 139 increased the overall pass rate from 43.3% to 52.1%, with almost 800 new passers (many of these repeaters). It would also yield a 40% increase in the Black passing rate, 26.1% increase in the Hispanic passing rate, a 23.5% increase in the Asian passing rate, and 17.6% increase in the White passing rate.

Third, all else being equal, we would expect this particular cohort of attorneys admitted to the California bar (i.e., admitted with a score of between 139 and 143.9) to have an ultimate career attorney discipline rate of around 10% or so. All else being equal. It might be there won’t be the resources to discipline so many new attorneys (unless there’s increased funding). And California is also simultaneously (and rightly!) looking at proposals to improve concerns about attorney discipline. We’ll see if this affect ultimate career discipline rates.

This isn’t to say lowering the cut score is the wrong choice—it’s just to say that with any choice there will be tradeoffs. 90% of those attorneys won’t be disciplined over the course of their career, for instance (and one could ask whether discipline rates are the best or only measure of attorney competence, or whether discipline captures all harm to the public). And maybe better surveillance will minimize the risk to the public. Time will tell. (Sadly, it appears California hasn’t yet investigated with precision the relationship between bar exam scores and discipline—maybe it will now.)

Fourth, I can’t help but wonder if the reduction comes at a time as California has been losing ABA-accredited law schools at a sharp rate. The collapse in bar passage scores has particularly hit California schools. California has lost three ABA-accredited law schools in three years: Whittier in 2017, followed by La Verne and Thomas Jefferson, both of which became California-accredited schools in recent months. California nearly lost a fourth ABA-accredited school as Western State found a last-minute institution that saved it. I wondered back in 2016 if California would move in this direction to save schools. Maybe it’s a reason. I admit, it’s not in the letter and likely would never be publicly identified. But it’s certainly something that will help the remaining law schools.

Fifth, what will this mean for California law schools? To shamelessly borrow from an earlier post speculating on what would happen if the cut score were lowered to 139:

At least some law schools will admit larger classes. To the extent law schools were reluctant to admit larger classes because of concerns about bar passage rates, those schools will be more inclined to admit larger student bodies. Of course, there are still other reasons that schools may not increase their class sizes, or at least not substantially—they are concerned about their LSAT and UGPA medians for USNWR rankings purposes, they may be worried about finding meaningful legal employment for a larger number of graduates, and so on. But, at least one barrier in the admissions calculus has been partially removed.

Higher-ranked law schools may begin admitting more students that recently historically matriculated to lower-ranked law schools. That is, a new kind of competition may begin. In light of the thought mentioned above, it may not simply be that schools admit larger classes; they may be grabbing applicants who would have attended lower-ranked schools. This would exert downward pressure on lower-ranked schools in the event that competition for their prospective students increased.

Higher-ranked law schools may see improved racial diversity profiles among incoming classes, potentially at the expense of lower-ranked schools. This is good news for highly-ranked schools and students from racially diverse backgrounds. The lower score will tend to benefit racial minorities, as the data has shown that minorities fail the bar at higher rates. So highly-ranked schools can admit more diverse student bodies with greater confidence of their success. Of course, this will exert downward pressure on lower-ranked schools, who may see their diversity applicant pools dwindle or face pools of applicants with worse predictors than in past years.

Law schools will experience more price sensitivity from prospective law students. That is, the value of the law degree should decline in California, as the volume of attorneys increases and the price for lawyers drops. That should, in turn, make law students more skeptical of the existing value proposition of a law degree. Law schools that have relied on high tuition prices have benefited from the high bar exam cut score, because opportunities for attorneys have been relatively scarce; the drop in cut score will dilute the value of the degree and perhaps require some cost-cutting at law schools. This is not to say that an artificial constriction on the supply of lawyers is a good thing because it props up costs (in my personal view, I think it's quite a bad thing); but, it is to say that lowering the score will have the effect of making cost-sensitivity an increasing possibility.

California-accredited law schools will have opportunities to thrive. San Joaquin, for instance, would have a projected bar pass rate of 50% in 2017 if the cut score were lowered from 144 to 139. Lincoln Sacramento would have a projected bar pass rate of 47%. These exceed some ABA-accredited schools and start to look quite attractive to prospective law students. That’s particularly true given the tuition at these institutions, which is about half the typical ABA-accredited law school.

Negative impact in USNWR for elite schools, and positive impact in USNWR for more marginal schools. This category may not be immediately obvious to observers considering bar exam pass rates. That is, some might ask, wouldn't higher bar exam passing rates improve a school's USNWR profile? Not necessarily--particularly not if the overall passing rate increases.

USNWR measures bar pass rate not in absolute terms but in relative terms--the margin between a school's first-time passing rate in a jurisdiction and that jurisdiction's overall pass rates. If School A has a passing rate of 90% and School B 75%, showing some gap that's only part of the story: School A had a 90% rate in a jurisdiction with an overall rate of 60%, which means it actually did quite well; but School B had a 75% rate in a jurisdiction with an overall rate of 80%, which means it actually did poorly. USNWR measures that relative performance.

So if School A sees its passing rate increase to 93%, but the jurisdiction's overall passing rate increases to 85%, that's bad for School A in USNWR terms--its ability to outshine others in the jurisdiction has dwindled. In a state as large as California and with such a relatively low first-time overall passing rate, this gives elite schools an opportunity to shine.

Stanford, for instance, boasted a 91% first-time bar passage rate in a jurisdiction with a 56.3% first-time pass rate in July 2017, a 1.62 ratio. If the bar pass cut score is dropped to 139, the bar projects a first-time pass rate of 64.5%. Even if its pass rate increases to a projected 96%, its ratio drops to 1.49, a 0.12-point drop. The same holds true for institutions like USC (-0.08), UCLA (-0.03), and Berkeley (-0.06). These are just one (very small!) factor in the USNWR ratings, and these figures are ultimately normalized and compared with other institutions nationally, but it will marginally hurt each of these schools as an institution in the rankings--even though it might benefit a (relatively) small cohort of graduates from these institutions each year taking the bar exam.

In contrast, schools that have had below-average bar exam performance would see a significant increase—some of them in my projections moving up 0.2 points in their ratios or even more. If the school is in the unranked tier, it might help get the school into the rankings; if they are ranked lower, it might help them move up the rankings, an added benefit to their graduates passing the bar at higher rates. And it might help increase the employment rates of these schools.

*

In short, this is a huge relief to many test-takers, and in particular to many repeaters who were close on a previous administration. It should result in thousands of new attorneys entering California in short order. The effects? Well, good for law schools in the short term, and good for recent graduates. For everything else, we’ll have to wait and see.

Disaggregating the debate over the bar exam and diploma privilege

Longtime readers of this blog know my frustration over conflating arguments in the debate over the bar exam. And it’s happening right now in a very public (and heated) way about diploma privilege. In my view, there are two very different arguments about diploma privilege that are too quickly conflated. (Reader, beware: much hedging and musing ahead….)

*

The first is a specific concern related to the coronavirus pandemic and administration of the summer/fall 2020 bar exam. That is, given the complications of administering a largely-indoor exam over two days to hundreds or even thousands of strangers who have been asked to travel and congregate together, perhaps alternatives should be considered. Diploma privilege is one such consideration. There are many alternatives, but it’s worth considering cohorts of test-takers—for instance, the privilege might not extend to out-of-state graduates, graduates of foreign law schools, of state-accredited law schools, of practicing attorneys in other states, or those who’ve previously failed the exam. Nevada, for instance, looking at alternatives, will put an online twist on an old practice-oriented format. I’m still skeptical of how online administrations will work (or, really, just about any alternative system!), but several states are moving toward similar formats.

It’s worth noting that many bar licensing authorities simply failed to think adequately, even in small ways, last spring about what might happen this summer. I wrote back in March that postponement seemed like a suboptimal choice because, as I noted, “it’s not clear when this pandemic will end.” And given the relationship between law school grades and the bar exam, I thought a minimally-disruptive alternative like substituting law school work product for the bar exam might be a good option. (For what it’s worth, no one’s asked me about this idea….)

A few states have already moved in the direction of diploma privilege. Utah’s strikes me as the most sensible so far of the versions of “diploma privilege,” if somewhat rough justice—it covers the vast majority of first-time test-takers in the state and many from out-of-state schools, even at the expense of some cohorts of would-be test-takers, but that’s what rough justice does here. Utah’s proposal has, I think, received some unfair criticism both as too generous and as too restrictive. Oregon’s strikes me as somewhat more curious in its rough justice but in line with Utah’s. Washington, in contrast, strikes me as potentially overbroad with the admission of a number of prospective test-takers who’ve failed the bar multiple times, but, as I noted, perhaps can be mitigated with oversight in the years to come.

In my view, the more honest assessment is this: “Taking a traditional written bar exam in a pandemic is not feasible. Any new bar exam system—online, or a new kind of test, or whatever it might be—is going to cause some uncertainty and problems. A rule like diploma privilege for first-time test-takers of ABA-accredited law schools might result in some small additional long-term risk of some members of the public receiving representation from attorneys who are not qualified to do so. But the alternatives are too costly—postponement has already failed, for instance. And given that the vast majority of test-takers pass the bar exam [depending on one’s state, of course!], diploma privilege is reasonable measure at this time [for a particular cohort, as so defined]. Let the state bar look at this cohort closely in the years to come to try to avoid any malpractice, attorney discipline, or other concerns that might arise.”

To me, that ought to be persuasive in many (not all!) jurisdictions. It’s honest, it concedes costs, it identifies the trade-off, and it tries to cabin the scope. It could be paired with other things like supervised practice as Utah is doing or as advocated elsewhere. Advocates of emergency diploma privilege tend, in my view, to overstate the historical comparisons, but emphasis on the emergency, I think, can go a long way.

In my view—and, my sense is, I’m an outlier among many—I think licensing authorities are genuinely trying to do the best they can, as divergent as the approaches might be. Yes, some are going to be more ham-handed than others or less empathetic than others. But I’m not so quick to ascribe ill motives or malice to licensing authorities tasked with many competing concerns and high degrees of uncertainty.

*

The second is a general concern that diploma privilege is superior to the traditional written bar exam. My own views on the traditional written bar exam are decidedly mixed.

I think the bar exam today reflects a fundamental distrust of law schools (that law schools, either specific schools or a cohort at given schools, admit, retain, and graduate too many students who lack the minimum competence to practice law) and the American Bar Association’s accreditation practices (the belief that they are too loose and fail to enforce admissions, retention, and standards, perhaps in part due to Department of Justice pressure). But I’ve suggested there’s trust in Wisconsin (and just two schools), and there might be opportunities for law schools and state bar licensing authorities to start developing that trust with one another again. On the other hand, the bar exam closely tracks law school performance, and performance on the bar tracks ultimate misconduct rates, which suggests a close consideration of what the alternative mechanisms will look like before switching admissions practices.

Frankly, proposals like Utah’s and Oregon’s suggest traits I’ve looked at before in trust between bar licensing authorities and law schools: in-state trust of two or three schools with already high passage rates (typically well over 80%) on the first attempt, and some trust of highly-performing out-of-state schools. It’s also a very different model than, say, Wisconsin, which has long required a series of state-specific courses as a condition of benefiting from the diploma privilege. Is the object of law school state-specific knowledge? Or general legal competence? Wisconsin’s is more the former, Utah’s and Oregon’s emergency proposals more the latter.

It is also, I think, very different to think of a how-to-handle-an-emergency diploma privilege, and ex-ante-new-bar-licensing-regime diploma privilege. The latter—the long-term change—would, I think, require a years-long phase in so that law schools could ensure they’ve adopted the admissions, retention, and graduation standards that the state bar prefers.

Statements claiming that the bar exam does “nothing,” then, or is “worthless,” or should be “abolished,” are, in my view, really a much deeper, and longer, debate—including what would replace it. I think it would require this “state-specific v. general competence” question to be addressed. It would require schools to reconsider admissions, retention, and graduation standards. It would reduce the role of the ABA and give greater scrutiny to what the ABA deems “compliant”—or at least turn the ABA into heeding state bar licensing authorities more closely. It would reconsider what post-graduation CLE and discipline looks like. It would also reexamine the role of character and fitness examinations (also deeply questioned in the academic literature) and the MPRE.

In short, I’m not a particular fan of “immediate diploma privilege now” because I think the bar exam still serves an important role in constraining law schools in their admissions, retention, and graduation standards. I think the bar exam isn’t necessarily the test I’d design—heavy on rote memorization and offered only twice a year—but I think replacing will take some effort. And any new program, even if instituted today, would likely only begin with the cohort graduating in the Class of 2024, after a year of admissions policies are in place with three years of retention and graduation standards for this cohort.

Now, I don’t say this as a desire to preserve the status quo by kicking the can down the road, postponing indefinitely to “study” issues to reach no resolution, and so on. It’s just to say—and I’m not a part of any of these discussions!—that a lot of thoughtful people have been thinking about a lot of thoughtful alternatives (versions of diploma privilege among them, but supervised practice, “bridge to practice,” etc.) for some time, and maybe state bar licensing authorities will finally get serious about these alternatives after this pandemic.

And it’s a reason I think this big debate is very different from the emergency debate. I think the bar exam has a place. I think it could be replaced. But that all requires a very different and much more structural reorientation of legal education and the state bar licensing regime than the pressing and temporary exigencies of administering a bar exam during a pandemic.

*

One more point. This is a debate over license standards—should a law license be conditioned on passing the bar exam? Graduating from an ABA-accredited school? Hours of supervised practice? One might have a different debate entirely about whether a bar license itself should be required.

Licenses, I think, can serve a public protection purpose. After all, if you may be imprisoned, we may think the average consumer (here, criminal defendant) lacks the ability to engage in the assessment of the quality of a self-described lawyer, at least when compared to the fairly catastrophic outcome of having incompetent representation. On the scale of “licenses for hair braiding” to “licenses for performing open heart surgery,” maybe law is closer to medicine than beauticians.

But we can still think about legal reforms where we don’t require a full J.D.-plus-bar-passage to do some legal services. Washington State has experimented (very modestly) with this for certain family law disputes. Utah and Arizona are moving toward narrowing the definition of “unauthorized practice of law” to allow more limited opportunities to engage in law-related activities. And maybe we’ll get to a point where passing the bar is not required to practice all forms of law, and there can be licensed and unlicensed practice, much like, say accounting.

Conditioning practice of law on passing the bar exam is, maybe in some ways, less onerous than conditioning practice of law on completing three years of education at an ABA-accredited school. In most states, however, you have to do both; in some states, you can complete alternative forms of education.

Maybe this debate will also spill over into our discussions about what the “practice” of law is, and who should or should not be excluded from that. Diploma privilege is (probably) one way of broadening opportunities for the practice of law. Other ways might follow.