Did ExamSoft cause the bar passage rate decline?

I’ve blogged about the sharp decline in the MBE scores and the corresponding drop in bar passage rates in a number of jurisdictions around the United States. I’m still struggling to find an explanation.

One theory is that the ExamSoft fiasco affected the MBE scores. Most states have two days of exams: a day of essays followed by a day of multiple choice in the MBE. The software most states use for the essay response portion had problems in July 2014--test-takers were unable to upload bar results in a timely fashion. As a result, students slept less and stressed more the night before the MBE, which may have yielded lower scores on the MBE.

We can test this in one small way: several states do not use ExamSoft. Arizona, Kentucky, Maine, Nebraska, Virginia, and Wisconsin all use Exam4 software; the District of Columbia does not permit the use of computers. If ExamSoft yielded lower scores, then we might expect bar passage rights to remain unaffected in places that didn’t use it.

But it doesn’t appear that the non-ExamSoft jurisdictions did any better. Here are the disclosed changes in bar passage rates of July 2013 in jurisdictions that did not use ExamSoft:

Arizona (-7 points)

District of Columbia (-8 points)

Kentucky (unchanged)

Virginia (-7 points)

These states have already disclosed their statewide passage rates, and they do not appear to be materially better than the other scores around the country.

It might still be a factor in the jurisdictions that use ExamSoft in conjunction with other variables. But it doesn’t appear to be the single, magic explanation for the decline. There are likely other, yet-unexplained variables out there.

 (I’m grateful to Jerry Organ for his comments on this theory.)