Judicial ‘Pepsi Challenge’ fails as law students are unable to identify judges’ gender from their anonymised rulings

Avatar photo

By Alex Aldridge on

Durham University law students’ attempt to tell the gender of a judge on the basis of their judgment alone has yielded a success rate of just 46%.

The experiment — a legal version, if you will, of the famous Pepsi Challenge (a blind Pepsi v Coke taste test) — was conducted following prompting by Supreme Court President Lord Neuberger amid controversy at the lack of female judges…

After outrage was sparked earlier this year by the promotion of yet more men to the UK Supreme Court’s ranks (maintaining its shocking 12:1 male/female ratio), Neuberger was asked by Joshua Rozenberg on Radio 4 whether women justices offered a different insight from men when deciding cases. The country’s top judge responded that it would be useful to conduct “a parlour game” to find out.

“What I’d quite like to do — it’s a bit like a parlour game — is to get 20 reasonably good lawyers in a room and present them with 100 judgments — 30 by women and 70 by men — and say ‘right, which were by men and which by women’, because I’d be quite interested to see the result,” said Neuberger, adding:

“And of course, even if it was completely random, that wouldn’t prove that women didn’t bring something different. I think that Lady Hale believes they do. And I’m quite prepared to agree that they might, I just am not sure.”

In response, Durham Law School professor Erika Rackley assembled 30 of her students to study 16 Court of Appeal  judgments (half by men, half by women). Anything identifying the gender of the judges was removed from the judgments, which spanned employment, family and criminal law.

Alongside the headline 46% success rate finding, it was discovered that the students were slightly better at spotting a female-authored judgment in the employment and family cases. But their success rate dropped to just 33% for the criminal cases.

Following the experiment, Professor Rackley urged the continuance of efforts towards creating a more diverse judiciary. “No one claims that a judge’s gender will make a difference in all cases. In reality what this experiment tested was the students’ assumptions as to the possible differences between men and women judges, rather than the real differences between them, ” she said.

Rackley continued: “The argument for the importance of having a more diverse judiciary is not that we will have women judging one way, and men the other. Rather it is that the insights of both will be fed into the law and lead to better decision-making on all sides.”

The experiment is documented in yesterday’s episode of Law in Action.

The show includes an interview with Lady Hale, who describes the judiciary as “judges and lawyers first, and men and women second”. Hale also denies being in favour of positive discrimination to remedy the current gender imbalance at the higher echelons of law. But the Supreme Court’s only female member says that she agrees with the provision in the Equalities Act which allows a candidate from an underrepresented group to be preferred where they are one of two candidates of equal merit.