But results should be approached with caution
A case-predicting robot has proved victorious in a man vs machine challenge previously described as “arrogant nonsense” by a top solicitor.
The prediction-off was organised by a bunch of Cambridge law students and graduates who first broke into the lawtech scene with their crime-identifying LawBot. Several rebrands later, the team is now called CaseCrunch and decided to prove its tech’s worth by pitting it against the brainpower of some of the country’s top lawyers.
According to CaseCrunch 112 lawyers took part in the challenge, these hailing from the likes of Allen & Overy, DLA Piper, Bird & Bird, Berwin Leighton Paisner and Eversheds Sutherland. These legal eagles were presented with factual scenarios of payment protection insurance (PPI) claims and were asked what the outcome of the claim would be, while the same scenarios were fed through CaseCrunch’s bot for predicting too. Two judges, a Cambridge law lecturer and a big data director, were tasked with making sure the challenge was fair.
CaseCrunch has now announced its win over the human teams, scoring an accuracy of 87% compared to lawyers’ 62%. The figures are impressive, but should lawyers be worried? Pinsent Masons‘ David Halliwell, speaking before the result was revealed, did note a machine victory could be a nightmare for lawyers:
@PinsentMasons is a sponsor for the #LawyerChallenge.
Apparently it will be either a "dream come true" or a "nightmare" – what do you think? pic.twitter.com/T2Ek5y9QFi— CaseCrunch (@Case_Crunch) October 22, 2017
But let’s approach this victory with caution. In the words of CaseCrunch’s scientific director, Ludwig Bull:
“These results do not mean that machines are generally better at predicting outcomes than human lawyers. These results show that if the question is defined precisely, machines are able to compete with and sometimes outperform human lawyers.”
What the robot cannot do, at this stage anyway, is emulate the personable nature of legal services. This was the ammunition behind top litigation lawyer David Greene’s CaseCrunch-directed outburst, in which he described the challenge as “arrogant nonsense”. Greene later told Legal Cheek:
“That arrogant toad ‘I told you so!’ Susskind has for long been banging on about technology and its effect on legal services. But his is generally about digital technology and the place it has in making the processing and provision of legal advice and assistance more efficient. That does not counter the fundament of the business being a people to people business.”
In response, Bull told us: “machines will not replace lawyers and [we aren’t] trying to change that. Machines can help lawyers understand the law and maybe even make it clearer and more just.”