High-flying law students perform worse in exams when using AI

Avatar photo

By Rhys Duncan on


Lower performers saw significant benefit

The grades of high achieving law students suffer when they are given access to artificial intelligence tools, new research has found. But low performers saw a significant increase in performance.

Forty-eight law students at the University of Minnesota were given a final paper without AI, and then, after prompt training, were given a second paper and permitted to use GPT-4, OpenAI’s latest and most advanced AI model.

On average, AI-powered students scored 29% better than when in mere human form. For lower performing students this improvement was 45%, although for those at the top of their class scores dropped by a whopping 20%. Whilst the AI could boost multiple choice scores, its essay writing wasn’t quite up to scratch.

The 2023 Legal Cheek Law Schools Most List

Commenting on the results in their paper, professors Choi and Schwarcz said “GPT-4’s impact depended heavily on the student’s starting skill level…This suggests that AI may have an equalizing effect on the legal profession, mitigating inequalities between elite and nonelite lawyers”.

Earlier this month, Legal Cheek reported on another study suggesting that AI will lead to more legal work being done by those without traditional qualifications, with the profession opening up to experts in the computing and coding fields.

Many firms have already sought technological solutions to expedite more administrative processes like document review, freeing up time, and allowing the human machines to work on more complex issues.

Related Stories

Solicitor aims to become first ‘AI-powered MP’

Andrew Gray hopes tech-focused approach will lead to election success

Jul 19 2023 9:17am

Will AI replace lawyers?

Technology associate Phillippa Stubbs considers whether the rapid evolution of large language models means the end of legal careers

Jul 17 2023 10:55am