News

‘Bin online exams’, says top barrister

By on
24

12KBW’s Daniel Sokol voices concerns over cheating and the impact ChatGPT could have on degree credibility

A barrister who represents students accused of cheating has shared his concerns surrounding online exams, and suggested binning them altogether.

In a recent opinion piece, Daniel Sokol, a barrister at 12 King’s Bench Walk, offers his perspective on the transition to online exams amidst the pandemic, saying the number of queries he’s received has increased.

After hearing about the prevalence of cheating from his clients, he conducted a survey of 900 university students across the UK. As many as one in six students confessed to cheating in online assessments this academic year, whilst more than half claimed to know someone who had cheated.

Sokol’s survey further found that 79% of students believed it was easier to cheat in online exams than in the traditional in-person format.

Popular methods of cheating reported included contacting friends and family or using a search engine on a separate device. Sokol suggests the increasing accessibility of artificial intelligence (AI) platforms like ChatGPT may provide another method for students to cheat.

He tested the viability of this theory by asking ChatGPT to provide a 100-word answer to a question. The chatbot responded with what former university lecturer Sokol judged to be a “decent effort, produced in seconds” and “sufficient to obtain a low 2:1”.

The 2023 Legal Cheek Chambers Most List

When probed further by Legal Cheek and whether this problem is particularly relevant for law exams, Sokol told us: “For law students, ChatGPT and other AI modules would be most helpful for essays and dissertations and less useful for problem questions and multiple choice exams based on particular scenarios.”

Whether this will push universities to return to in-person examinations is unclear but this new technology may drive them to increase monitoring for online exams. “Law schools need to convey to students what is and is not permissible with regards to the use of AI in assessments,” says Sokol. He gives the example of an essay question asking about the problems with a fault system of accident compensation and the case for reform. “Can a student put the question to ChatGPT and use the response as a starting point for their answer? Does that constitute academic misconduct? Does that depend on the nature and the degree of change from the original AI response?”

Sokol advises caution to any students tempted to use AI in their assessments, warning them to “ask academic staff, in advance and in writing, about the rules on this issue to avoid inadvertent (and potentially career-altering) breaches of academic integrity”.

He also suggests in his article that universities conduct all important assessments in-person as “that is the only way to maintain standards and reassure employers and others that a candidate’s degree was obtained honestly”. He adds:

“It is now too easy to cheat in online exams and too many students are currently doing so, undetected. This trend will only worsen with the development of AI. Sometimes, old-fashioned methods are the best.”

This is not the first time ChatGPT has made the news. In a recent exclusive by Legal Cheek, one student demonstrated that it could successfully answer questions on the Watson Glaser test.

For all the latest commercial awareness info, news and careers advice:

Sign up to the Legal Cheek Newsletter

24 Comments

has sat online exams

100%. Online exams are a joke.

(23)(13)

Yawn

Another comment about online exams from someone who never had to take them and never had their studies disrupted by a pandemic…

1. *Yes*, they are easier than in-person exams, but so what? What is the legal profession’s obsession with exams – the GDL/LPC/BTC are ultimately not that difficult whether you take them in-person or online? The format hardly matters. The vast majority of those who currently pass online exams would pass in-person exams, but online exams have actually enabled neurodiverse/disabled candidates to perform better than in in-person exams. So where is the harm in having online exams?

2. Doesn’t this barrister have anything better to do than worry about trivial exams which he is not going to have to take again? Strikes me of a ‘well we had it tough, so you must have it tough’ attitude towards younger people.

3. One of the *few* positives that came out of the pandemic was the re-thinking of exams and what it means to test knowledge. In practice and in life, you will have access to near-infinite sources of material. Why arbitrarily test someone’s knowledge/understanding *without* those sources? Questions can easily be tweaked so that ChatGPT and other AI tools don’t give good answers to them – the logic in those programs is REALLY quite terrible so would not stand up to any sensibly-written exam Q and therefore would prove useless.

(34)(16)

Me

These are moderately sensible arguments, but all miss the point that the piece is trying to make – that online exams provide little meaningful way to verify that the work submitted was produced wholly and exclusively by the candidate.

(10)(3)

Thank you for your response

Fair enough in response to my arguments – perhaps I got the tone/thrust of the piece wrong.

But there are plenty of ways to verify the work submitted in online exams – Inspera (used by BPP for example) is a ‘lockdown browser’ + webcam-viewed which prevents you from accessing anything other than hard copy notes, so is a pretty good way of verifying work in an online exam, no?

The reason I made the comments I did is because there is almost no sensible discussion of online exams made by people who have not had to take them. The default position is ‘oh it’s easy to cheat let’s scrap them’ rather than actually engaging in any critical thinking about the positives and negatives.

(12)(1)

Manners

You can’t get Google to do replies at the end of a 1 day hearing.

(7)(0)

Gladiatrix

Daniel and other barristers represent students accused of cheating, which you could have found out very easily. His concern is students being unfairly accused because of a lack of guidance about the use of AI.

(2)(1)

Get That Bag

When I did the BPTC, my personal tutor voiced concerns at the considerable numbers of students relying on 15+ year old reports to be allowed up to 50% extra examination time.

If a £60k pupillage could be at stake, I’m not surprised at what people are willing to do.

(8)(11)

Laura

You do realise most people get diagnosed as a child for disabilities right? And those disabilities are for life? Why would it even matter if the reports were 15 years ago for a disability like dyslexia/autism/adhd?

Sounds like ableism at its finest and certainly not the position of someone in education to dispute.

(11)(2)

ren

online exams are easier to do but the questions are often adjusted to be “harder” to factor that in, as compared to in person exams which are ” harder” to do

but on scale the questions become easier

might even balance out some way or another

(10)(1)

Ox 2nd year

I can’t speak for other universities, but Oxford has managed to adjust the difficulty of final exams last year such that the proportion of firsts awarded is identical to pre-online levels.

There seems very little reason to actually move back to in-person exams. ChatGPT doesn’t cite cases or statutes. If that is enough to score a low 2.1 at your unviersity in law, congratulations, your degree is worthless anyway.

(10)(4)

Oxo 2nd year

A 2.1 from anywhere outside the very top universities is best viewed as just a 2:2 in old money.

(12)(8)

Archibald O'Pomposity

Adjusting the difficulty of the questions is NOT the same as adjusting the grade/class thresholds. Which one was it?

(1)(1)

ox 2nd year

yes it literally is

harder question = less people write first class answers

why would the class threshold be adjusted? it’s standardised as 70 everywhere for a first

(1)(1)

Anon

Not really relevant but ChatGPT can cite cases and statues with the right input.

(3)(1)

Instathat

Closed book intense back-to-back in person examinations are a better test of what is needed for the Bar.

(6)(1)

anon2

Obvious troll is obvious.

(2)(4)

Why?

Why? It seems a fair point.

(2)(0)

Laura

Do you know who gets affected disproportionately by in-person exams? Neurodivergent people. For many, this is the only way that university and higher education becomes accessible. Penalise the people that choose to cheat, yes, put in better controls, make them harder to reflect the online nature (personally, if you can type in a question on the internet or using AI or whatever and get the direct answer, your questions are too simple for graduate level) but don’t scrap them all together.

Sick of discussions like this being had without reference to a significant proportion of the student population. Shame on legal cheek and the barrister for not mentioning it.

(8)(11)

Archibald O'Pomposity

If, and only if, neuro-divergent lawyers who have sat neuro-divergent-friendly exam formats achieve at least as effective legal performance in real roles as their non-neuro-divergent counterparts who sat traditional exams, then you may have a point. Adding in emotive language doesn’t enhance your argument, mind.

(5)(2)

Gladiatrix

This is about the potential deleterious impact on all students, there is nothing ‘shameful’ about it and your censorious response is just silly.

(4)(3)

Me, myself and horde of unsavoury characters

I’m neurodiverse and I cannot disagree more with Laura’s post as to what suits me when it comes to exams. For me, in-person exams are far better than online and closed book exams and better than open book exams. I know other people are different. The OP lumped all of the neurodiverse community into a single group and that does not work. That does not mean that it is not a relevant consideration, it clearly is, but what matters is greater specificity as to who would be affected in the way the OP says.

(2)(1)

Sad Barrister

Sokol’s article is a bit confused, but its thrust is correct.

If ChatGPT continues at its current rate of improvement, such that it is able to turn out e.g. really first-class historical or legal analysis rather than just low-mid 2i work (as at present), open book exams and coursework will no longer be a plausible method of examining students. Ironically, this may impact Masters/PhDs just as much as undergraduate work. It is quite likely that, in the next five years, an AI will be able to bang out a serviceable PhD in (at most) a few hours.

The inevitable conclusion will be closed-book exams. Whether online or in-person, these exams will have to be strictly invigilated. Sokol is probably wrong to suggest that strictly-invigilated online exams are *so* much worse than in-person exams that they are the inevitable solution. Online invigilation services will themselves be able to use AI and ML to determine unusual patterns in student behaviour (e.g. a sudden flurry of improved writing after a “toilet break”, or unusual eye patterns suggesting hidden notes or another monitor). As outlined by other commenters, online exams do bring real benefits to students.

This is quite a sad state of affairs. If you could guarantee honestly, open book exams would be a great idea (at least in some instances) because, as Sokol would know from his own Masters and PhD, thinking skills (and, more generally, contributions to knowledge) don’t come from rote-learning but from close cooperation and discussion with the work of others.

(5)(0)

Archibald O'Pomposity

“If you could guarantee honestly [sic]…”

If you could guarantee honesty in this context alone, there wouldn’t be a need for lawyers.

(0)(3)

Crimbo SAQ monster

What? Don’t BPTC students still have to turn up to a hangar in Docklands to sit exams with 9,000 others at once?

Them were t’days.

(3)(0)

Comments are closed.

Related Stories