Since its wide release in November last year, students’ access to OpenAI’s ChatGPT, an artificial intelligence programme which pulls information from a finite dataset to produce a response to any prompt or question input, has been contentious. In January, a lecturer from Deakin University, Australia, claimed to have detected the rule-breaking use of ChatGPT in a fifth of the postgraduate summer assessmentsshe marked.
Across the Atlantic, the software made headlines after taking the United States Medical Licensing exam and the Bar exam, passing (or thereabouts) both. In New York, Virginia, California, and Alabama access to ChatGPT has been denied to all devices accessing the internet via public school servers.
Both Turnitin, a plagiarism detection software used by universities internationally, and OpenAI themselves are currently developing software to accurately detect the use of chatbot-produced work. Turnitin has also advised teachers that chatbot writing might be manually identified for its repetition, blandness, and lack of a distinct voice.
It’s a formative time for AI’s relationship with education. We’re watching decisions being made in real time which are going to shape how the evolving technology is incorporated into curriculums of the future- or entirely rejected from it.
Despite the distrusting response AI has demonstrated online in the past months, news from the International Baccalaureate (IB) last week seems to mark a coming tide change. The IB is a programme of study and assessments equivalent to UK A Levels offered in over 140 countries and 120 schools in the UK. And, as of this week, it will tolerate students’ use of ChatGPT in their submitted assessments.
IB’s headline is a little misleading. The Times reported that Matt Glanville, head of assessment principles and practice, said IB students must handle ChatGPT as they would ‘any quote or material adapted from another source, it must be credited in the body of the text and appropriately referenced in the bibliography.’ Glanville also addressed concerns aroundstudents copying work produced by the bot and submitting it as original work for assessment. Taking a similar line to Turnitin, Glanville said students’ access to AI writing software would necessitate changes to how students are assessed. Now that AI can produce essays ‘at the touch of a button’, Glanville said educators will need pupils ‘to master different skills such as understanding if the essay is any good or if it has missed context, has used biased data or if it is lacking in creativity.’
The decision not to penalise students who use ChatGPT in their IB assessment submissions seems forward thinking. Assessors are embracing important changes to the capabilities of new tech rather than clinging to the tried and tested. When we look at the fine print of this choice, however, a question arises; are we missing the point of burgeoning AI tech like ChatGPT?
Glanville lays out parameters which will allow students to use ChatGPT as a referenced information source, something to include in your bibliography alongside peer reviewed scholarship. But that is not what the software is designed for. Not only that, but the characteristics of its nature as a chatbot directly inhibit its use as a researching tool with complete accuracy.
Primarily, this is an issue derived from the data ChatGPT uses to form its answers. According to Science Focus, this database is pretty limited. It struggles to pull information any more up to date than 2021, which I tested this by asking the bot some pop culture questions, such as the name of Paris Hilton’s new baby and the UK’s ruling monarch. ChatGPT failed to correctly answer both. Or perhaps, in the case of the latter, it’s still in a state of denial.
Although its database is 570GB and 300 billion words strong, it’s fixed. If the most accurate and up-to-date information necessary to a user’s question isn’t in this database, ChatGPT will produce a poor or incorrect answer. If you were dead set on using an AI chatbot as a researching tool, though, an alternative which would likely yield better results would be ChatSonic, a lesser-known bot which pulls from Google search to answer its prompts with the most current available information. There are drawbacks to this too, the most obvious being the bot’s vulnerability to misinformation. By sourcing information more broadly, ChatSonic runs the risk of regurgitating anything currently circulating online in its answers?
ChatGPT also has accuracy issues. According to the Entrepreneur, ChatGPT scored a B to B- on the final exam of a typical Master of Business Administration course. It produced incorrect answers to basic maths questions and answered poorly when asked analytical process questions. The myriad of horror stories detailed online about the bot completely fabricating references and bibliographies attest to the fact that ChatGPT just is not an accurate research tool. And why should it be? It’s a chatbot.
ChatGPT was at capacity the first few times I tried to access it, but after finally gaining access I tried feeding it a few GCSE exam questions to see how it would fare.
In response to an English language prompt to debate the fast fashion industry, the bot produced a few balanced, lackluster paragraphs. ChatGPT’s conclusion was almost compelling as it arrived at a middle-of-the-road “fast fashion bad” close, which is quite a lot more opinion than I had come to expect from the bot. To its detriment, its use of first-person plural pronouns was a little jarring coming from a computer and it estimated of the fast fashion industry’s value at $2.5 trillion dollars. I thought this seemed a little high, and a brief Google search suggested the bot seems to have pulled that out of thin air. (The value I found for 2021 was $91,226.92 million. I’m no mathematician, but that seems like a reach.) Otherwise, but it didn’t feel too far estranged from any run-of-the-mill school essay.
When I gave ChatGPT a GCSE history question about Medieval and Early Modern law, it didn’t do so well. Although balanced, the bot’s answer was noticeably dry. Its arguments were generalised, thrice repeated, and only one historical example to support its argument. To cut it some slack, though, I doubt OpenAI thought to add any AQA mark schemes into its database. By rewording the question, I could draw a more useful set of bullet points similarities and differences from the bot, and when I asked it to write an essay from those bullet points instead, the answer was pretty comprehensive.
In the spirit of investigative integrity, I also promoted ChatGPT to write this article about the controversy surrounding its use in education. ChatGPT told me it could ease workload for teachers if employed as a grading tool, increasing efficiency and objectivity. It highlighted the risks of students using it to plagiarise and its own potential unreliability. It was a pretty dull, utilitarian read because, as the bot reminds you whenever you ask it anything political or opinion based, it’s an AI language model. Besides the bare facts it uses to build answers, it just doesn’t have anything to say about fast fashion, or Early Modern Law, or even its own validity. For better or worse, it’s just not quite good enough for that yet.
IB’s decision to invite its students to understand and use ChatGPT is an outlier to the trend across education, but it could be a future-proof one. Students currently using ChatGPT is assessments, either with the blessing for their assessors or not, are failing to understand its intended use and the drawbacks of them. Assessments which use chatbots too heavily will suffer, but this won’t always be the case. ChatGPT may be one of the first chatbots to capture so much international attention, but it won’t be the last. And the technology, as is already happening in ChatGPT’s competitor, ChatSonic, will become more reliable and useful in education settings.
Educators who set a president of exclusion in AI’s infancy are setting themselves up for some long-distance backtracking at best. At worst, a cohort of students and teachers are going to find themselves grappling with an unnecessarily steep learning curve further down the line when the usefulness of AI in education becomes undeniable. The technology will improve, and it will become more integrated into both our work and play. Resistance isn’t just futile, it’s self-sabotaging.
Feature image by Christin Hume on Unsplash.