The Problem
When people first learn about ChatGPT's abilities, one of their main worries is that it could be used by students to plagiarise their work. While I normally like to focus on the positive aspects of AI, I admit that this is a valid concern.
A particularly difficult area for language teachers are the speaking exams, as the students can prepare many of their answers in advance and then learn them, both at GCSE and at A-Level (if you are not familiar with the UK system, there is a short overview here). The question is, in how far are students allowed to use AI to prepare and practise for these exams?
As languages teachers we have of course had similar problems with translating websites for years because some students have been using them to write their answers. The use of translating software is difficult to prove conclusively and is often not followed up rigorously , even where it is obvious. Many teachers feel that the effort of learning the structures off by heart so they can be performed confidently and pronounce correctly, is enough to earn them the grade, even if they had help with the content.
The availability of ChatGPT now exacerbates the problem because the students need to put even less effort into creating the content as they don't even need to write an English version of their texts and they can use the ChatBot to create answers that are very similar in style and level to what they would normally produce or to what is expected in the exam.
On the other hand, AI can be an amazing tool for speaking practice and gives the students the opportunity to experience a dialogue that is less predictable and therefore more realistic than the conversations they normally have in a classroom or exam setting. Many companies are now developing AI-powered learning apps that can help our students practise all aspects of the speaking exam, including listening, pronunciation and grammar (see my article here). But where does practice end and cheating start?
The Exam Boards
In spring 2023, the Joint Council for Qualifications (JCQ) , "a membership organisation comprising the eight largest providers of qualifications in the UK", published their guidance document "AI Use in Assessments: Protecting the Integrity of Qualifications - Guidance for Teachers & Assessors". This document lays the foundation for exam boards in the UK on writing their own guidance and rules on using AI in official exams like GCSE and A-Level, but they have been rather slow to do so.
Here is a summary of the JCQ document (by ChatGPT)
Acknowledging AI Use (Page 7): Students must reference the sources they used when producing work for an assessment, including AI tools. If an AI tool provides details of the sources it used to generate content, these sources must be verified by the student and referenced in their work. Students must also acknowledge the use of AI and show how they used it. This allows teachers and assessors to review whether the use of AI was appropriate in the context of the assessment.
What is AI Misuse (Page 5): AI misuse is defined as the submission of work for assessments that is not the student's own, which can be considered a form of plagiarism. Examples of AI misuse include copying or paraphrasing sections of AI-generated content, using AI to complete parts of the assessment, failing to acknowledge the use of AI tools, and submitting work with intentionally incomplete or misleading references or bibliographies.
Other Ways to Prevent Misuse (Page 8): To prevent misuse, education and awareness of staff and students is key. Actions that should be taken include restricting access to online AI tools on centre devices and networks, setting reasonable deadlines for submission of work, examining intermediate stages in the production of work, and not accepting work which staff suspect has been taken from AI tools without proper acknowledgement.
Centre Engagement with and Discussion of AI (Page 6): Centres must discuss the use of AI and agree on their approach to managing students’ use of AI. They should make students aware of the appropriate and inappropriate use of AI, the risks of using AI, and the possible consequences of using AI inappropriately in a qualification assessment.
Automated Detection (Page 10): Several programs and services use statistical analysis to determine the likelihood that content was produced by AI. These include the OpenAI Classifier, GPTZero, and The Giant Language Model Test Room (GLTR). These tools could be used as a check on student work and/or to verify concerns about the authenticity of student work.
I would like to add a word of caution to the last point: Automated detection programs are currently very unreliable and I personally would not feel confident to accuse a student of cheating, purely based on the outcome of one of those checks. There seems to be constant discussions on this topic on social media, with many teachers showing how their own texts have been labelled as AI generated even though they wrote them themselves.
The survey
As far as I can tell, there has been no specific guidance on AI in MFL and even our visiting A-level examiner in summer 2023 said he had no training or guidance from the exam board on this.
As there was so little official guidance, I designed a survey for language teachers to hear where they stand on this issue. As I write this, only 14 teachers have answered the survey, but I hope the questions will spark some discussion among school staff and between teachers and their students about what is acceptable and what is not. Below are the results so far and a summary written by ChatGPT. The survey is still open and can be filled in here.
This survey is aimed at UK MFL teachers: In your opinion, where does plagiarism starts in regards to using chatGPT and similar to prepare for SPEAKING exams, e.g. GCSE general conversation or IRP for A-Level.
1= That’s cheating, 5=I would encourage my students to do this.
Any other comments on plagiarism?
A lot of these aren't cheating. But I wouldn't encourage them because they sound like a bad idea.
They still need to be the judge for AI sources.
Plagiarism in this case is using technology to write your answers (or do the work for you). Using it to practice it is not plagiarism is using technology or AI as an aid for learning (i.e. text to speech to learn pronunciation - AI to ask questions on the IRP)
Summary of Results (by ChatGPT)
The results of the survey show varying opinions about different methods of using AI, such as ChatGPT, in preparing for speaking exams. From the data provided, I can observe that there are several methods that have a wider range of responses, indicating that they may be more controversial among the respondents.
1. "Students ask AI to write answers to speaking questions in the general conversation and learn it off by heart" has ratings ranging from 1 to 5. This indicates that some teachers consider this as cheating while others encourage it. This could be controversial as it brings up the debate on whether learning answers created by AI is an authentic form of learning, or if it is undermining the student's ability to think critically and respond spontaneously.
2. "Students ask AI to come up with answers to possible questions that they might be asked on their IRP" also has a wide range of ratings from 1 to 5. Similar to the first point, this could be seen as controversial due to the debate on whether students should be coming up with their own answers to demonstrate understanding, or if using AI-generated responses is a valid form of preparation.
3. "ChatGPT writes the full research text for them in English and students translate it into the target language themselves" shows ratings mainly at the lower end (1 and 2) but also has a 5. This indicates that while most teachers might consider this as cheating or disapprove of it, some might see it as a viable way for students to focus on translation skills. This can be controversial as it highlights the tension between focusing on content creation vs. language translation in MFL education.
4. "Students using AI to proof-read and correct answers to general conversation questions that they have written" has ratings ranging from 3 to 5, showing that most teachers are somewhat positive or neutral about this method. However, it could still be controversial among educators who believe that students should learn to proofread and correct their work without the help of AI.
The most controversial question appears to be the second one, "Students ask AI to write answers to speaking questions in the general conversation and learn it off by heart", as it directly relates to the core of language learning, which is being able to communicate effectively and spontaneously in the target language. Teachers might have varying opinions on whether memorizing AI-generated responses is beneficial or detrimental to the language learning process.
Overview of UK speaking exams (written by ChatGPT)
GCSE (taken age 16)
1. Role Play: Students are given a role-play card with a scenario and a series of tasks to complete. They have time to prepare, and then they enact the scenario with the examiner, who plays another role in the scenario.
2. Photo Card: Students are given a photo related to one of the themes of the GCSE course. They need to answer a series of questions about it, which typically involves describing the photo, offering opinions about the situation in the photo, and speculating about aspects of it.
3. General Conversation: This is a broader conversation with the examiner about two of the themes studied during the GCSE course. Students need to be able to respond to questions, express opinions, provide information, and use a range of grammatical structures. Although this is meant to be a spontaneous conversation, in practice students know most of the questions in advance and can prepare answer for them.
Students are assessed on their pronunciation and intonation, range and accuracy of vocabulary and grammar, ability to respond to and interact with the examiner, and the fluency and coherence of their speech.
A-Level (taken age 18)
In the Modern Foreign Languages (MFL) A-Level speaking exam in the UK, as of my knowledge cutoff in September 2021, students generally need to undertake the following:
1. Discussion of a Sub-theme: The exam typically begins with a discussion on a sub-theme based on a photo card.
2. Presentation and Discussion of Individual Research Project (IRP): Students deliver a presentation (usually around 2 minutes long) on their individual research project. This project should be rooted in one of the themes or a subject of interest related to the culture/society of a country where the language is spoken. After the presentation, a discussion with the examiner about the project follows, about 8-10 minutes long.
Students choose and research the topic of their IRP in their own time and can only receive generalised help and feedback from their teachers and students should not speak about the same topic as their classmates.
The assessment focuses on students' understanding and use of the target language, their capacity to interact and hold a discussion in the language, their range of vocabulary and grammatical structures, and their pronunciation and intonation. Additionally, their ability to analyse and evaluate issues related to the sub-themes and their individual research project is evaluated.
(Back to top)
Comments