OpenAI, the company behind ChatGPT, has just published its prompt engineering guide, which brings together the main strategies for getting the best results from any model of ChatGPT, whether paid or free. You can find it here: Prompt engineering - OpenAI API.
Many of the tactics, or strategies, are obvious if you have played around with ChatGPT, while others are more technical and you might not realise that they could help solve more complex tasks. In this article, I will summarise the prompt engineering guide and provide some specific examples for (language) teachers. Here is an overview of their strategies and tactics, which will be explained in more detail below.
Write clear instructions
Include more details
Ask the model to adopt a persona
Label the parts of your prompt clearly
Specify the steps
Provide examples
Ask for a specific length
Provide reference text
Use a reference text
Ask for citations.
Split complex tasks into simpler subtasks
Use intent classification to identify the most relevant instructions for a user query
For dialogue applications that require very long conversations, summarise or filter previous dialogue
Summarise long documents piecewise and construct a full summary recursively
Give the model time to "think"
Instruct the model to work out its own solution before rushing to a conclusion
"Let's think step by step"
Write clear instructions
Include more details
The more specific details you provide, the better response you will get. Many teachers who use ChatGPT for the first time use vague prompts like "Write a lesson plan for a French lesson about technology." As a result, they are disappointed with the average and often uninteresting output. However, the quality improves with each additional piece of information you provide. Think back to when you were a trainee teacher – what kind of details did you need to include in your lesson plans for your mentor? You should provide the same level of detail to ChatGPT! Writing out all the details like in the example below might seem like a chore and some teachers might decide that it is quicker to plan the lesson themselves, but you can create your own collection of prompts that you can reuse for different classes and simply modify some of the details. It's a good idea to start your own prompt database, whether it's a simple document or spreadsheet, a Chrome extension like Superpower ChatGPT, or by creating a Notion site similar to this one: Languages Teaching Prompts Homepage.
Here is a detailed example prompt:
“I need a lesson plan for a 45 min lesson for a class of 12 year olds learning French. They are in their second year of learning and at CEFR level A1. I have a class of 25 students, they don’t have any devices but I have an internet enabled smartboard. I want them to be introduced to vocabulary to speak about using different types of technology in simple sentences as well as giving opinions. I want to use listening and reading input flooding, a whole group game as well as a paired activity. They have recently revised the present tense verb endings and I would like to include a reminder of that.”
Ask the model to adopt a persona
ChatGPT can be anything from a creative story writer to a life coach, but you need to let it know what kind of working partner you currently need. Some people have found that starting their prompts with "You are an experienced teacher working in a middle school" has improved the responses. You can also specify the type of voice you want the chatbot to have, for example, "Give short and precise answers without comments or explanations" or "Make your response informal and include a little joke."If you want, you can also include this type of information in your custom instructions to ensure that the chatbot responds consistently every time.
Label the parts of your prompt clearly
The most time-saving discovery for me personally was the use of placeholders. They allow you to save more complex prompts like the lesson planning one above in a way that makes it easy to change the parameters and adapt longer prompts.
Your parameters can take different forms:
“””Text”””
[Text]
<Text>
You can also label the parts in your prompt in a more natural way. For example:
"Instructions: Create ten sentences based on the vocabulary list below. Vocabulary list: computer=ordinateur…"To make your prompts even quicker to adapt to any situation, add placeholders in the main text and then list them at the end of the prompt, so you just have to fill in the variables.
For example, the long prompt above could look like this:
“I need a lesson plan for a [length] min lesson for a class of [age] year olds learning French. They are in their [year] year of learning and at CEFR level [CEFR]. I have a class of [number of students] students, they don’t have any devices but I have an internet enabled smartboard. I want them to be introduced to vocabulary to speak about [topic] in simple sentences as well as giving opinions.
Length:
Age:
Year:
CEFR:
Number of students:
Topic:”
Specify the steps
If you are using a more complex prompt with ChatGPT and the output is not what you are looking for, try breaking it into individual steps. Even if you give it all the steps at the same time, it will often improve the outcome. And if there are still problems, you can check each step to see where things have gone wrong.
Example prompt:
“Step 1: write a short paragraph in English about my last holiday.
Step 2: Translate it into Spanish
Step 3: list all adjectives from the English text.
Step 4: Create a table with these adjectives in English and Spanish.”
Note: ChatGPT has been trained in English more than in any other language. So, whenever possible, perform complex tasks, such as finding adjectives, in English and translate them later.
Provide examples
If you need texts in a specific style, providing examples will ensure that the new text is similar to what you are looking for. If you want a sample answer for an exam question, for example, the output will be more useful if you paste in example answers for similar questions, copied from textbooks or exam specifications. The same applies if you want to use the chatbot to write other texts, such as letters to parents or course descriptions.
Example prompt:
“Here is an example exam task: [paste in exam task] and here is an ideal answer for this task [paste in sample answer]. Now write me a sample answer for this exam question in a similar style and length: [new exam task]"
Ask for a specific length
ChatGPT is not very good at sticking to an exact word count, so if you ask it to write 90 words about "my last holiday", you will probably get anything between 70-100 words. Therefore, make sure you count the words and manually adjust the length if necessary. ChatGPT is generally better at producing a certain number of paragraphs or bullet points, so use these in your instructions whenever possible.
Provide reference text
Use a reference text
The main flaw in using ChatGPT is its tendency to "hallucinate" or invent answers whenever it is unable to find the correct answer immediately. One way to reduce this risk is by using prompts that explicitly forbid guessing.
OpenAi suggests this prompt (slightly adapted by me):
“Use the provided article in square brackets to answer the following questions. If the answer cannot be found in the articles, write "I could not find an answer. Here is the article [paste in article] and here are the questions: [questions]"
You should be aware that the longer the text, the less accurate the references will be and the less likely that the chatbot will find the appropriate part of the text. It has been shown that it is best at finding information from the very beginning and the very end of a text but less accurate in the middle. This effect applies mainly to very long texts of many pages, so if you are using shorter text like a newspaper article or a page from a textbook, you probably won't notice this effect.
Ask for citations
Asking for exact citations is an extension of the strategy above, and allows you to check the text quickly to make sure the answer is not hallucinated.
Here is the OpenAi’s prompt (simplified by me):
“You will be provided with a text. Your task is to answer the question using only the provided document and to cite the passage(s) of the document used to answer the question. If the document does not contain the information needed to answer this question then simply write: "Insufficient information." If an answer to the question is provided, it must be annotated with a citation. Use the following format for to cite relevant passages ({"citation": …}).
Here is the text: [paste text]
Here is the question: [ask question]”
To make sure that the answer is accurate, you can now use the provided quote to check if it is in the text by pasting it into the "find in text" feature (on PC browsers and documents, it can normally be started by clicking ctrl+F).
For tasks that require very long conversations, summarise or filter previous dialogue
Every dialogue with the chatbot has a limited "context window", meaning that as you keep chatting it starts forgetting what happened earlier in the conversation. This can be quite annoying for a longer project such as writing a story or planning an escape room. When I tried to plan a crime story it kept changing the characters' names and changing their motives and alibis. To reduce the risk of that, ask the chatbot from time to time to summarise the most important points of the conversation so far, to refresh its memory.Summarise long documents piecewise and construct a full summary recursivelyChatGPT has a limited "token limit" meaning it can only take in a certain amount of words at a time. In the free version, the current limit is about 1000-1500 words which can be a problem with summarising very long texts. OpenAI therefore recommends to cut up the text into chunks and summarise each one individually, then add all the summaries and ask for an overall summary. This should reduce the amount of hallucinations and make the end result more accurate.
Give the model time to "think"
Instruct the model to work out its own solution before rushing to a conclusion
As mentioned above, ChatGPT is trained to always provide answers and to provide them quickly, which means the chatbot sometimes prefers to give the first best answers rather than the correct one. It is especially weak with math and logic problems, even relatively simple ones. In most cases, I would avoid using the free version for these kinds of problems, but this strategy can improve outcomes.
Here are two examples provided by OpenAI:
USER
Problem Statement: I'm building a solar power installation and I need help working out the financials. - Land costs $100 / square foot - I can buy solar panels for $250 / square foot - I negotiated a contract for maintenance that will cost me a flat $100k per year, and an additional $10 / square foot What is the total cost for the first year of operations as a function of the number of square feet. Student's Solution: Let x be the size of the installation in square feet. 1. Land cost: 100x 2. Solar panel cost: 250x 3. Maintenance cost: 100,000 + 100x Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000
CHATGPT
The student's solution is correct.
But the student's solution is actually not correct! We can get the model to successfully notice this by prompting the model to generate its own solution first.
USER:
First work out your own solution to the problem. Then compare your solution to the student's solution and evaluate if the student's solution is correct or not. Don't decide if the student's solution is correct until you have done the problem yourself.
Problem Statement: I'm building a solar power installation and I need help working out the financials. - Land costs $100 / square foot - I can buy solar panels for $250 / square foot - I negotiated a contract for maintenance that will cost me a flat $100k per year, and an additional $10 / square foot What is the total cost for the first year of operations as a function of the number of square feet. Student's Solution: Let x be the size of the installation in square feet. 1. Land cost: 100x 2. Solar panel cost: 250x 3. Maintenance cost: 100,000 + 100x Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000
CHATGPT
Let x be the size of the installation in square feet. 1. Land cost: 100x 2. Solar panel cost: 250x 3. Maintenance cost: 100,000 + 10x Total cost: 100x + 250x + 100,000 + 10x = 360x + 100,000 The student's solution is incorrect. They made an error in the maintenance cost calculation, using 100x instead of 10x. The correct total cost for the first year of operations is 360x + 100,000.
Encourage the chatbot to “think step by step”
This last advice for prompting was not mentioned in the OpenAi prompt guide, but is a well known way of improving ChatGPT answers was proven to work in research for example by Cornell University (see here: ). The only thing you need to do is to add "Let's think step by step" at the end of your more complex prompts. In the Cornell study, this short sentence increase accuracy in some arithmetic tasks from 17% to 78%, so it is definitely worth using in any logic and maths tasks.
This last advice for prompting was not mentioned in the OpenAI prompt guide, but is a well-known way of improving ChatGPT answers that has been proven to work in research, for example by Cornell University (see here: Large Language Models are Zero-Shot Reasoners). The only thing you need to do is to add "Let's think step by step" at the end of your more complex prompts. In the Cornell study, this short sentence increased accuracy in some arithmetic tasks from 17% to 78%, so it is definitely worth using in any logic and maths tasks.
In conclusion, OpenAI's newly published prompt engineering guide provides a valuable resource for users looking to make the most of ChatGPT. By employing these strategies you can often get more useful results than through simple dialogue. Remember to adapt these tactics to suit your specific needs and create a database of prompts for efficient future use.
Comments