Prompt engineering is the process of designing high-quality prompts that guide LLMs to produce accurate outputs. This process involves tinkering to find the best prompt, optimizing prompt length, and evaluating a prompt’s writing style and structure in relation to the task.
The Google’s prompting guides provide samples and effective prompting examples
Prompting techniques
Zero shot, one-shot and few-shot
With zero-shot or general prompting, no examples are provided. However, sometimes the response is correct, and the next step is to provide one example (one-shot) or few examples.
System, contextual and role prompting
System, contextual and role prompting are all techniques used to guide how LLMs generate text, but they focus on different aspects:
- System prompting sets the overall context and purpose for the language model. It defines the ‘big picture’ of what the model should be doing, like translating a language, classifying a review etc. (return a single word, for exaple)
- Contextual prompting provides specific details or background information relevant to the current conversation or task. It helps the model to understand the nuances of what’s being asked and tailor the response accordingly.
- Role prompting assigns a specific character or identity for the language model to adopt.
Step-back prompting
Step-back prompting is a technique for improving the performance by prompting the LLM to first consider a general question related to the specific task at hand, and then feeding the answer to that general question into a subsequent prompt for the specific task.
This ‘stepback’ allows the LLM to activate relevant background knowledge and reasoning processes before attempting to solve the specific problem. By considering the broader and underlying principles, LLMs can generate more accurate and insightful responses. Step-back prompting encourages LLMs to think critically and apply their knowledge in new and creative ways. It changes the final prompt doing the task by utilizing more knowledge in the LLM’s parameters than would otherwise come into play when the LLM is prompted directly.
Chain of thought
Chain of Though is prompting is a technique for improving the reasoning capabilities of LLMs by generating intermediate reasoning steps. This helps the LLM generate more accurate answers. You can combine it with few-shot prompting to get better results on more complex tasks that require reasoning before responding as it’s a challenge with a zero-shot chain of thought.
Self-consistency
Self-consistency combines sampling and majority voting to generate diverse reasoning paths and select the most consistent answer. It improves the accuracy and coherence of responses generated by LLMs.
Self-consistency gives a pseudo-probability likelihood of an answer being correct, but obviously has high costs. It follows the following steps:
- Generating diverse reasoning paths: The LLM is provided with the same prompt multiple times. A high temperature setting encourages the model to generate different reasoning paths and perspectives on the problem.
- Extract the answer from each generated response.
- Choose the most common answer.
Tree of Thoughts (ToT)
It generalizes the concept of CoT prompting because it allows LLMs to explore multiple different reasoning paths simultaneously, rather than just following a single linear chain of thought. The following image shows CoT on the left vs ToT on the right
.png)
ReAct
ReAct (REasoning and Acting) is a prompting technique that combines reasoning with action-taking. This action-taking capability makes the system different from traditional CoT or ToT setups/prompting
Automatic prompting
Automatic prompting is the usage of the model to select a prompt:
- the model is asked to generate multiple prompts
- the model is then asked to evaluate the prompts
Multi-modal prompting
It refers to a technique where you use multiple input formats to guide a large language model, instead of just relying on text. This can include combinations of text, images, audio, code, or even other formats, depending on the model’s capabilities and the task at hand.
Best practices
- Provide examples (a la one-shot, few shot)
- Make prompt concise, clear and easy to understand
- Be clear about the desired output
- Provide instructions, ideally positive instructions. Avoid telling the model what it should not do
- Control the response length
- Use variables in prompt to re-use them
- For few-shot prompting with classification tasks, provide examples from various classes
- CoT: always use temperature 0, since there is only one good answer
- Document various prompting attemps