The LLM matching task uses a chatbot content data source in a knowledge base to find passages that match the chatbot user's question. You can limit the search to specific conversations and passages.
You need an administrator or publisher role in your team to change a task prompt.
The task contains one prompt:
-
LLM Matching returns the passage ID of the passage that best matches the chatbot user's utterance.
Prompt variables: ${userQuestion}, ${chatHistory}, ${content}.
Check the top of the usage page to see if this is the primary usage and whether your changes must be published to take effect. If there is no note indicating your changes must be published, then the changes are applied immediately and will not appear in versions.
You can use automated tests to test a prompt or model against a set of example questions and generate suggested prompt improvements.
Changes to this prompt may impact your users' experience when the chatbot is matching their questions, so you should test your changes thoroughly before you publish. In addition to automated tests, you can test the prompt by asking questions in TestBot. LLM matching responses are shown in the TestBot logs as Detection Type: LLM Matching after the passage that was started.
Prompt variables and requirements
The prompt uses the following prompt variables:
- ${userQuestion}: the original user utterance to rephrase.
-
${chatHistory}: the transcript of the chat so far, which can be included to add more context to the chatbot user's utterance.
You can control whether the chat history is included using the Populate ${chatHistory} field. - ${content}: the knowledge base content to search.
You don't need to provide a value for these variables: they're automatically supplied when the chatbot uses the task to match a passage. Make sure you don't change or delete any of the prompt variables.
Suggested customisations
The default prompt instructs the model to return only the passage ID. You should keep this instruction as it is to ensure the LLM matching feature functions properly, but you may like to adjust the instructions to be more specific to your needs. Consider your passage content and the types of changes that would best help the LLM model determine whether a passage answers a user's question:
- Add your organisation's context, such as a banking organisation or city council.
- Request that irrelevant details such as dates, email addresses, or phone numbers are ignored when matching passages.
- Provide specific instructions if the utterance contains a specific term or the intent of the utterance matches that term.
This technique can help provide appropriate matching for terms relating to regulatory compliance or other highly sensitive issues, where the chatbot should respond with a passage that does not directly address the question.