LLM Matching task

The LLM matching task matches passages to your chatbot user's utterance using a knowledge base of your chatbot content. LLM Matching runs after keyword questions have been matched, but before NLP question matching:

If you have enabled the rewrite utterance task, the matching conditions you configure will affect whether LLM matching uses the chatbot user's original utterance or the rewritten utterance.

The LLM matching task is in the TrueIntent usage. You need an administrator or publisher role in your team to create a usage.

By default, new usages you create are set as the primary usage (the usage the chatbot will use). You can create a usage without setting it as primary if you need to.

When you create the TrueIntent usage, the LLM matching task is disabled. Disabled tasks won't appear in your live chatbot, but you can still configure and test them using automated tests. When you're ready to use the LLM matching task in your live chatbot, enable the task and publish the change.

When editing tasks and usages, check the top of the usage page to see if it is the primary usage and whether your changes must be published to take effect. If there is no message indicating your changes must be published, then the changes are applied immediately and will not appear in versions.

LLM matching may match your chatbot user's questions to different passages than your NLP training phrases, so you should test your changes thoroughly before you publish. You can create an automated test to test your prompt against a variety of utterances and generate improvements.

You can also test the task by asking questions in the Test Bot once the task is enabled. Questions that have been matched by LLM matching are shown in the test bot logs as Detection type: LLM Matching under the passage that was matched. The LLM matching task is not used during regression tests.

To create a TrueIntent usage and configure an LLM matching task, you must have:

• Generative AI configurations enabled for your chatbot by inGenious AI support.
• A connector to power the task. 
  You can see the list of connectors that support the task when you create the usage.
• A knowledge base containing a published chatbot content data source.

To create a TrueIntent usage and configure the LLM matching task:

1. Click Manage in the left navigation, then click LLM Usages. 
   
2. Click your primary TrueIntent usage.
   If you don't have a TrueIntent usage, create one:
   1. Click + LLM Usage.
   2. In the Type field, select TrueIntent.
   3. Optionally, edit the Name for the usage.
   4. If you don't want the chatbot to use this task yet, clear the Set As Primary checkbox.
   5. Select the Connector to use for the task or select Create compatible Connector to create a new connector.
      Only compatible connectors will be available.
      You must have an administrator role in your team to create connectors.
   6. Click Create.
3. Click the LLM Matching tab.
4. Leave the Enable LLM Matching field unselected for now.
   You can configure the task and test it in automated tests without enabling it.
   You'll need to enable the task if you want to test it in TestBot. 
   The task won't run in your live chatbot until you've enabled it and published that change.
5. By default, the inGenious AI platform waits a maximum of 10 seconds for the model to generate a result. If it doesn't receive a result within that time, it starts the passage that was matched by your NLP questions. 
   If you want to change this period:
   1. Expand the Timeout section.
   2. Update the number of seconds the inGenious AI platform will wait.
6. If you want to edit the default prompt used to generate the summary, type any edits into the Prompt area.
   Make sure you don't delete the prompt variables: ${userQuestion}, ${chatHistory}, and ${content}.
7. If you want to include the transcript of the chat for context, select Populate ${chatHistory}.
   If this field is not selected, the ${chatHistory} prompt variable is blank.
8. Select the Knowledge Base that contains the chatbot content data source.
9. Select the Maximum number of Potential Answers the model should retrieve as possible passages.
   A higher number can give more accurate matches, but will take longer to respond to the user and may need a connector to a higher-tier LLM model.
10. Select whether to match passages using:
    • Passage Content.
    • Training Phrases in the passage questions.
    • Annotations you have added to passages and conversations.
11. If you want to limit the chatbot content that can be matched against passages:
    1. Select either:
       1. Include if you want to match passages against only the conversations and passages you specify.
       2. Exclude if you want to match passages against all chatbot content except the conversations and passages you specify.
    2. Select Specific Conversations and Passages.
    3. Expand the Conversations List and Passage List sections and click + Conversation or + Passage to add the conversations and passages.
       If you add a conversation, all passages from that conversation are included. You don't need to add those passages separately.
       Adding a conversation does not add the passages from any child conversations. Add child conversations separately.
       To remove all the conversations or passages you've selected, click the menu at the top of the list, then click Delete All.
12. Click Save.

Enable LLM matching

The LLM matching task is only available to your live chatbot once you've enabled the task and published the change. To enable the task: 

1. Click Manage in the left navigation, then click LLM Usages.
2. Click the primary TrueIntent usage.
   Only the task from the primary usage will be used.
3. Click the LLM Matching tab.
4. Click Enable LLM Matching.
5. Click Save.

You must publish the change for it to take effect.