Create an LLM Usage

Create a LLM Usage to enable advanced features that use Large Language Models (LLMs) to help you maintain your chatbot content, such as generating training phrase suggestions or handover transcript summaries for your live agents. The LLM Usage defines which LLM Connector (the provider and model) to use, and the prompt to send to generate the results.

The content generated by the LLM is never visible to your chatbot users and won't impact their experience.

LLM Usages are divided into scopes for the different features they support. To use these features:

1. Make sure you've created an LLM Connector for the LLM provider you want to use.
2. Create an LLM Usage for the scope you want to use:
   • Create an LLM Usage for handover transcript summaries.
3. Set that LLM Usage as the primary for that scope.

Changes to most LLM Usage scopes are applied to your chatbot immediately, but some must be published before they take effect in your live chatbot.

Each usage is preconfigured with a default prompt for the scope, making them ready to use out-of-the-box. You can view or customise this prompt from the LLM Usage page. Each LLM Usage is chatbot-wide, using the same prompt for the whole chatbot, but some features allow you to add additional prompt instructions per-use, such as to customise transcript summaries.

Create an LLM Usage for handover transcript summaries

Changes to LLM Usages for handover transcript summaries must be published before they take effect in your live chatbot. The generated summary will not be visible to your chatbot users.

Make sure you have created an LLM Connector for the provider and model you want to use.

1. Select your team, and the chatbot you want to edit.
2. Open the Manage section of the left navigation and click LLM Usages.
3. Click + LLM Usage.
4. In the Usage Scope, select Handover Transcript Summary.
5. Type a distinctive name for the usage.
6. Click Create.
7. In the new usage screen, select the LLM Connector to use.
   If you can't see the LLM Connector you want to use, check the LLM Connector model's capabilities, listed under the model name on the LLM Connector page. Handover transcript summaries require a model capable of text generation and text summarisation.
8. Click Save.
9. If you want to edit the default prompt used to generate the summary, type any edits into the Prompt area and click Save.
   Make sure you don't remove the prompt parameters: ${additionalContext} and ${transcript}.
10. In the list of LLM Usages on the left, click the menu next to the usage you just created.
11. Click Set as Primary for Handover Transcript Summary.

Set an LLM Usage as primary

Set an LLM usage as Primary to designate it as the one the chatbot should use.

1. Select your team, and the chatbot you want to edit.
2. Open the Manage section of the left navigation and click LLM Usages.
3. In the list of LLM Usages on the left, click the menu next to the usage you want to set as the primary.
4. Click Set as Primary.