Create an automated test

Create an automated test to test and refine your prompt for an LLM task.

You need an administrator or publisher role in your team to create automated tests.

Each automated test is designed for a single task, such as rewriting utterances. You'll need a CSV of test cases to use as input for the task to generate, such as a list of example utterances to rewrite. 

Each task needs different input, so there's no predefined format for the CSV. When you upload your CSV, you'll select which columns in the CSV correspond to the inputs required by the task and steps. Make sure the first row in the CSV is your column headings. The CSV can have a maximum of 1000 rows and a maximum size of 50Mb.

Once you've run some tests, you can use either your uploaded CSV or one of the result sets as inputs for your test. You can replace also the CSV data at any time by uploading a new CSV.  You can also download a result set as a CSV to upload it into another automated test.

You can create an automated test for:

• Rewriting utterances
• Relevancy checks
• LLM matching

Once you've created your test, follow the steps in sequence:

1. Step 1: Run the task to generate output to test.

2. Step 2: Evaluate the output.

3. Step 3: Generate insights and prompt suggestions.