Monitor AI configs

Overview

This topic explains how to monitor the performance of your AI configs. Performance metrics for AI configs are available in the LaunchDarkly user interface if you track AI metrics in your SDK.

Monitor an AI config

To monitor the performance of an AI config:

1. Navigate to the detail page for the AI config. Select the Monitoring tab.
2. At the top of the tab, select the environment you want to monitor. Performance metrics are specific to each environment.

• To monitor an environment that's not shown, click the + icon to add another environment to the Monitoring tab.

3. Review the available metrics:

• Output satisfaction rate is the percentage of "thumbs up" ratings provided by end users who have encountered the AI config in this environment.
• Generation count is the number of successful generations completed using the AI config in this environment.
• Input tokens used is the total input tokens used by the AI config in this environment.
• Output tokens used is the total output tokens used by the AI config in this environment.
• Average tokens per generation is the total input tokens used divided by the number of successful generations completed by the AI config in this environment.
• Average duration per generation (ms) is the total duration of calls to your LLM provider divided by the number of total generations using the AI config in this environment.
• Error rate is the percentage of errors out of the total number of generations attempted.
• Average time to first token (ms) is the mean time it takes to generate the initial token.

The data on the Monitoring tab updates approximately each minute. For each metric, the results are broken out by AI config variation. Metrics with no results display "No data." If there is no data for a particular variation, that variation is not included in the total displayed at the top of the metric card.

Here is a partial image of the Monitoring tab: