Tracking AI metrics
Read time: 10 minutes
Last edited: Feb 07, 2025
The AI configs product is only available in early access for customers on select plans. To request early access, navigate to AI configs and join the waitlist.
The AI SDKs are designed for use with the AI configs product. The AI SDKs are currently in an alpha version.
Overview
This topic explains how to record metrics from your AI model generation, including duration, generation, satisfaction, and several token-related metrics. This feature is available for AI SDKs only.
About AI metrics
To help you track how your AI model generation is performing, the AI SDKs provide options to record metrics from your model generation. LaunchDarkly displays these metrics on the AI config Monitoring tab in the user interface.
All SDKs include individual track* methods to record the following metrics:
- duration
- token usage
- generation success
- generation error
- time to first token
- output satisfaction
Additionally, some AI SDKs include provider-specific track_[model]_metric methods. These methods take the result of the provider-specific call as a parameter, and record all of the following metrics:
- duration
- token usage
- generation success
- generation error
The provider-specific methods are a useful shorthand if you're working with those providers. You can always call the track* methods manually to record additional metrics.
Both the individual track* methods and the provider-specific track_[model]_metric methods are called from the tracker. The tracker is associated with a specific customization call.
AI SDKs
This feature is available for all of the AI SDKs:
.NET AI
Expand .NET AI SDK code sample
Use the TrackRequest function to record metrics from your AI model generation.
The tracker is returned from your call to customize the AI config, and is specific to that AI config. Make sure to call Config again each time you use the tracker and generate content from your AI model, so that your metrics are correctly associated with the customized AI config variation.
Here's how:
If you would like to do any additional tracking, besides what LaunchDarkly provides, it is your responsibility to fill in the Response object with the data you want to track.
You can also use the SDK's other Track* functions to record these metrics manually. The TrackRequest function is expecting a response, so you may need to do this if your application requires streaming.
Each of the Track* functions sends data back to LaunchDarkly. The Monitoring tab of the AI config in the LaunchDarkly UI aggregates data from the Track* functions from across all variations of the AI config.
Here's how to record metrics manually:
Make sure to call Config again each time you use the tracker and generate content from your AI model.
To learn more, read LDAIConfigTracker.
Go AI
Expand Go AI code sample
Use the TrackRequest function to record metrics from your AI model generation.
The tracker is returned from your call to customize the AI config, and is specific to that AI config. Make sure to call Config again each time you use the tracker and generate content from your AI model, so that your metrics are correctly associated with the customized AI config variation.
Here's how:
Alternatively, you can use the SDK's other Track* functions to record these metrics manually. The TrackRequest function is expecting a response, so you may need to do this if your application requires streaming.
Each of the Track* functions sends data back to LaunchDarkly. The Monitoring tab of the AI config in the LaunchDarkly UI aggregates data from the Track* functions from across all variations of the AI config.
Here's how to record metrics manually:
To learn more, read Tracker.
Node.js (server-side) AI
Expand Node.js (server-side) AI SDK code sample
Use one of the track[Model]Metrics functions to record metrics from your AI model generation. The SDK provides separate track[Model]Metrics functions for several of the models that you can select when you set up your AI config variations in the LaunchDarkly user interface.
The tracker is from your call to customize the AI config, and is specific to that AI config. Make sure to call config again each time you use the tracker and generate content from your AI model, so that your metrics are correctly associated with the customized AI config variation.
Here's how:
You can also use the SDK's other track* functions to record these metrics manually. You may need to do this if you are using a model for which the SDK does not provide a convenience track[Model]Metrics function. The track[Model]Metrics functions are expecting a response, so you may also need to do this if your application requires streaming.
Each of the track* functions sends data back to LaunchDarkly. The Monitoring tab of the AI config in the LaunchDarkly UI aggregates data from the track* functions from across all variations of the AI config.
Here's how to record metrics manually:
Make sure to call config again each time you use the tracker and generate content from your AI model.
To learn more, read LDAIConfigTracker.
Python AI
Expand Python AI SDK code sample
Use one of the track_[model]_metrics functions to record metrics from your AI model generation. The SDK provides separate track_[model]_metrics functions for several of the models that you can select when you set up your AI config variations in the LaunchDarkly user interface.
The tracker is returned from your call to customize the AI config, and is specific to that AI config. Make sure to call config again each time you use the tracker and generate content from your AI model, so that your metrics are correctly associated with the customized AI config variation.
Here's how:
You can also use the SDK's other track* functions to record these metrics manually. You may need to do this if you are using a model for which the SDK does not provide a convenience track_[model]_metrics function. The track_[model]_metrics functions are expecting a response, so you may also need to do this if your application requires streaming.
Each of the track* functions sends data back to LaunchDarkly. The Monitoring tab of the AI config in the LaunchDarkly UI aggregates data from the track* functions from across all variations of the AI config.
Here's how to record metrics manually:
Make sure to call config again each time you use the tracker and generate content from your AI model.
To learn more, read LDAIConfigTracker.