Data exploration¶

On a deployment's Monitoring > Data exploration tab, you can interact with a deployment's stored data to gain insight into model performance and use that data to calculate custom metrics. The Data Exploration Summary includes the following functionality, depending on the deployment type:

Configure data exploration range¶

In the deployment from which you want to export stored training data, prediction data, or actuals, click the Monitoring > Data exploration tab and configure the following settings to specify the stored training data, prediction data, or actuals you want to export:

Export deployment data¶

On the Data Exploration Summary (or, for Generative AI deployments, the Data export tab of the Data Exploration Summary), you can download a deployment's stored data. This can include training data, prediction data, actuals, and custom metric data. You can use the exported data to compute and monitor custom business or performance metrics on the Custom metrics tab or outside DataRobot. To export deployment data for custom metrics, verify that the deployment stores prediction data, generate data for a specified time range, and then view or download that data.

Export a deployment's production data¶

To access deployment data export for prediction data, actuals, or custom metric data, on the Data Exploration Summary page, locate the Production data panel. On the Production data panel, in the Generate button, click the down arrow and select one of the data generation options. The availability of the following options depends on the data stored in the deployment for the model and time range selected:

Production data appears in the table below the panels. You can identify the data type in the Exported data column.

Export a deployment's training data¶

To access deployment data export for training data, on the Data Exploration Summary page, locate the Training data panel and click Generate training data to generate data for the specified model and time range:

The training data appears in the Training Data panel. Click the down arrow to choose between Open training data and Download training data :

Review and download data¶

After the production or training data are generated, you can view or download the data. Production data appears in the table below the panels, where you can identify the data type in the Exported data column. Training data appears in the Training data panel.

Use exported deployment data for custom metrics¶

To use the exported deployment data to create your own custom metrics, you can implement a script to read from the CSV file containing the exported data and then calculate metrics using the resulting values, including columns automatically generated during the export process.

This example uses the exported prediction data to calculate and plot the change in the time_in_hospital feature over a 30-day period using the DataRobot prediction timestamp (DR_RESERVED_PREDICTION_TIMESTAMP) as the DateFrame index (or row labels). It also uses the exported training data as the plot's baseline:

import pandas as pd
feature_name = "<numeric_feature_name>"
training_df = pd.read_csv("<path_to_training_data_csv>")
baseline = training_df[feature_name].mean()
prediction_df = pd.read_csv("<path_to_prediction_data_csv>")
prediction_df["DR_RESERVED_PREDICTION_TIMESTAMP"] = pd.to_datetime(
    prediction_df["DR_RESERVED_PREDICTION_TIMESTAMP"]
)
predictions = prediction_df.set_index("DR_RESERVED_PREDICTION_TIMESTAMP")["time_in_hospital"]
ax = predictions.rolling('30D').mean().plot()
ax.axhline(y=baseline - 2, color="C1", label="training data baseline")
ax.legend()
ax.figure.savefig("feature_over_time.png")

DataRobot column reference¶

DataRobot automatically adds the following columns to the prediction data generated for export:

Explore deployment data quality¶

On the Data exploration tab of a Generative AI deployment, click Data quality to explore prompts, responses, user ratings, and custom metrics matched by association ID. This view provides insight into the quality of the Generative AI model's responses, as rated by users and based on any Generative AI custom metrics you implement:

Locate specific rows by filtering the table to only display rows containing Prompt values, Response values, or Actual values. Click Search to locate a specific row in the Data quality table by prompt:

Click the open icon to expand the details panel, displaying a row's full Prompt and the Response matched with the prompt by association ID: