Model Iterations¶
Availability information
Because Incremental learning for large datasets is off by default, the resulting Model Iterations insight is also off. Contact your DataRobot representative or administrator for information on enabling incremental learning.
Feature flags:
- Enable Incremental Learning
- Enable Data Chunking
The Model Iterations insight allows you to compare trained iterations and, optionally, assign a different active iteration or continue training. The active iteration—the iteration of the model that has the highest accuracy—is the basis for other Leaderboard insights and is used for making predictions. As a result, the Leaderboard for an IL experiment does not show all models; instead, it shows a representative of each model. As training continues, the best model is assigned as the active model at the end of each iterative round.
Identifying information is listed on the Leaderboard model card:
And the Model Overview, which also indicates which iteration is the active iteration for that model:
Available actions¶
Click View experiment info to view details of the built experiment, including total rows and iterations:
For each model built, you can potentially:
Options may be unavailable or not present if they are not applicable. For example, if all iterations were run, the option to train any remaining iterations is unavailable. However, if only one iteration ran, the option to change the active iteration is present but unavailable until you train the remaining iterations.
Change active iteration¶
For models with more than one iteration run, the most accurate iteration is marked with an Active badge. For a selected model, click Change active iteration to select a different iteration:
Selecting a new active iteration updates:
- The score on the model card, and therefore the model ranking.
- The scores in the Training scores section at the top of Model Overview.
Resume training¶
If, when configuring the experiment, you selected Stop training when model accuracy no longer improves, any model that was not identified by DataRobot as the most accurate runs only a single iteration. You can, however, train new model iterations on any untrained increments. Click Train remaining iterations to resume training for the selected model. With this selection, DataRobot trains all remaining iterations until early stopping is applied, not just the next increment. Click Train remaining iterations again to train all remaining iterations, overriding the early stopping setting.
Once selected, DataRobot begins training the next iteration, making an entry in the table. Click Stop training at any time to stop the process and save only the completed iterations.
Understand the table¶
Each model displays a table containing an iteration log with the following fields:
| Table element | Description |
|---|---|
| Model iteration | Number of the model iteration. |
| Status badges | Badges indicating the state of the iteration, either Active, Training in progress, Errored, or Queued for iterations for which training has been requested. |
| Cumulative rows | Total number of rows processed for the iteration. For example, if increment size is set to 1000 rows, increment 3 would represent 1000 + 1000 + 1000 rows, so cumulative rows would be 3000. |
| Accuracy columns | Accuracy metric values for the validation and holdout partitions. The applied metric is indicated in the column header. |