Skip to content

On-premise users: click in-app to access the full platform documentation for your version of DataRobot.

Modeling > Specialized workflows > Visual AI > Visual AI overview

Visual AI overview

Working with image features in DataRobot follows the same workflow as that of non-image binary and multiclass classification (what kind of plant?) and regression (best listing price?) projects. Behind the scenes, DataRobot automates the preparation, selection, and training of a wide variety of deep learning models, recommending the model that is most accurate or the fastest for deployment. Visual AI allows you to combine supported image types, either alone with a single class label or in combination with all other supported feature types in a single dataset. Once you have uploaded images, you can preview them within DataRobot.

The following sample use cases illustrate the importance of visual learning:

  • Manufacturing: Automate the quality control process by enabling models to identify defects
  • Healthcare: Automated disease detection and diagnosis
  • Energy: Analyze images from drones to make energy assets safer or more efficient
  • Public safety: Detect intruders from security cameras
  • Insurance: Risk analysis and claims assessment

Because processing images is an intensive and data rich process, Visual AI requires using deep learning models for decision making. These models use advanced math and millions of parameters, and without automation, require users have expertise in neural networks only to generate "black box models," which businesses are hesitant to deploy.

How Visual AI works

DataRobot makes image processing possible by turning images into numbers, a process known as “featurizing." As numbers, they can be passed to subsequent modeling tasks (algorithms) so that they can be combined with other feature types (numeric, categorical, text, etc.). Visual AI uses pre-trained models to turn images into numeric vectors and feed those vectors to the final modeler (e.g., XGBoost, Elastic Net, etc.) with all other features. This technology can make changes to, and extract features from, the model levels, expanding the output of the pre-trained models and featurizers. By fine-tuning the model parameters, you can control the feature creation process to meet your requirements. See the deep dive for more detail.

Workflow overview

The sections that follow describe the Visual AI workflow:

  1. Create an image-processing ready dataset.

  2. Create projects from the AI Catalog or via local file upload.

  3. Preview images for potential data quality issues.

  4. Build models using the standard DataRobot workflow.

  5. Review the data after building.

  6. Evaluate models on the Leaderboard using:

    • The blueprint.
    • Activation maps, which highlight the parts of the image the model used to make predictions.
    • Image embeddings to see projections of images, allowing you to visually check what is similar to DataRobot and help identify outliers.
    • The Neural Network Visualizer to view each level and layer in the model's neural network.
    • Standard DataRobot insights (Confusion Matrix, Feature Impact, and Lift Chart, for example).

  7. Fine-tune model parameters for higher or lower granularity or to use a different featurizer.

  8. Select a model to use for making predictions via Make Predictions, the DataRobot API, or batch predictions.

Updated November 8, 2023