End-to-end code-first generative AI experimentation¶
This notebook outlines a comprehensive overview of the generative AI features DataRobot has to offer. It uses methods that are part of the experimental version of the DataRobot Python client, included in the client's tarball. Contact your DataRobot representative or administrator for information on accessing the experimental DataRobot Python library, or use pip install datarobot-early-access.
The following cell provides example usage.
See the following code block for example usage:
import datarobot as dr
from datarobot._experimental.models.genai.vector_database import VectorDatabase
from datarobot._experimental.models.genai.vector_database import ChunkingParameters
from datarobot._experimental.models.enums import VectorDatabaseEmbeddingModel
from datarobot._experimental.models.enums import VectorDatabaseChunkingMethod
from datarobot._experimental.models.genai.playground import Playground
from datarobot._experimental.models.genai.llm import LLMDefinition
from datarobot._experimental.models.genai.llm_blueprint import LLMBlueprint
from datarobot._experimental.models.genai.llm_blueprint import VectorDatabaseSettings
from datarobot._experimental.models.genai.chat_prompt import ChatPrompt
from datarobot._experimental.models.genai.vector_database import CustomModelVectorDatabaseValidation
from datarobot._experimental.models.genai.comparison_prompt import ComparisonPrompt
from datarobot._experimental.models.genai.custom_model_llm_validation import CustomModelLLMValidation
Create a Use Case¶
Generative AI works within a Datarobot Use Case, so to begin, create a new use case.
use_case = dr.UseCase.create()
This workflow uses a vector database with GenAI models. Next, you will upload a dataset with a set of documents.
dataset = dr.Dataset.create_from_file("resumes.zip")
Create a vector database¶
chunking_parameters = ChunkingParameters(
embedding_model=VectorDatabaseEmbeddingModel.JINA_EMBEDDING_T_EN_V1,
chunking_method=VectorDatabaseChunkingMethod.RECURSIVE,
chunk_size=128,
chunk_overlap_percentage=25,
separators=[],
)
vdb = VectorDatabase.create(dataset.id, chunking_parameters, use_case)
Use the following cell to retrieve the vector database and make sure it has completed.
vdb = VectorDatabase.get(vdb.id)
assert vdb.execution_status == "COMPLETED"
Create an LLM playground¶
Create an LLM playground that will host large language models (LLMs).
playground = Playground.create(name="New playground for example", use_case=use_case)
Use the following cell to see what kind of LLMs are available. By default this method returns as a dict for easy readability. It includes a list of allowed settings for each LLM along with any constraints on the values.
llms_dict = LLMDefinition.list()
print(llms_dict)
As an example, use GPT 3.5.
llms = LLMDefinition.list(as_dict=False)
gpt = llms[0]
To interact with GPT, you need to create an LLM blueprint with the settings that you want. Since the allowed settings depend on the LLM type, take a generic dict with those values that will be validated during the creation of the blueprint. Review the allowed settings for GPT.
print([setting.to_dict() for setting in gpt.settings])
Create an LLM blueprint¶
llm_settings = {
"system_prompt": (
"You are a pirate who begins each response with 'Arrr' "
"and ends each response with 'Matey!'"
),
"max_completion_length": 256, # Let's ask for short responses
"temperature": 2.0, # With a high degree of variability
}
Use some vector database settings that could be different from the defaults when the database was created.
vector_database_settings = VectorDatabaseSettings(
max_documents_retrieved_per_prompt=2,
batch_size=32,
max_tokens=128,
)
llm_blueprint = LLMBlueprint.create(
playground=playground,
name="GPT",
llm=gpt,
llm_settings=llm_settings,
vector_database=vdb,
vector_database_settings=vector_database_settings,
)
Chat with the LLM blueprint¶
Use the following cell to chat with the LLM blueprint.
prompt1 = ChatPrompt.create(
llm_blueprint,
"How can a pirate resume be evaluated?",
wait_for_completion=True,
)
print(prompt1.result_text)
The example cell above returns a truncated output, confirmed by result_metadata.
print(prompt1.result_metadata.error_message)
Try to chat again with a larger max_completion_length. LLM settings can be updated within a chat prompt as long as the LLM blueprint hasn't yet been saved. Doing so updates the underlying settings of the LLM blueprint.
llm_settings["max_completion_length"] = 1024
prompt2 = ChatPrompt.create(
llm_blueprint,
"Please answer the previous question again.",
llm_settings=llm_settings,
wait_for_completion=True,
)
print(prompt2.result_text)
You can keep the new max_completion_length and confirm that the LLM blueprint has been updated.
llm_blueprint = LLMBlueprint.get(llm_blueprint.id)
print(llm_blueprint.llm_settings)
Next, lower the temperature to modify the reply, as the response may be unsatisfactory.
llm_settings["temperature"] = 0.8
prompt3 = ChatPrompt.create(
llm_blueprint,
"Please summarize the best pirate resume.",
llm_settings=llm_settings,
wait_for_completion=True,
)
print(prompt3.result_text)
Confirm that the LLM is using information from the vector database.
print([citation.text for citation in prompt3.citations])
Save the LLM blueprint so that you lock in the settings and can use it with a comparison prompt.
llm_blueprint = llm_blueprint.update(is_saved=True)
Next, create a new LLM blueprint from the existing one so that you can change some settings and compare the two.
new_llm_blueprint = LLMBlueprint.create_from_llm_blueprint(llm_blueprint, name="new blueprint")
Remove the vector database from the new blueprint, modify the settings as shown below, and save the blueprint.
llm_settings["system_prompt"] = "You are an AI assistant who helps to evaluate resumes."
new_llm_blueprint = new_llm_blueprint.update(
llm_settings=llm_settings,
remove_vector_database=True,
is_saved=True,
)
You can deploy an LLM blueprint from the DataRobot UI.
Add a custom model vector database¶
Add a custom model vector database from a deployment. This example uses a previously deployed text generation model. Validate it for use as a vector database.
custom_model_vdb_validation = CustomModelVectorDatabaseValidation.create(
prompt_column_name="prompt",
target_column_name="response",
deployment_id="652da8617f7054ab1a0719a8",
wait_for_completion=True,
)
assert custom_model_vdb_validation.validation_status == "PASSED"
# Now create a vector database
custom_model_vdb = VectorDatabase.create_from_custom_model(
name="custom model vdb",
use_case=use_case,
validation_id=custom_model_vdb_validation.id,
)
custom_model_vdb = VectorDatabase.get(custom_model_vdb.id)
assert custom_model_vdb.execution_status == "COMPLETED"
Add a custom model LLM¶
Next, add a custom model LLM from a deployment (creating a custom model LLM is out of scope for this workflow). Validate the text generation model for use as an LLM.
custom_model_llm_validation = CustomModelLLMValidation.create(
prompt_column_name="prompt",
target_column_name="response",
deployment_id="652da8617f7054ab1a0719a8",
wait_for_completion=True,
)
assert custom_model_llm_validation.validation_status == "PASSED"
Now you can create an LLM blueprint using the custom model LLM and the custom model vector database.
custom_model_llm_blueprint = LLMBlueprint.create(
playground=playground,
name="custom model LLM with custom model vdb",
llm="custom-model",
llm_settings={"validation_id": custom_model_llm_validation.id},
vector_database=custom_model_vdb,
)
Save the custom model blueprint so you can use it in a comparison.
custom_model_llm_blueprint = custom_model_llm_blueprint.update(is_saved=True)
You can now compare the LLM blueprints.
comparison_prompt = ComparisonPrompt.create(
[llm_blueprint, new_llm_blueprint, custom_model_llm_blueprint],
"Summarize the best resume",
wait_for_completion=True,
)
print([result.result_text for result in comparison_prompt.results])