DataRobot APIのリソース > APIリファレンスドキュメント > DataRobot REST API > Comparison Prompts (GenAI)

Comparison Prompts (GenAI)¶

This page outlines the operations, endpoints, parameters, and example requests and responses for the Comparison Prompts (GenAI).

GET /api/v2/genai/comparisonPrompts/¶

List the comparison prompts associated with a comparison chat or a set of LLM blueprints.

Code samples¶

# You can also use wget
curl -X GET https://app.datarobot.com/api/v2/genai/comparisonPrompts/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
llmBlueprintIds	query	any	false	Only retrieve the comparison prompts associated with the specified LLM blueprint IDs. Either this parameter or `comparisonChatId` must be specified, but not both.
comparisonChatId	query	string	false	Only retrieve the comparison prompts associated with the specified comparison chat ID. Either this parameter or `llmBlueprintIds` must be specified, but not both.
offset	query	integer	false	Skip the specified number of values.
limit	query	integer	false	Retrieve only the specified number of values.

Example responses¶

200 Response

{
  "count": 0,
  "data": [
    {
      "comparisonChatId": "string",
      "creationDate": "2019-08-24T14:15:22Z",
      "creationUserId": "string",
      "id": "string",
      "results": [
        {
          "chatContextId": "string",
          "citations": [
            {
              "page": 0,
              "source": "string",
              "text": "string"
            }
          ],
          "comparisonPromptResultIdsIncludedInHistory": [
            "string"
          ],
          "confidenceScores": {
            "bleu": 0,
            "meteor": 0,
            "rouge": 0
          },
          "executionStatus": "NEW",
          "id": "string",
          "llmBlueprintId": "string",
          "resultMetadata": {
            "cost": 0,
            "errorMessage": "string",
            "estimatedDocsTokenCount": 0,
            "feedbackResult": {
              "negativeUserIds": [],
              "positiveUserIds": []
            },
            "finalPrompt": "string",
            "inputTokenCount": 0,
            "latencyMilliseconds": 0,
            "metrics": [],
            "outputTokenCount": 0,
            "totalTokenCount": 0
          },
          "resultText": "string"
        }
      ],
      "text": "string",
      "userName": "string"
    }
  ],
  "next": "string",
  "previous": "string",
  "totalCount": 0
}

Responses¶

Status	Meaning	Description	Schema
200	OK	Successful Response	ListComparisonPromptsResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

POST /api/v2/genai/comparisonPrompts/¶

Request the execution of a new comparison prompt.

Code samples¶

# You can also use wget
curl -X POST https://app.datarobot.com/api/v2/genai/comparisonPrompts/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Body parameter¶

{
  "comparisonChatId": "string",
  "llmBlueprintIds": [
    "string"
  ],
  "text": "string"
}

Parameters

Name	In	Type	Required	Description
body	body	CreateComparisonPromptRequest	true	none

Example responses¶

202 Response

{
  "comparisonChatId": "string",
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "id": "string",
  "results": [
    {
      "chatContextId": "string",
      "citations": [
        {
          "page": 0,
          "source": "string",
          "text": "string"
        }
      ],
      "comparisonPromptResultIdsIncludedInHistory": [
        "string"
      ],
      "confidenceScores": {
        "bleu": 0,
        "meteor": 0,
        "rouge": 0
      },
      "executionStatus": "NEW",
      "id": "string",
      "llmBlueprintId": "string",
      "resultMetadata": {
        "cost": 0,
        "errorMessage": "string",
        "estimatedDocsTokenCount": 0,
        "feedbackResult": {
          "negativeUserIds": [],
          "positiveUserIds": []
        },
        "finalPrompt": "string",
        "inputTokenCount": 0,
        "latencyMilliseconds": 0,
        "metrics": [],
        "outputTokenCount": 0,
        "totalTokenCount": 0
      },
      "resultText": "string"
    }
  ],
  "text": "string",
  "userName": "string"
}

Responses¶

Status	Meaning	Description	Schema
202	Accepted	Successful Response	ComparisonPromptResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

DELETE /api/v2/genai/comparisonPrompts/{comparisonPromptId}/¶

Delete an existing comparison prompt.

Code samples¶

# You can also use wget
curl -X DELETE https://app.datarobot.com/api/v2/genai/comparisonPrompts/{comparisonPromptId}/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
comparisonPromptId	path	string	true	The ID of the comparison prompt to delete.

Example responses¶

422 Response

{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

Responses¶

Status	Meaning	Description	Schema
204	No Content	Successful Response	None
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

GET /api/v2/genai/comparisonPrompts/{comparisonPromptId}/¶

Retrieve an existing comparison prompt.

Code samples¶

# You can also use wget
curl -X GET https://app.datarobot.com/api/v2/genai/comparisonPrompts/{comparisonPromptId}/ \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Parameters

Name	In	Type	Required	Description
comparisonPromptId	path	string	true	The ID of the comparison prompt to retrieve.

Example responses¶

200 Response

{
  "comparisonChatId": "string",
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "id": "string",
  "results": [
    {
      "chatContextId": "string",
      "citations": [
        {
          "page": 0,
          "source": "string",
          "text": "string"
        }
      ],
      "comparisonPromptResultIdsIncludedInHistory": [
        "string"
      ],
      "confidenceScores": {
        "bleu": 0,
        "meteor": 0,
        "rouge": 0
      },
      "executionStatus": "NEW",
      "id": "string",
      "llmBlueprintId": "string",
      "resultMetadata": {
        "cost": 0,
        "errorMessage": "string",
        "estimatedDocsTokenCount": 0,
        "feedbackResult": {
          "negativeUserIds": [],
          "positiveUserIds": []
        },
        "finalPrompt": "string",
        "inputTokenCount": 0,
        "latencyMilliseconds": 0,
        "metrics": [],
        "outputTokenCount": 0,
        "totalTokenCount": 0
      },
      "resultText": "string"
    }
  ],
  "text": "string",
  "userName": "string"
}

Responses¶

Status	Meaning	Description	Schema
200	OK	Successful Response	ComparisonPromptResponse
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

PATCH /api/v2/genai/comparisonPrompts/{comparisonPromptId}/¶

Edit an existing comparison prompt. Editing may involve adding new prompt result metadata or executing this comparison prompt on new LLM blueprints.

Code samples¶

# You can also use wget
curl -X PATCH https://app.datarobot.com/api/v2/genai/comparisonPrompts/{comparisonPromptId}/ \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer {access-token}"

Body parameter¶

{
  "additionalLLMBlueprintIds": []
}

Parameters

Name	In	Type	Required	Description
comparisonPromptId	path	string	true	The ID of the comparison prompt to edit.
body	body	EditComparisonPromptRequest	true	none

Example responses¶

202 Response

"string"

Responses¶

Status	Meaning	Description	Schema
202	Accepted	Successful Response	string
422	Unprocessable Entity	Validation Error	HTTPValidationErrorResponse

To perform this operation, you must be authenticated by means of one of the following methods:

BearerAuth

Schemas¶

Citation

{
  "page": 0,
  "source": "string",
  "text": "string"
}

Citation

Properties¶

Name	Type	Required	Description
page	integer¦null	false	The source page number where the citation was found.
source	string¦null	true	The source of the citation (e.g., a filename in the original dataset).
text	string	true	The text of the citation.

ComparisonPromptResponse

{
  "comparisonChatId": "string",
  "creationDate": "2019-08-24T14:15:22Z",
  "creationUserId": "string",
  "id": "string",
  "results": [
    {
      "chatContextId": "string",
      "citations": [
        {
          "page": 0,
          "source": "string",
          "text": "string"
        }
      ],
      "comparisonPromptResultIdsIncludedInHistory": [
        "string"
      ],
      "confidenceScores": {
        "bleu": 0,
        "meteor": 0,
        "rouge": 0
      },
      "executionStatus": "NEW",
      "id": "string",
      "llmBlueprintId": "string",
      "resultMetadata": {
        "cost": 0,
        "errorMessage": "string",
        "estimatedDocsTokenCount": 0,
        "feedbackResult": {
          "negativeUserIds": [],
          "positiveUserIds": []
        },
        "finalPrompt": "string",
        "inputTokenCount": 0,
        "latencyMilliseconds": 0,
        "metrics": [],
        "outputTokenCount": 0,
        "totalTokenCount": 0
      },
      "resultText": "string"
    }
  ],
  "text": "string",
  "userName": "string"
}

ComparisonPromptResponse

Properties¶

Name	Type	Required	Description
comparisonChatId	string¦null	false	The ID of the comparison chat associated with the comparison prompt.
creationDate	string(date-time)	true	The creation date of the comparison prompt (ISO 8601 formatted).
creationUserId	string	true	The ID of the user that created the comparison prompt.
id	string	true	The ID of the comparison prompt.
results	[ComparisonPromptResultApiFormatted]	true	The list of comparison prompt results.
text	string	true	The text of the user prompt.
userName	string	true	The name of the user that created the comparison prompt.

ComparisonPromptResultApiFormatted

{
  "chatContextId": "string",
  "citations": [
    {
      "page": 0,
      "source": "string",
      "text": "string"
    }
  ],
  "comparisonPromptResultIdsIncludedInHistory": [
    "string"
  ],
  "confidenceScores": {
    "bleu": 0,
    "meteor": 0,
    "rouge": 0
  },
  "executionStatus": "NEW",
  "id": "string",
  "llmBlueprintId": "string",
  "resultMetadata": {
    "cost": 0,
    "errorMessage": "string",
    "estimatedDocsTokenCount": 0,
    "feedbackResult": {
      "negativeUserIds": [],
      "positiveUserIds": []
    },
    "finalPrompt": "string",
    "inputTokenCount": 0,
    "latencyMilliseconds": 0,
    "metrics": [],
    "outputTokenCount": 0,
    "totalTokenCount": 0
  },
  "resultText": "string"
}

ComparisonPromptResultApiFormatted

Properties¶

Name	Type	Required	Description
chatContextId	string¦null	false	The ID of the chat context for this prompt.
citations	[Citation]	true	The list of relevant vector database citations (in case of using a vector database).
comparisonPromptResultIdsIncludedInHistory	[string]¦null	false	The list of IDs of the comparison prompt results included in this prompt's history.
confidenceScores	ConfidenceScores¦null	true	The confidence scores that measure the similarity between the prompt context and the prompt completion.
executionStatus	ExecutionStatus	true	The execution status of the comparison prompt by this LLM blueprint.
id	string	true	The ID of the comparison prompt result.
llmBlueprintId	string	true	The ID of the LLM blueprint that produced the result.
resultMetadata	ResultMetadata¦null	false	The additional information about the prompt result.
resultText	string¦null	true	The text of the prompt completion.

ConfidenceScores

{
  "bleu": 0,
  "meteor": 0,
  "rouge": 0
}

ConfidenceScores

Properties¶

Name	Type	Required	Description
bleu	number	true	BLEU score.
meteor	number	true	METEOR score.
rouge	number	true	ROUGE score.

CreateComparisonPromptRequest

{
  "comparisonChatId": "string",
  "llmBlueprintIds": [
    "string"
  ],
  "text": "string"
}

CreateComparisonPromptRequest

Properties¶

Name	Type	Required	Restrictions	Description
comparisonChatId	string¦null	false		The ID of the comparison chat to associate the comparison prompt with.
llmBlueprintIds	[string]	true	maxItems: 10	The list of LLM blueprint IDs that should execute the comparison prompt.
text	string	true	maxLength: 500000	The text of the user prompt.

EditComparisonPromptRequest

{
  "additionalLLMBlueprintIds": []
}

EditComparisonPromptRequest

Properties¶

Name	Type	Required	Restrictions	Description
additionalLLMBlueprintIds	[string]	false	maxItems: 10	The list of additional LLM blueprint IDs that should execute this comparison prompt.

ExecutionStatus

"NEW"

ExecutionStatus

Properties¶

Name	Type	Required	Restrictions	Description
ExecutionStatus	string	false		Job execution status.

Enumerated Values¶

Property	Value
ExecutionStatus	[`NEW`, `RUNNING`, `COMPLETED`, `ERROR`]

FeedbackResult

{
  "negativeUserIds": [],
  "positiveUserIds": []
}

FeedbackResult

Properties¶

Name	Type	Required	Restrictions	Description
negativeUserIds	[string]	false		The list of user IDs whose feedback is negative.
positiveUserIds	[string]	false		The list of user IDs whose feedback is positive.

HTTPValidationErrorResponse

{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

HTTPValidationErrorResponse

Properties¶

Name	Type	Required	Restrictions	Description
detail	[ValidationError]	false		none

ListComparisonPromptsResponse

{
  "count": 0,
  "data": [
    {
      "comparisonChatId": "string",
      "creationDate": "2019-08-24T14:15:22Z",
      "creationUserId": "string",
      "id": "string",
      "results": [
        {
          "chatContextId": "string",
          "citations": [
            {
              "page": 0,
              "source": "string",
              "text": "string"
            }
          ],
          "comparisonPromptResultIdsIncludedInHistory": [
            "string"
          ],
          "confidenceScores": {
            "bleu": 0,
            "meteor": 0,
            "rouge": 0
          },
          "executionStatus": "NEW",
          "id": "string",
          "llmBlueprintId": "string",
          "resultMetadata": {
            "cost": 0,
            "errorMessage": "string",
            "estimatedDocsTokenCount": 0,
            "feedbackResult": {
              "negativeUserIds": [],
              "positiveUserIds": []
            },
            "finalPrompt": "string",
            "inputTokenCount": 0,
            "latencyMilliseconds": 0,
            "metrics": [],
            "outputTokenCount": 0,
            "totalTokenCount": 0
          },
          "resultText": "string"
        }
      ],
      "text": "string",
      "userName": "string"
    }
  ],
  "next": "string",
  "previous": "string",
  "totalCount": 0
}

ListComparisonPromptsResponse

Properties¶

Name	Type	Required	Description
count	integer	true	The number of records on this page.
data	[ComparisonPromptResponse]	true	The list of records.
next	string¦null	true	The URL to the next page, or `null` if there is no such page.
previous	string¦null	true	The URL to the previous page, or `null` if there is no such page.
totalCount	integer	true	The total number of records.

MetricMetadata

{
  "costConfigurationId": "string",
  "customModelId": "string",
  "evaluationDatasetConfigurationId": "string",
  "formattedName": "string",
  "formattedValue": "string",
  "name": "string",
  "sidecarModelMetricValidationId": "string",
  "value": null
}

MetricMetadata

Properties¶

Name	Type	Required	Description
costConfigurationId	string¦null	false	The ID of the cost configuration.
customModelId	string¦null	false	The ID of the custom model used for the metric.
evaluationDatasetConfigurationId	string¦null	false	The ID of the evaluation dataset configuration.
formattedName	string¦null	false	The formatted name of the metric.
formattedValue	string¦null	false	The formatted value of the metric.
name	string	true	The name of the metric.
sidecarModelMetricValidationId	string¦null	false	The validation ID of the sidecar model validation(in case of using a sidecar model deployment for the metric).
value	any	true	The value of the metric.

ResultMetadata

{
  "cost": 0,
  "errorMessage": "string",
  "estimatedDocsTokenCount": 0,
  "feedbackResult": {
    "negativeUserIds": [],
    "positiveUserIds": []
  },
  "finalPrompt": "string",
  "inputTokenCount": 0,
  "latencyMilliseconds": 0,
  "metrics": [],
  "outputTokenCount": 0,
  "totalTokenCount": 0
}

ResultMetadata

Properties¶

Name	Type	Required	Description
cost	number¦null	false	The estimated cost of executing the prompt.
errorMessage	string¦null	false	The error message for the prompt (in case of an errored prompt).
estimatedDocsTokenCount	integer	false	The estimated number of tokens in the documents retrieved from the vector database.
feedbackResult	FeedbackResult	false	The user feedback associated with the prompt.
finalPrompt	any	false	The final representation of the prompt that was submitted to the LLM.

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	[object]	false		none
»» additionalProperties	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	object	false		none
»» additionalProperties	any	false		none

anyOf

Name	Type	Required	Restrictions	Description
»»» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
»»» anonymous	[object]	false		none
»»»» additionalProperties	string	false		none

continued

Name	Type	Required	Description
inputTokenCount	integer	false	The number of tokens in the LLM input. This number includes the tokens in the system prompt, the user prompt, the chat history (for history-aware chats) and the documents retrieved from the vector database (in case of using a vector database).
latencyMilliseconds	integer	true	The latency of the LLM response (in milliseconds).
metrics	[MetricMetadata]	false	The evaluation metrics for the prompt.
outputTokenCount	integer	false	The number of tokens in the LLM output.
totalTokenCount	integer	false	The combined number of tokens in the LLM input and output.

ValidationError

{
  "loc": [
    "string"
  ],
  "msg": "string",
  "type": "string"
}

ValidationError

Properties¶

Name	Type	Required	Restrictions	Description
loc	[anyOf]	true		none

anyOf

Name	Type	Required	Restrictions	Description
» anonymous	string	false		none

or

Name	Type	Required	Restrictions	Description
» anonymous	integer	false		none

continued

Name	Type	Required	Restrictions	Description
msg	string	true		none
type	string	true		none

更新しました May 14, 2024

このページは役に立ちましたか？

ありがとうございます。どのような点が役に立ちましたか？

より良いコンテンツを提供するには、どうすればよいでしょうか？

アンケートにご協力いただき、ありがとうございました。