Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
comments and CL
  • Loading branch information
MilesHolland committed Oct 24, 2024
commit 23490203321a2769f948bdbb6c971bd83aca318d
1 change: 1 addition & 0 deletions sdk/evaluation/azure-ai-evaluation/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
## 1.0.0b5 (Unreleased)

### Features Added
- Added `GroundednessProEvaluator`, which is a service-based evaluator for determining response groundedness.

### Breaking Changes
- Renamed environment variable `PF_EVALS_BATCH_USE_ASYNC` to `AI_EVALS_BATCH_USE_ASYNC`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ class GroundednessProEvaluator(RaiServiceEvaluatorBase):
:param azure_ai_project: The scope of the Azure AI project.
It contains subscription id, resource group, and project name.
:type azure_ai_project: ~azure.ai.evaluation.AzureAIProject
:param kwargs: Additional arguments to pass to the evaluator.
:type kwargs: Any

**Usage**

Expand All @@ -31,7 +33,9 @@ class GroundednessProEvaluator(RaiServiceEvaluatorBase):
"resource_group_name": "<resource_group_name>",
"project_name": "<project_name>",
}
eval_fn = GroundednessProEvaluator(azure_ai_project)
credential = DefaultAzureCredential()

eval_fn = GroundednessProEvaluator(azure_ai_project, credential)
result = eval_fn(query="What's the capital of France", response="Paris", context="Paris.")

**Output format**
Expand All @@ -52,7 +56,9 @@ class GroundednessProEvaluator(RaiServiceEvaluatorBase):
"resource_group_name": "<resource_group_name>",
"project_name": "<project_name>",
}
eval_fn = GroundednessProEvaluator(azure_ai_project)
credential = DefaultAzureCredential()

eval_fn = GroundednessProEvaluator(azure_ai_project, credential)
conversation = {
"messages": [
{"role": "user", "content": "What is the capital of France?"},
Expand Down Expand Up @@ -89,6 +95,7 @@ def __init__(
eval_metric=EvaluationMetrics.GROUNDEDNESS,
azure_ai_project=azure_ai_project,
credential=credential,
**kwargs,
)

@override
Expand Down