Context Verification

Context Verification ensures that LLM responses are accurate and faithful to the provided context, helping to prevent misinformation and hallucinations.

​

Available Verification Methods

The Context Verification API provides three methods to assess the quality and accuracy of responses:

1. Context Recall (context_recall): Evaluates how well the response recalls information from the provided context. This method checks if the response includes relevant details that should be mentioned based on the context.

2. Context Precision (context_precision): Assesses the precision of context usage in the response. It ensures that the response does not include irrelevant or extraneous information not supported by the context.

3. Faithfulness Verification (faithfulness): Verifies the faithfulness of the response to both the input and the context. This method detects any misinformation, hallucinations, or distortions, ensuring the response accurately represents the provided information.

​

When to Use Context Verification

Use Context Verification to ensure your application provides accurate and relevant responses, especially when dealing with external knowledge sources. Specifically:

• Context Recall: Use this method when you need to verify that the response includes necessary and relevant information from the provided context. Ideal for applications where omitting key details can lead to misunderstandings.

• Context Precision: Use this method when it’s important that the response only contains information supported by the context. This is crucial for maintaining accuracy and avoiding the inclusion of irrelevant or incorrect details.

• Faithfulness Verification: Use this method to confirm that the response is faithful to the input and context, ensuring there are no fabrications or inaccuracies. Essential for applications where trust and accuracy are paramount, such as legal, medical, or educational content.

​

Examples

​

Example 1: Context Recall

Verify that the response recalls necessary information from the context.

import requests

url = "https://platform.composo.ai/api/v1/evals/verify/context_recall"
headers = {
    "API-Key": "YOUR_API_KEY"
}
payload = {
    "input": "Explain the process of photosynthesis.",
    "response": "Photosynthesis is how plants make food using sunlight.",
    "context": [
        "Photosynthesis involves converting light energy into chemical energy.",
        "It occurs in the chloroplasts of plant cells."
    ],
    "reference": "Plants use photosynthesis to convert sunlight into chemical energy stored in glucose."
}

response = requests.post(url, headers=headers, json=payload)
result = response.json()

print(f"Context Recall Score: {result['score']:.2f}")  # Example output: 0.85

​

Example 2: Context Precision

Ensure the response includes only information relevant to the provided context.

import requests

url = "https://platform.composo.ai/api/v1/evals/verify/context_precision"
headers = {
    "API-Key": "YOUR_API_KEY"
}
payload = {
    "input": "Tell me about the Eiffel Tower.",
    "response": "The Eiffel Tower is a tower in Paris, and it's close to the Great Wall of China.",
    "context": [
        "The Eiffel Tower was constructed from 1887 to 1889.",
        "It was designed by Gustave Eiffel's company."
    ]
}

response = requests.post(url, headers=headers, json=payload)
result = response.json()

print(f"Context Precision Score: {result['score']:.2f}")  # Example output: 0.50

​

Example 3: Faithfulness Verification

Verify the faithfulness of the response to the input and context.

import requests

url = "https://platform.composo.ai/api/v1/evals/verify/faithfulness"
headers = {
    "API-Key": "YOUR_API_KEY"
}
payload = {
    "input": "Who wrote 'To Kill a Mockingbird'?",
    "response": "'To Kill a Mockingbird' was written by Harper Lee and published in 1960.",
    "context": [
        "'To Kill a Mockingbird' is a novel by Harper Lee.",
        "It addresses serious issues like racial injustice and moral growth."
    ]
}

response = requests.post(url, headers=headers, json=payload)
result = response.json()

print(f"Faithfulness Score: {result['score']:.2f}")  # Example output: 1.00

​

Understanding the Score

The Score is a continuous value between 0 and 1 indicating the evaluation result for the specific verification method:

• Context Recall:

  • 0: The response does not recall relevant information from the context.
  • Values between 0 and 1: The response partially recalls information from the context.
  • 1: The response fully recalls necessary information from the context.

• Context Precision:

  • 0: The response includes information not supported by the context.
  • Values between 0 and 1: The response includes some irrelevant or unsupported information.
  • 1: The response includes only information supported by the context.

• Faithfulness Verification:

  • 0: The response contains fabrications or inaccuracies compared to the input and context.
  • Values between 0 and 1: The response has some inaccuracies or is partially unfaithful.
  • 1: The response is completely faithful to the input and context, with accurate and reliable information.