Get up and running with Composo in under 5 minutes. This guide will help you evaluate your first LLM response and understand how Composo delivers deterministic, accurate evaluations.

​
What You’ll Build

In this 5 minute quickstart, you’ll:
  • Set up your Composo account and API access
  • Evaluate an LLM response for quality and accuracy
  • Understand how to interpret Composo’s scores and explanations
  • Learn the difference between reward (0-1 scoring) and binary (pass/fail) evaluations

​
Step 1: Create Your Account

Sign up for a Composo account at platform.composo.ai.

​
Step 2: Generate Your API Key

  1. Navigate to Profile → API Keys in the dashboard
  2. Click Create New API Key
If your organization has a fine-tuned model with Composo, all API keys created with organization accounts will automatically route to that finetuned model.

​
Step 3: Run Your First Evaluation

First, install the SDK: 
pip install composo
Now let’s evaluate a customer service response for empathy and helpfulness using the Composo SDK: 
from composo import Composo

# Initialize the client with your API key
composo_client = Composo(api_key="YOUR_API_KEY")

# Example: Evaluating a customer service response
result = composo_client.evaluate(
    messages=[
        {"role": "user", "content": "I'm really frustrated with my device not working."},
        {"role": "assistant", "content": "I'm sorry to hear that you're experiencing issues with your device. Let's see how I can assist you to resolve this problem."}
    ],
    criteria="Reward responses that express appropriate empathy if the user is facing a problem they're finding frustrating"
)

# Display results
print(f"Score: {result.score}")
print(f"Analysis: {result.explanation}")

​
Understanding the Results

Composo returns:
  • Score: A value between 0 and 1 (e.g. 0.86 means the response strongly meets your criteria)
  • Explanation: Detailed analysis of why the response received this score
Example output: 
Score: 0.67/1.00
Analysis: The assistant demonstrates appropriate empathy by acknowledging the user's frustration and offering assistance, though there was room for slightly more personalized emotional validation.

​
Step 4: Try Different Evaluation Types

​
Binary Evaluation (Pass/Fail)

For simple pass/fail checks, use the binary endpoint: 
from composo import Composo

composo_client = Composo(api_key="YOUR_API_KEY")
result = client.evaluate_binary(
    messages=[
        {"role": "user", "content": "I'm having headaches every morning. What should I do?"},
        {"role": "assistant", "content": "You should consult a healthcare professional for proper advice."}
    ],
    criteria="Response fails if the assistant provides medical advice."
)

print(f"Passed: {result.passed}")
print(f"Explanation: {result.explanation}")

​
RAG Accuracy Evaluation

Evaluate how faithfully an LLM uses retrieved context: 
from composo import Composo

composo_client = Composo(api_key="YOUR_API_KEY")

# Example: Evaluating how well an LLM uses provided context
result = composo_client.evaluate(
    messages=[
        {
            "role": "user", 
            "content": """What is the current population of Tokyo?

Context:
According to the 2020 census, Tokyo's metropolitan area has approximately 37.4 million residents, making it the world's most populous urban agglomeration. The Tokyo Metropolis itself has 14.0 million people."""
        },
        {
            "role": "assistant", 
            "content": "Based on the 2020 census data provided, Tokyo has 14.0 million people in the metropolis proper, while the greater metropolitan area contains approximately 37.4 million residents, making it the world's largest urban agglomeration."
        }
    ],
    criteria="Reward responses that accurately use the provided context and cite specific data points"
)

print(f"Score: {result.score}")
print(f"Explanation: {result.explanation}")

​
What’s Next?

Now that you’ve made your first evaluation, explore more advanced features:
  1. SDK Documentation - Learn how to use the Python SDK
  2. Writing Effective Criteria - Learn how to craft precise evaluation criteria for your use case
  3. Criteria Library - Browse pre-built criteria for common evaluation scenarios
  4. REST API Reference - Explore all available endpoints and parameters
  5. Use Cases - See examples for RAG, customer service, content generation, and more