Should I include system message when evaluating with Composo?

  • It’s optional whether you include in an evaluation API call, but can be helpful to include them as they provide useful context, so we do recommend trying with this included.

What’s the context limit?

  • 120k tokens

What’s an expected response time?

  • 5-15s per API call

Can I run parallel requests?

  • Yes, but we recommend limiting to 5 parallel API calls

What are the rate limits?

  • We use token bucket rate limiting for API requests
  • Unpaid users: 500 requests per hour with a bucket size of 20
  • Paid users: Higher limits based on your plan

What languages are supported?

  • Our evaluation models support all major languages plus code. A good rule of thumb is that if you don’t need a specialised model to deal with your language then we can handle it