This feature is available in Evidently Cloud. Check pricing details. Reach out if you’d like a demo.
Evidently Cloud lets you generate synthetic test inputs (and outputs) to evaluate your AI system. You can use it for:
  • Experiments. Create test data to see how your LLM app handles it.
  • Regression testing. Validate changes before deployment.
  • Adversarial testing. Check how your system handles tricky or unexpected inputs.
Once you generate the data, you can run it through your AI system and evaluate the results using the Evidently Cloud or Evidently Python library as usual.

Synthetic inputs

Generate inputs from description.

RAG dataset

Generate Q&A dataset from the knowledge source.

Adversarial tests

Generate inputs to test for vulnerabilities.

​
Example

For example, here is how you can generate test inputs.