Synthetic edge cases and tricky inputs
Adversarial tests are designed to challenge AI models by exposing weaknesses and vulnerabilities. These inputs may attempt to:
Evidently Cloud lets you automate adversarial test generation based on defined categories of risk.
You can configure your own adversarial dataset.
In the Evidently UI, start a new Project or open an existing one.
Choose a predefined adversarial scenario:
You can choose the following categories:
After selecting a scenario
Some categories allow customization, such as selecting specific forbidden topics (e.g., legal, financial, or medical advice).
You can configure multiple scenarios at once.
You can choose to:
Combine multiple scenarios into a single dataset. If you select multiple categories (e.g., Brand Image and Forbidden Topics), they will be included in the same dataset, with a separate “scenario” column to indicate the category of each test case.
Export each scenario separately. Generate individual datasets for each selected test type.
Once generated, you can:
Dataset API. How to work with Evidently datasets.