Synthetic edge cases and tricky inputs
Adversarial tests are designed to challenge AI models by exposing weaknesses and vulnerabilities. These inputs may attempt to:
Evidently Cloud lets you automate adversarial test generation based on defined categories of risk.
You can configure your own adversarial dataset.
In the Evidently UI, start a new Project or open an existing one.
Choose a predefined adversarial scenario:
You can choose the following categories:
After selecting a scenario
Some categories allow customization, such as selecting specific forbidden topics (e.g., legal, financial, or medical advice).
You can configure multiple scenarios at once.
You can choose to:
Combine multiple scenarios into a single dataset. If you select multiple categories (e.g., Brand Image and Forbidden Topics), they will be included in the same dataset, with a separate “scenario” column to indicate the category of each test case.
Export each scenario separately. Generate individual datasets for each selected test type.
Once generated, you can:
Dataset API. How to work with Evidently datasets.
Synthetic edge cases and tricky inputs
Adversarial tests are designed to challenge AI models by exposing weaknesses and vulnerabilities. These inputs may attempt to:
Evidently Cloud lets you automate adversarial test generation based on defined categories of risk.
You can configure your own adversarial dataset.
In the Evidently UI, start a new Project or open an existing one.
Choose a predefined adversarial scenario:
You can choose the following categories:
After selecting a scenario
Some categories allow customization, such as selecting specific forbidden topics (e.g., legal, financial, or medical advice).
You can configure multiple scenarios at once.
You can choose to:
Combine multiple scenarios into a single dataset. If you select multiple categories (e.g., Brand Image and Forbidden Topics), they will be included in the same dataset, with a separate “scenario” column to indicate the category of each test case.
Export each scenario separately. Generate individual datasets for each selected test type.
Once generated, you can:
Dataset API. How to work with Evidently datasets.