You must independently execute Reports on a chosen cadence. Consider tools like Airflow. You can send Reports from different steps in your pipeline. For example:
first, send data quality, data drift and prediction drift checks
after you get the delayed labels, send a ML quality checks results.
You can backdate your Reports with a custom timestamp.
Set up a Dashboard to track results over time: using pre-built Tabs or configure your own choice of monitoring Panels. Check the Dashboard guide.
5
Configure alerts
Set up alerts on Metric values or Test failures. Check the section on Alerts.
Running Tests vs Reports. Structuring your evaluations as Tests - as opposed to monitoring lots of metrics at once - can help reduce alert fatigue and simplify configuration when evaluating multiple conditions at once. For example, you can quickly verify that all columns in the input data are within a defined min-max range.