There are multiple angles of quality that we pay attention to:
- Quality of the inputs (source data quality)
- Quality of the deliverables (KPIs, benchmarks, forecasts, models, recommendations, etc.)
- Quality of the development pipeline (change control and testing before release)
- Quality of the value or operations pipeline (detecting changes in data, monitoring models and watching for drift, etc.)
Multiple roles may be involved in ensuring quality:
- The developers (e.g. data scientist, data engineer, ML engineer, data analyst, or other)
- The development team (e.g. peers reviews, manager reviews, etc.)
- The product owner (deserves special mention)
- The user or a representative of the user (e.g. UAT)
I’d like to borrow from an excellent QA methodology created by FedEx named: Quality Driven Management
The key principles in QDM are:
- Quality is defined by the customer
- Be scientific
- Measure measure measure
- Optimize business performance
- Quality involves teamwork
- View failures as opportunities
In the weeks and months to come I will be writing more about how to apply these principles to ensure adequate quality in BI and AI projects while also remaining fast. More to come!
