From Data Quality Monitoring to Successful Data Governance
Anna Yen CEO, Athemaster Co, Ltd.
Founder, Taiwan Data Engineering Association
Chairperson, Taiwan Data Management Association, TDAMA
Founder, Taiwan Data Engineering Association
Chairperson, Taiwan Data Management Association, TDAMA
The rise of the AI boom has made organizations realize that the effective application of data is the fuel driving the AI revolution. Data Governance is the key to unlocking this potential, and its importance for corporate business decision-making continues to grow. For enterprises committed to excellence in data governance, beyond adopting “Automated Data Cataloging” as a starting point, monitoring and optimizing data quality—as outlined in the DAMA (Data Management Association) framework—is a critical means of implementation.
I believe that when evaluating data quality, enterprises should think like a restaurant chef sourcing ingredients. This involves setting “standards” and “expectations” (judging flavor and appearance) and performing “inspections” (verifying goods against those standards). To ensure the same dataset can be shared and used across different departments, data quality cannot be judged by subjective standards. Instead, it requires pre-classified verification standards. Only through statistical results can the condition of an item be determined across specific dimensions. By mastering these sub-items of data quality, companies can further optimize specific aspects of their data. Conversely, poor-quality data is like debris that shouldn’t be on a dinner plate. An enterprise that implements data governance is like a restaurant with a robust ingredient management system: only then can the customer experience be truly controlled.
How can we accurately grasp data quality?
I believe that when evaluating data quality, enterprises should think like a restaurant chef sourcing ingredients. This involves setting “standards” and “expectations” (judging flavor and appearance) and performing “inspections” (verifying goods against those standards). To ensure the same dataset can be shared and used across different departments, data quality cannot be judged by subjective standards. Instead, it requires pre-classified verification standards. Only through statistical results can the condition of an item be determined across specific dimensions. By mastering these sub-items of data quality, companies can further optimize specific aspects of their data. Conversely, poor-quality data is like debris that shouldn’t be on a dinner plate. An enterprise that implements data governance is like a restaurant with a robust ingredient management system: only then can the customer experience be truly controlled.
How can we accurately grasp data quality?
- Establish Evaluation Standards: Quantify the characteristics of the target data.
- Execute Inspection: Use statistical methods such as sampling and scanning to calculate quantities.
- Evaluate Results: Use the predefined standards to assess the statistical results, achieving the goal of quantifying data characteristics.
Through these three procedures, quality is converted into numbers that everyone can understand and verify using the same criteria. Verification is conducted across dimensions such as:
- Syntactic Accuracy
- Completeness
- Timeliness & Uniqueness
- Consistency
- Validity & Semantic Accuracy
Through continuous, scheduled, and localized verification, enterprises can consistently monitor data quality. However, based on my experience working with various firms, there is a practical warning: Avoid setting the Data Quality Management team’s KPIs solely on improving a “total composite score.”
If a team focuses only on a total score, they may prioritize items where the percentage is easiest to boost, potentially ignoring existing data products or the strategic direction of company policy. This results in “putting the cart before the horse.” Data governance must align with the organization’s shared goals—whether that is cybersecurity, AI model efficiency, or identifying new data-driven opportunities. By linking governance metrics to actual business objectives, teams avoid wasting effort on low-priority data, ensuring that governance results remain correlated with actual revenue and efficiency.
Only properly managed data can deliver value in the AI era. “Data Governance” is essentially the “management of data management,” encompassing organization, regulations, and processes. The effectiveness of an enterprise’s data governance can be measured by the accuracy, reliability, and connectivity of its data. To stand out in an AI-driven competitive landscape, enterprises must do more than just build a sound governance framework; they must implement correct data verification and monitoring methods. By continuously focusing on and optimizing data quality, organizations ensure the maximization of their data value, maintaining a decisive advantage in the long run.
The content of this article has been authorized by Chairperson Anna Yen. Unauthorized reproduction is prohibited.
If a team focuses only on a total score, they may prioritize items where the percentage is easiest to boost, potentially ignoring existing data products or the strategic direction of company policy. This results in “putting the cart before the horse.” Data governance must align with the organization’s shared goals—whether that is cybersecurity, AI model efficiency, or identifying new data-driven opportunities. By linking governance metrics to actual business objectives, teams avoid wasting effort on low-priority data, ensuring that governance results remain correlated with actual revenue and efficiency.
Only properly managed data can deliver value in the AI era. “Data Governance” is essentially the “management of data management,” encompassing organization, regulations, and processes. The effectiveness of an enterprise’s data governance can be measured by the accuracy, reliability, and connectivity of its data. To stand out in an AI-driven competitive landscape, enterprises must do more than just build a sound governance framework; they must implement correct data verification and monitoring methods. By continuously focusing on and optimizing data quality, organizations ensure the maximization of their data value, maintaining a decisive advantage in the long run.
The content of this article has been authorized by Chairperson Anna Yen. Unauthorized reproduction is prohibited.