The First Step in Data Governance: Automated Data Cataloging
Anna Yen CEO, Athemaster Co, Ltd.
Founder, Taiwan Data Engineering Association
Chairperson, Taiwan Data Management Association, TDAMA
Founder, Taiwan Data Engineering Association
Chairperson, Taiwan Data Management Association, TDAMA
In today’s era of information explosion and AI dominance, the utilization of Big Data has become a critical priority for enterprises, leading to the ever-increasing importance of Data Governance. At its core, data governance is about “governance”—much like public or environmental governance. It requires not only considering the subject of governance but also making rolling adjustments based on the nature of the organization, its members, processes, and mindsets. Data governance is as complex as any other governance issue; there is no one-size-fits-all policy. However, organizations can still establish a broad framework to categorize key priorities and sequences across different domains.
The First Step of Data Governance
According to a 2020 Gartner report, 42% of surveyed organizational leaders stated that the difficulty in implementing data governance lies in the “challenge of accessing, measuring, and monitoring governance outcomes related to data and analytics.” Governance issues often require reforms in organization and workflow, involving high complexity and uncertainty—data governance is no exception. However, business owners and leaders usually expect predictable results before committing to major changes. This often causes organizations to feel stalled at the very starting line. Where, then, should the first step of data governance begin?
Based on my experience collaborating with various industries, if an organization adopts “Automated Data Cataloging” as its starting point—using the data catalog as a hub to gradually integrate other governance modules—it can effectively facilitate organizational reform alongside data governance. A data catalog is much like a product catalog in our daily lives, used to verify if a product meets our needs. It serves as a reference table for work objectives, functioning similarly to the consumer product selection process.
Based on my experience collaborating with various industries, if an organization adopts “Automated Data Cataloging” as its starting point—using the data catalog as a hub to gradually integrate other governance modules—it can effectively facilitate organizational reform alongside data governance. A data catalog is much like a product catalog in our daily lives, used to verify if a product meets our needs. It serves as a reference table for work objectives, functioning similarly to the consumer product selection process.
Three Key Characteristics of a Data Catalog
The data catalog possesses three characteristics that make it an ideal first step for data governance:
- Tangible Visualization
It provides a “concrete” representation of governance results. With the assistance of a data catalog, the technical barrier to entry is lowered, allowing various roles within a data team to access and operate the data effectively.
- Automated Generation
Catalogs can be produced automatically without manual intervention. By scanning the metadata of data sources, a catalog is generated without involving business logic inventory, actual data extraction, or manual input. This increases efficiency and reduces human error.
- Empowering Data Product Reuse
Since the data catalog presents data clearly to users, team members no longer need to investigate or request data through multiple interfaces. This shortens the time required to create data products and helps enterprises generate and “reuse data products.”
Once a data catalog is established, the organization’s data is displayed like commodities within reach. Users can browse items via tags, timestamps, and other information to find the data they need for ideation, inquiry, and discussion. This significantly reduces the time spent on redundant investigations or recreating existing data.
A Governance Framework to Amplify Data Value
The value of data governance is multifaceted. Helping to realize the value of data and delivering predictable results is the primary momentum for the continuous promotion of data governance. A practical and stable data governance framework benefits different roles within an organization, helping them generate and reuse data products. This, in turn, accelerates and amplifies the value of data, providing complete, credible, and usable data to support the enterprise’s steady progress through the long journey of digital transformation.
The content of this article has been authorized by Chairperson Anna Yen. Unauthorized reproduction is prohibited.
The content of this article has been authorized by Chairperson Anna Yen. Unauthorized reproduction is prohibited.