The Cornerstone of Data Strategy Development – An Overview of the Implementation Framework and Key Elements of a Data Middle Platform
Dr. Eric Tsai
Associate Vice President
Business Development and Project Execution
Advant Analytics Tactics Ltd.
Associate Vice President
Business Development and Project Execution
Advant Analytics Tactics Ltd.
Introduction to the Data Middle Platform
The Data Middle Platform is a highly popular concept in recent years. Its core philosophy is data sharing. By utilizing technical methods such as data aggregation, data standardization, and data virtualization, it is designed to form a platform that centrally manages data sources and applications, supporting hundreds of internal business applications. This helps to avoid redundant construction and incomplete definitions of data indicators. Simply put, the Data Middle Platform abstracts, modularizes, and services enterprise data, knowledge, and processes to flexibly support front-end applications and innovative services. Therefore, the Data Middle Platform is not merely a technical topic—it is a conceptual framework, a strategic choice, and an organizational form.
The Core Capabilities of a Typical Data Middle Platform Include:
The Core Capabilities of a Typical Data Middle Platform Include:
- Data Aggregation and Integration Management: Collecting and integrating data from various sources, including both structured and unstructured data.
- Data Processing Workflow and Management: Cleaning, transforming, and processing raw data to extract valuable information.
- Data Servitization Management: Providing processed data in standardized formats (such as API services) to upper-layer application systems.
- Data Value-Adding Functions and Management: Discovering the business value of data through analysis and mining, creating value for the enterprise.
Important Concepts Beyond Functional Capabilities Include:
- The highest guiding principle of the Data Middle Platform is the establishment and activation of enterprise data assets.
- The Data Middle Platform is not merely a centralized processing hub for enterprise data; more generally, it manages the enterprise’s data services.
- It does not overturn all existing system operation modes; existing data usage models can be retained.
- Providing autonomous data services and standardized data services is a key capability of the Data Middle Platform.
- The Data Middle Platform must be operated in conjunction with data governance—especially for cross-business-line data integration. Appropriate governance strategies are necessary to ensure the platform’s sustainable operation.
Implementing and Operating a Data Middle Platform Brings Numerous Benefits to Enterprises:
- Enhanced Decision Support: Providing accurate and timely data resources and analytical tools helps decision-makers make better decisions, improving business efficiency and competitiveness.
- Enhanced Operations and Collaboration: The Data Middle Platform breaks down data silos, accelerates collaboration, strengthens operations, and enhances enterprise operational models.
- Reduced Costs and Resource Waste: It reduces repetitive tasks, lowers costs, and avoids waste of resources, making enterprises more competitive.
- Increased Data Value: Through its mechanisms, enterprises can activate data, achieve co-construction and sharing, and thereby enhance data value.
- Optimized Business Processes: Analyzing and optimizing business processes improves efficiency and flexibility while lowering costs and risks.
- Fostered Product Innovation: By deeply understanding market needs and user feedback, the platform enables agile and rapid product development and innovation.
Data Middle Platform Architecture
The architecture of a Data Middle Platform can generally be described in multiple layers. As illustrated in Figure 1, the foundational data platform layer of the Data Middle Platform serves as the entry point for connecting to enterprise backend systems or databases. This foundation layer also includes an analytics engine that provides out-of-the-box support for data analysis and AI modeling. The value-added layer stores data such as business tags, indicators, scoring information, and more.
The analytics workflow block houses components for data analysis, processing pipelines, and models. To broaden the scope of services, it is recommended that the analytics workflow block be equipped with interfaces that support no-code, low-code, and all-code development. This enables rapid data preparation and enhances analytical efficiency. The user interface module not only provides standard reports and dashboards, but also packages data processing flows through the middle platform and converts them into API services. This strengthens the scalability of data services. Data services can be delivered through multiple channels, such as dashboards, database tables, flat file exports, or APIs. Delivering data services in a standardized and controllable way helps ensure data quality and timeliness. In addition to system management mechanisms, the middle platform management module must also consider the usage status and performance tracking of the data services provided through the platform.
The analytics workflow block houses components for data analysis, processing pipelines, and models. To broaden the scope of services, it is recommended that the analytics workflow block be equipped with interfaces that support no-code, low-code, and all-code development. This enables rapid data preparation and enhances analytical efficiency. The user interface module not only provides standard reports and dashboards, but also packages data processing flows through the middle platform and converts them into API services. This strengthens the scalability of data services. Data services can be delivered through multiple channels, such as dashboards, database tables, flat file exports, or APIs. Delivering data services in a standardized and controllable way helps ensure data quality and timeliness. In addition to system management mechanisms, the middle platform management module must also consider the usage status and performance tracking of the data services provided through the platform.
Figure 1. The architecture of a Data Middle Platform
The user roles supported by the Data Middle Platform are extensive, ranging from business users and data analysts to system engineers. Therefore, the platform’s operation is modular and can be triggered based on varying user needs.
Figure 2. Operating Model-1
Figure 2. Operating Model-2
As illustrated in Figure 2, business users access relevant information directly through dashboards housed in the Data Middle Platform, while data scientists can call upon data processing flows and analytical workflows in combination with existing tag data for advanced data analysis and modeling.
As previously stated, the Data Middle Platform is a cross-business-line integrated data service platform. However, it is designed to categorize and manage data services. This layered governance framework—established according to this design—can address the complexities and governance challenges typically associated with data architectures. By incorporating internal enterprise data services into the platform through standardized operating procedures, and by leveraging the platform’s quality control, value enhancement, and data sharing mechanisms, organizations can effectively amplify the value of their data assets.
As previously stated, the Data Middle Platform is a cross-business-line integrated data service platform. However, it is designed to categorize and manage data services. This layered governance framework—established according to this design—can address the complexities and governance challenges typically associated with data architectures. By incorporating internal enterprise data services into the platform through standardized operating procedures, and by leveraging the platform’s quality control, value enhancement, and data sharing mechanisms, organizations can effectively amplify the value of their data assets.
Key Elements for Building a Data Middle Platform
A Data Middle Platform is a conceptual framework. How an enterprise successfully develops a Data Middle Platform based on its own needs can be explored from several perspectives:
圖三、數據中台實施成功要素
- Data Development Strategy: Establish efficient data sharing, emphasize the creation and value of data assets, and reduce redundant system investments.
- Development Drivers: Plan around dual pillars of “business-driven” and “technology-driven” approaches. It is recommended to prioritize “business-driven” in the early stages to quickly support business applications.
- Promote Integration and Refinement: Break down silos within the organization, including data, visualization, and processes.
- Dedicated Unit: Form a cross-functional team that includes members from both IT and business departments.
- Priority of Objectives: The dedicated team should prioritize objectives based on their business value, considering (1) resource allocation and (2) opportunity costs.
- Development Model: Adopt an agile development model, aiming to build a flexible, shareable, composable, and reusable service platform.
- Operations and Monitoring: Establish internal data cycles within the platform, and monitor both quality and performance.
Finally, an important reminder: the implementation of a middle platform involves the integration of existing enterprise data and service processes. Through the platform, enterprises can reshape and optimize these processes. Deconstructing, reengineering, and even innovating existing workflows is a key task that must be clearly defined from the early stages of Data Middle Platform construction. In addition to internal personnel participation, seeking assistance from external consultants with relevant experience can help guide planning and execution in a more neutral and efficient manner.