Data Mesh - the evolution of data platform
Data mesh is a solution architecture for the specific goal of building business-focused data products.
What is data mesh?
Data mesh is a new architectural approach that emphasizes the decentralization of data ownership, governance, and infrastructure. It responds to the limitations of traditional centralized data warehouses, which can be inflexible, expensive to maintain, and difficult to scale.
Figure: The evolution of data solution
Key elements of data mesh
Data product
Cleaned & transformed for analytical purpose
Transformation
Merged & transformed to answer questions
Global policies
Rules of play by The federated governance group
Benefits
Why adopting data mesh can be advantageous?
A data mesh approach can help break down these data silos and enable more effective sharing and collaboration across different departments and systems.
Teams can iterate and experiment with data pipelines and analytics independently
Establishing explicit data ownership and governance, enabling data stewards to take responsibility for data quality and ensure that data is accurate, consistent, and up to date.
A data mesh approach with clear data ownership and consent can help ensure that data is securely and appropriately shared only with authorized parties, while still enabling data discovery and exploration.
A central data team cannot handle all the analytical questions of management and product owners quickly enough. Data mesh decentralizes these tasks, enabling multiple domain teams to work concurrently, thereby reducing bottlenecks and accelerating data delivery.
Distributing data ownership and responsibilities to domain-specific teams allow quicker responses to changing business needs.
Is Data Mesh for your company?
Data mesh solution template - Architecture & Pre-defined modules?
Data mesh architecture, from a logical perspective, decentralizes data ownership and governance, distributing data responsibility to individual domain-oriented teams. This approach aims to improve data scalability and accessibility by treating data as a product and fostering a more agile and self-service data ecosystem.
Figure: Data Mesh logical view
Governance group
Define global policy, guidance to manage data across domain.
Establish and maintain data practice.
Data platform team
Design, build and maintain data platform that forms the foundation of data mesh architecture.
Manage data infrastructure and ensure it is robust, secure and meet requirement of domain team.
Collaborate with domain team to understand data requirement and provide technical support when needed.
Domain team
Own and know their business domain.
Design, build and run solution on their own.
Develop, maintain, and expose data product to other domains within organization.
Built-in technologies