5 Compelling Reasons Chief Data and Analytics Officers are Moving to Data Mesh

The COVID-19 pandemic taught us a crucial lesson: to survive and thrive in today’s volatile, uncertain, complex, and ambiguous world, organizations must manage change and make decisions more quickly than ever. As Maverick in Top Gun would say, “I feel the need… the need for speed!”

To adapt to this reality, Chief Data and Analytics Officers must help their organizations reimagine themselves around customer journeys, product development, and other value-creation processes. This often means moving away from multilayered, command-and-control functional structures into simpler forms. Enterprise agility involves redistributing authority and decision-making to autonomous teams, each with a clear purpose and the skills and resources required to achieve business objectives.

For example:

• Retailers are organizing teams around specific product categories to boost volume and margin.
• Financial services providers are structuring teams around “know your customer” and anti-money laundering activities to improve regulatory compliance.
• Life Sciences companies are organizing around the drug discovery process to speed up time to market and increase profitability.

To achieve business agility, organizations must not only change their structure but also who is responsible and accountable for managing data. Chief Data and Analytics officers are reevaluating traditional centralized data management approaches that are slow and cumbersome, as they make quickly adapting to new business situations difficult. Increasingly they are finding the new architectural approach of data mesh a compelling option to empower business agility by enabling domain teams to take charge of data curation and use.

What Is Data Mesh?

Data mesh is an approach to data and data interfaces for efficient and business-aligned consumption and value delivery. It serves business domains with relevant, timely, high-quality data views and perspectives packaged as business-relevant data products or services. These data products encapsulate all the functionality required for a specific business need, such as product demand forecasting. We will cover data products in more detail in an upcoming blog.

Data mesh shifts the paradigm of designing data architecture to be oriented around business domains, which are functional activities and processes required for a business outcome. For example, delivering consistent and personalized customer experiences across channels and touchpoints. It deconstructs traditional monolithic data architecture into decentralized components designed to support specific business domain needs. And it changes the operating model for data governance to a federated approach that requires chief data and analytics officers to carefully balance the need for domain team autonomy with the need for cross-domain interoperability.

How Data Mesh Enables Agility

Chief Data and Analytics officers that have moved to data mesh say the characteristics that enable business agility include:

Local Autonomy: With data mesh, the centralized data architecture is distributed across smaller, independent units owned and managed by individual teams, promoting a more agile and flexible approach to data management.

Self-Service: Data mesh encapsulates all the capabilities that facilitate an end goal using data. This simplifies understanding and use, empowering individuals of all technical skill levels to make informed decisions and act based on real-time data.

Continuous iteration: Data mesh is built on the principles of continuous iteration, allowing teams to experiment with their data products and infrastructure, make changes quickly, and iterate based on feedback.

Interoperability: In data mesh, the focus of governance shifts from top-down control to guiding interoperability of data products. The framework provides a structure for creating local autonomy while ensuring adherence to the minimum set of global rules.

Five Compelling Business Reasons For Data Mesh

The whole premise of data mesh is to help organizations make business decisions faster by shifting responsibility and accountability for data and analytics to the domain teams that best understand the business situation and context. This paradigm shifts promotes a culture of innovation and adaptability, enabling organizations to respond rapidly to changing market conditions and stay ahead of their competitors. Five compelling business reasons chief data and analytics officers report for moving to data mesh are:

Faster time-to-market: By identifying market trends and customer needs faster, businesses using data mesh can launch products and services more quickly than their competitors, gaining a competitive edge.

Better customer satisfaction: Companies using data mesh can tailor their products and services to meet specific customer needs, resulting in better overall customer satisfaction.

Improved innovation: Data mesh accelerates innovation by allowing organizations to pivot and adapt to new ideas and technologies quickly, freeing them from rigid structures and processes.

Increased efficiency: Data mesh helps organizations optimize processes more quickly, allowing them to identify and address inefficiencies faster and reduce costs, improving their bottom line.

Greater resilience: Data mesh makes businesses more resilient by enabling them to adapt to changes in the market, such as economic downturns or disruptions to supply chains, and emerge stronger on the other side.

Best Practices For Data Mesh Success

To successfully implement data mesh, keep the following best practices in mind.

Design domains around one business capability. The model for a domain only needs to include the necessary business capabilities for the domain. And each model should only contain the relevant business entities and attributes within the domain context. Also, map the connections to other domains to create a relationship graph of shared entities and attributes. To learn more about Domains watch for our upcoming blog on the topic.

The federated governance model requires clearly defined boundaries of what the central governance team is accountable for and what the individual domain teams are responsible for.  Document the processes and workflow that will be used to coordinate activities between domain teams. And create a communication process to provide greater visibility across teams and trust between teams.

Data products require careful consideration of what business problem is being solved. They should encapsulate all the needed functionality, including data pipelines, curated data sets, machine learning algorithms, and visualizations. Don’t assume data stewards, data engineers, and business analysts should be repurposed as data product managers, define the skills and knowledge needed and then determine who will be a good fit.

Self-serve is much more than a data catalog. While catalogs play an essential role in helping data consumers search for and find data products, consumers also need to trust and understand what they find. Semantic layers and ontologies help reconcile glossaries reflecting local domain language to create a shared understanding of data products and their connection. Consumers also need guidance on the compliant use of data products through governance policies and data contracts to describe explicit terms and conditions for use. To learn more about the data mesh principle of self-serve data platforms, watch for our upcoming blog.

Conclusion

Decision-making speed and agility are critical to success in today’s fast-changing world. Data mesh is an architectural design pattern that distributes responsibility for data to small autonomous teams focused on specific business domains. By taking a business-centered, technology-enabled approach, data mesh helps organizations generate more business value from their data.