Data mesh: Transforming data from byproduct to product

One of the four principles of data mesh is data as a product. The core tenet of this principle is a shift in mindset from data as a byproduct of transactional systems and processes to data purposefully designed and packaged as a “product” for an analytical need. This shift in mindset is facilitated by applying product management practices to the design of data products, including defining the product vision and strategy, creating the development roadmap, and ongoing management of quality and usability.

Activities in the product management process include:

Understanding the user: The first step in the product management process is to understand the user. This includes understanding the user’s needs, wants, and pain points.
Scoping the problem: Once you understand the user, you need to define the scope of the problem that your product will solve. What is the user trying to achieve? What are the obstacles that are preventing them from achieving their goals?
Setting goals: What metrics will be used to measure product success? How progress against the metrics be tracked and communicated?
Brainstorming solutions: Once you have defined the problem, you can start brainstorming solutions. What are different ways that you can solve the user’s problem?
Prototyping: Once you have some ideas for solutions, you can start prototyping. A prototype is a rough, working model of your product. It doesn’t have to be perfect, but it should be enough to give you a sense of how the product will work.
Testing: Once you have a prototype, you must test it with users. This will help you to identify any problems with the product and to make sure that it is meeting the user’s needs.
Iterating: Once you have tested your product, you need to iterate. This means making changes to the product based on the feedback that you received from users.

The shift in mindset to data products requires new skills and disciplines that your current data and analytics teams might not have. Don’t assume data stewards, engineers, and scientists should be repurposed as data product managers. Define the requirements first and then determine who will be a good fit based on those requirements. You may need to look outside traditional data and analytics roles to find the right people.

Some things to consider include:

Technical skills: Product managers need to have a strong understanding of the technical aspects of their products. This includes understanding the underlying technologies, the capabilities of the team, and the limitations of the platform.
Business skills: Product managers need to have a strong understanding of the business use case. This includes understanding the potential impact on revenue, costs, and risk management.
Process discipline: Product managers need to be able to manage the product development process effectively. This includes setting clear goals, defining the roles and responsibilities of team members, and tracking progress.
Communication skills: Product managers need to be able to communicate effectively with a variety of stakeholders. This includes communicating the product vision, the product roadmap, and the product status.
Project management discipline: Product managers are accountable for the success of their products. This means being responsible for setting and meeting goals, managing the product development process, and communicating with stakeholders.

As mentioned in our article, Data Mesh: Understanding Decentralized Domain Ownership, with data mesh, responsibility and accountability for data are distributed to domain teams that best understand the business needs and context. This includes creating and managing data products. And while there is local domain autonomy in determining data product characteristics like business entities, attributes, and hierarchies, domain teams are also responsible for ensuring the interoperability and reusability of data products across domains.

In her book, Data Mesh: Delivering Data-Driven Value at Scale, Zhamak Dehghani outlines the attributes data products need to ensure usability. Keep this in mind as you start on your data products journey.

Discoverability: Data products should be easy to find and explore. This can be done by providing a central catalog or registry of data products, as well as by making data products discoverable through search.
Addressability: Each data product should have a unique, persistent address that can be used to access it programmatically or manually. This address should be consistent even as the data product evolves over time.
Understandability: Data products should be easy to understand. This means providing clear documentation about the data product’s semantics, syntax, and schema. It also means providing sample datasets and example code to help users get started.
Trustworthiness and Truthfulness: Data products should be trustworthy and truthful. This means providing information about the data product’s quality, timeliness, completeness, and accuracy. It also means providing data provenance and lineage information.
Native Accessibility: Data products should be accessible to a wide range of users, with different needs and expectations. To do this, data products should be made available in multiple formats, so that users can access them using their preferred tools. For example, a data product could be made available as a spreadsheet, a SQL database, or a REST API.
Interoperability: Data products should be easily combined with other data products. This allows users to create more complex and powerful analyses. To make data products interoperable, they should conform to the minimum standards defined as part of your governance activities.
Valuable on its Own: Data products should be valuable to users, even if they are not combined with other data products. This means that data products should be well-curated and contain high-quality data. Additionally, data products should be well-documented so that users can understand their contents and how to use them.
Secure: Data products should be secure, so that users can be confident that their data is protected. This means that access should be controlled and there are clear policies for compliant use. Additionally, data products should be regularly monitored for security vulnerabilities.

Conclusion

Data products are the primary deliverable of data mesh that enable organizations to make business decisions faster and quickly adapt to changing business conditions. The change in mindset from data as a byproduct to data as a product requires more rigor and discipline in the curation and consumption of data for analytical needs.

Discover the benefits of creating an intuitive data catalog to fit your needs! Request a demo of DataGalaxy’s Data Knowledge Catalog, an all-in-one Data Catalog that offers out-of-the-box actionability with fully-customizable attributes, powerful visualization tools, standardized business glossaries, and AI integration to help organizations easily document, link, and track all their metadata assets on one dynamic platform.

To learn more about the business benefits, see 5 Compelling Reasons Chief Data and Analytics Officers are Moving to Data Mesh.