Data is the most valuable asset of the 21st century. It is being generated at unprecedented speed and already drives decision-making processes in many organizations, generating astonishing value.
Everyone agrees big data is an invaluable source of insights. It allows firms to optimize everything they do, from customer targeting to supply chain operations. Being able to identify, collect, process and analyze data to get meaningful, actionable results has become the key to building agile organizations that are fit to survive in today’s data-driven business landscape.
Still, embracing big data presents difficulties that might look insurmountable at first. Most companies lack the necessary skills and know-how to build a strong corporate data culture, develop a data strategy and reconsider their whole approach to information and knowledge management.
Embracing the right data management principles can really help companies identify the reference points they need when approaching big data. We identified 5 guiding principles of data management that apply to all types of business, big or small, B2C or B2B:
- Craft a Data Management Strategy
- Define Ownership & Stewardship
- Use Metadata
- Ensure Data Quality
- Maximize Data Use
Embrace these principles to avoid common data issues and launch successful data projects.
Principle #1: Craft a Data Management Strategy
Enterprise Data Management is how an organization defines, integrates and effectively retrieves data to generate accurate, consistent insights. Because big data is in continuous expansion and evolution, identifying, accessing and using it is becoming harder and data management can be perceived as overwhelming.
This introduces us to the first data management principle:
#1: Taking a strategic approach to data management is essential for the success of your data initiatives.
An effective data management strategy entails the creation of a tailored data management system. A data management system is a framework that smoothly integrates all the software, hardware, workflows, and culture that define data management in an organization. As such, companies must identify which are the most appropriate processes, resources and technologies for their data strategy.
Further, in order to produce relevant results, data management requires companies to focus on data precision, granularity and meaning. Your data strategy should therefore cover in detail themes such as:
- which data the organization should use, when and how;
- which data sources are trustworthy and which shouldn’t be taken into account
- how to store and secure data;
- how to ensure data quality;
- how to manage the documentation associated with the data management system.
Principle #2: Define Ownership & Stewardship
Data Management often involves multiple data sources and different organizations. Indeed, data analytics becomes much more valuable once different data sources are integrated together: data integration allows firms to extract meaningful insights and drive profitability.
At the same time, the integration of several sources also makes it challenging to estimate data’s value. In fact, data itself doesn’t offer any intrinsic value, but the insights it generates when combined with other sources can be worth huge sums and be quantified. This raises important questions: who is legally in charge of the data? Who can legally access a dataset and profit from it?
#2: Good data management requires to clearly identify who owns the data
Indeed, when data ownership is unclear, critical data governance issues emerge. Ownership implies legal rights and responsibilities. It entails complete managerial and financial control over data, including the right to destroy it once it becomes useless or too expensive to store.
Ownership is usually granted to the organization who commissioned the data, rather than to the company who collected it. However, proprietary rights might refer to one specific data item as well as to a merged dataset or a value added dataset. As such, one company could own a dataset while other organisations own the data, which makes data governance even more complex.
On the other hand, legal clarity over data ownership ensures data is not misused, neglected or lost and protects the owners’ right to data royalties. Data owners should have total control over their datasets and set up procedures and processes to help employees use the data they need. Data ownership should not be confined to the IT department and access should be granted to all departments in order to shape a successful data strategy.
Organizations should clearly appoint data owners and data stewards and ensure all team members understand their role in the company’s data management system. Usually, owners are designated among senior employees and are held accountable for the quality of a defined dataset, but don’t get involved in the day-to-day activities. Stewards instead are responsible for the day-to-day management and care of a dataset based on the company’s data policy.
If you want to promote a collaborative approach to data management and avoid the creation of knowledge silos, make sure that ownership and stewardship are clearly defined for every project and employees follow procedures.
Principle #3: Use Metadata
In order to facilitate data access, data must be stored in the proper infrastructure, such as a data lake or a data warehouse, depending on your needs. Further, organizations should ensure that each dataset is properly documented and tracked in order to help employees effectively identify, manage and use data.
That’s where metadata comes into play. Metadata is the GPS system that allows your employees to easily navigate through data and keep track of any change that takes place in your databases.
#3: Use metadata to ensure data is properly managed and effectively used
Metadata is all the data that refers to your data. It tracks how data has been collected, verified, reported, and analyzed. It should give insights about the content, characteristics and use of each dataset and provide information about the content, geographic extent, currency and accessibility of the data.
To ensure proper use of metadata, each organization should develop standard corporate procedures and promote a careful, detailed approach to metadata. Companies should compile a departmental catalogue of their data and a list of all their datasets. Identifying and documenting your datasets is a good way to ensure you are not collecting or purchasing the same data twice.
Principle #4: Ensure Data Quality
Data increasingly guides how successful companies make choices. Hence, it is critically important to guarantee decision-makers can trust the data they work with. Creating data quality standards ensures datasets meet the needs of the employees accessing them.
#4: Establishing data quality standards ensures you can trust the data used in your decision-making processes
Data quality standards establish the minimum requirements data must have in order to be taken into account. Different projects might have different standards according to the goal they aim to achieve. Nonetheless, data quality requires to be defined consistently throughout the whole organization. It is therefore recommended to define some basic standards that apply to everyone in the company and to periodically test data against those standards.
One of the keys to ensure the right level of data quality is to carefully manage the data life cycle. Some data, or even datasets, can become irrelevant after a certain amount of time. Understanding data lifecycles allows firms to only store, manage and use data that is appropriate and relevant to a specific project, thus reducing the cost of storing data.
Some best practices to increase data quality include:
- - requiring employees to justify why the company should invest in such data and how its costs can be recovered;
- - integrating the data management team with experts from the departments that will use the datasets;
- - continuously running data management audits to evaluate to what extent enterprise data procedures are being followed;
- - ensuring that data is stored and maintained effectively until it becomes unnecessary or uneconomical.
Principle #5: Maximize Data Use
The fifth principle we identified is actually a summary of the four principles we’ve described so far. The common goal of all these principles is to help companies make the most out of their data. However, being able to maximize data use is so critical we decided to stress its importance by making it a principle on its own.
#5: The real value of data is how you use it
As we’ve already stated in this article, value itself has no value. How you use it makes the difference. To boost the efficiency of their data strategy, firms should therefore ensure all roles in the organization are able to flexibly access and use data. When possible, they should also be open to share some data with external stakeholders who request it.
Finally, companies should invest time and resources in crafting accurate data definitions, formats and standards; ensuring data quality; and reviewing workflows to avoid mistakes and optimize processes. Developing privacy and security policies as well as standardized rules and procedures that regulate data access and definition will help employees follow best practices and avoid potentially onerous data security issues.
Big data is revolutionizing how companies operate. Effective data governance requires continuous efforts, but promises great benefits in return. It should be the number one priority of any company that wants to stay relevant in the coming years.
Having in place a set of well-defined organizational principles will help companies successfully embrace big data from the outset. As principles are the pillars of big data projects, make sure everyone in the company understands their importance by promoting transparent communication on the ratio behind each principle.