Data governance technologies have evolved so that the best among them provide complete transparency into an organization’s data landscape, and provide business users a hassle-free experience when searching for, accessing, and applying data.
Keeping enterprise data working effectively for any organization is about a lot more than just the data. Rather, it’s about ensuring the health of the data ecosystem as information flows and functions across the entire data supply chain. Fundamental to those data management efforts is data governance, which, at its core, delivers visibility into, strengthens accountability for, and enables utilization of, an organization’s data assets.
Data governance is not a new practice, but it is a constantly developing one. Data governance today is very different than it was five or even two years ago. The evolution of data governance spans years of trial and error and progressing technologies. Every misstep experienced, and every advancement made, marked another opportunity to create better data governance solutions featuring business process automation and focused on increasing data accountability and understanding.
The Beginning of Data Governance
In many organizations, data governance initiatives began as a manual documentation exercise using basic tools like spreadsheets or Microsoft SharePoint. Business stakeholders from a single line of business would document data definitions and define ownership/stewardship responsibilities for their department’s data assets using these simple tools. Often, these efforts were confined to a specific department or project.
Within the IT department, data governance efforts were often concerned with risk mitigation from a compliance perspective or managing and understanding technical data lineage. This helped users understand how data had moved and transformed over time to ensure regulatory compliance.
However, this model of data governance siloed business units and IT; and failed to leverage data as a critical business asset. Also, as the volume of data grew over time, leveraging spreadsheets and SharePoint to document governance efforts quickly became unscalable and unsustainable.
As a result, organizations realized they needed to move away from a siloed approach and converge both business and technical data worlds. Instead of utilizing spreadsheets to document information about data, they adopted data governance-oriented tools from vendors.
From Spreadsheets to Vendor-Based Tools
Data governance tools helped organizations merge business and technical data into singular views, manage basic lineage, and create workflows to manage and provide transparency into the data assets. The tools helped organizations offer business definitions and ownership/stewardship responsibilities for an organization’s data assets. As a result, everyone in the organization knew what resource to access when he or she had questions about their data.
At first, this worked great. But, as the volume and depth of data increased due to the introduction of new technologies like big data stacks, streaming data and data lakes, these tools were quickly overwhelmed.
Regulatory requirements, such as GDPR, BCBS 239, CCAR, Solvency II, and MiFID also put an increased emphasis on data. Users started finding new and creative ways to utilize data, and they increasingly wished for the ability to emulate the ‘Amazon Marketplace’ experience when searching, requesting and accessing their organization’s data assets.
Modern Data Governance
Today, new data governance technologies and a complete data governance framework address the challenges that traditional legacy tools struggled with by leveraging machine learning, automation and recommendation engines in the collection, validation, and analyses of the data.
**The goal is to considerably reduce the manual efforts historically involved in populating and maintaining a data governance tool. Also, today’s tools should make searching for data a simple and successful process for business users.** By providing an intuitive and easy-to-use interface tailored to how they consume data, it emulates the Amazon customer experience model for fast and easy shopping.
To accomplish these goals, organizations must expand data access, data quality, and machine learning analytics to business users that wish to self-service their data needs.
Today’s on-demand economy has also created demanding consumers, and the data consumer is no different. Business users frequently need to find, sort and analyze data fast, as speed to insights matter in a fast-paced and hyper-competitive business climate. Modern data governance can also bring previously siloed disciplines together to empower business users to do more with data without a need for technical expertise.
**By connecting the disciplines of visual data prep, data quality, machine learning, governance workflows, and dashboards, organizations can empower business users to perform functions which previously required the technical intervention and expertise of IT resources.**
Today’s modern tools allow business users to utilize an intuitive drag-and-drop interface to quickly combine data sets, apply prepackaged data quality routines without complex coding and subsequently analyze data by applying machine learning algorithms to enrich the analysis.
Ultimately, business users are empowered to quickly consume the output in visual dashboards with meaningful data metrics upon which decisions can be made.
Evolution to a business-centric data governance model also requires zero-code workflows, with easy-to-use interfaces and pre-defined workflow routines, so business users can expedite the creation of new workflows and edit existing workflows without dependence on coding expertise.
This facilitates the evolution to a more automated, self-sufficient governance framework that is designed around the business user. Data governance processes and technologies are going to continue to evolve. It will always be an ongoing process with no end date in sight.
However, for those who continue to maintain and develop their data governance model, they will increase efficiencies and profitability across the enterprise and help gain a competitive advantage.