Getting the Most out of Big Data
The definition of big data is evolving. Originally, the word “big” was a reference to the size and complexity of the data ecosystem of an organization. The amount of data produced is growing at exponential rates and the amount of data collected, processed, or disseminated is becoming more complex and interwoven. However, this never-ending, expanded cycle of data integration is in contrast with the finite amount of resources an organization has to discover and translate that data into valuable information. If we focus our limited resources on traditional data management activities like data standards, defining source systems for each data element, or evaluating data quality, we could become so consumed with perfecting the data that we won’t spend enough time using it. We need to target our data management activities by prioritizing data assets so that we can also support transforming that data into insightful information. Big data isn’t just describing the volume of the data; it is also describing the value of the data and how it can be transformed into powerful, decision-making information. That’s the big question: how do we use big data to improve efficiency within government and the lives of citizens?
Building Data Governance
Getting started on this journey requires participation and support throughout the organization. The Chief Data Officer (CDO) should have the authority, skills, and ability to bridge the gap between the organizational leaders who determine the core strategic vision and goals, the key executives and program offices who establishes key metrics, and the front line data stewards and analytical teams that can transform data into information that aligns with the defined metrics.
How do we use big data to improve efficiency within government and the lives of citizens?
Data governance should always focus on business outcomes. It is easy to fall into the discussion on the latest technological tools used to analyze data, and often these conversations focus on solutions before the problem. Technology solutions are necessary to support big data activities. Incremental data discovery projects that transform data into insights for specific business needs, or drive major decisions for an organization, should then be used to inform the broader technology strategy. A “technology first” approach has a high risk of failure, but modernizing technology is a key component to successful data management. The CDO will need to help navigate the conversations between business outcomes, defined metrics, data assets, and technology solutions; combined, they can help deliver a successful big data program.
Assessing the Status Quo
It is difficult to begin recommending priorities on how to address big data without first understanding your current capabilities, which requires acknowledging areas that are successful, as well as areas that need improvements.
A working body should be established to research, document, and map out the various data activities that are currently in use, as well as understand the technology associated with them. It is imperative to be fully aware of how data is being used across an organization, and the costs associated with managing that data. The team conducting the assessment should be led by the CDO and include individual program Subject Matter Experts (SMEs), IT professionals managing the legacy technology, and individuals with experience and skills using the latest modern technology. Elements that could be included in this assessment, may be highlighting recent migration efforts to cloud platforms, and correspondingly, identifying the lack of cloud management experts that can take full advantage of what the environment has to offer. Alternately, identifying recent purchases of analytical software in relation to the lack of in-house expertise or training programs to allow staff to take full advantage of the tools. Identify your organization’s data challenges and assess how to implement a successful data program, while understanding your current data environment.
The assessment should include a realistic projection of where the organization can be in the next two years. Areas to focus on should include the organization's data needs based on the defined business metrics, the current technology in use, and the skill sets of the workforce responsible for managing and analyzing data. The CDO should lead the assessment and be responsible for briefing and receiving feedback from various governance bodies across the organization, with the intent of awareness and overall buy in.
The complete assessment of the status quo will help the organization establish priorities that will drive better use of their big data, and foster impactful decisions to their business or mission. This is also an essential factor when determining resource allocation or requesting additional funding that support big data activities.
Data Driven Culture
The underlining factor to managing and using big data well, is how easily it can be received and implemented across the enterprise. Organizations that have been in existence for a while will have established processes, procedures, and technological solutions for sharing data and information. In some cases, there may be siloed use of data or stand alone technology solutions. It is important for the CDO to not only understand the landscape of these data environments, but to convince the people managing these efforts that coming together will make everyone more successful. Leveraging the experiences of current data practitioners and leaders from various disciplines, will drive broader acceptance and consistent use of the organization’s big data environment.
Big Data is Important
Leveraging the power of big data doesn’t mean you need to control or even understand the characteristics of every data element at your disposal. Rather, it’s about being able to sift through the universe of data, figuring out what is important, extracting the data, transforming it into useful information, and ensuring that it is used in impactful ways to help your organization. You don’t need the latest technology stack to get started, but you do need a commitment to organizing and prioritizing information management as it is vital to the success of the organization.