5 Data-Driven Steps Deeper into Your Big Data Lake

Avi Kalderon, Big Data and Analytics Practice Leader, New Vantage Partners

Big data lakes have created a lot of change, a lot of angst and a lot of opportunity. Among those opportunities is a different way of looking at – and creating – insights for transforming your business. When done correctly, you can increase the speed and agility in which you measure your performance and course-correct to adapt to the fast changing business conditions around you. Here are five ways in which you can maximize the value of your data assets.

1. Load the Data First

Instead of modeling the data first, load the data first, then model based on the content and meaning of the data that matters the most for the decision at-hand. The switch adds power, saves time and enhances your ability to understand and make faster, more focused, smarter decisions.

 You can increase the speed and agility in which you measure your performance and course-correct to adapt to the fast changing business conditions 

Breaking data barriers -- blending multiple sources, mining for new insights, analyzing sets and correlations and linking internal and external data environments to create new insights, new analytic views, and new business opportunities – maximizes the value in the shortest possible delivery time. Adopting a “Data First” approach also assesses potential cost reductions by augmenting your traditional platforms with the big data lake, resulting in less dependence on higher-cost software and platforms over time and allowing for a ‘brute-force’ approach to crunching data at efficiencies of scale never possible before. Combine that with machine learning tools and you shift the heavy lifting paradigm by letting the database do the bulk of the analysis and propose new insights to you.

2. Opportunities in storing everything

Data storage is cheaper and easier to manage on commodity hardware and open source software, freeing capital for analytics and ideation geared toward new problem solving. By not discarding or ‘filtering’ data due to performance and cost concerns, you are able to maximize the depth and breadth of your analytics and increase its accuracy.

Data discovery will be easier with more accessibledata, enable faster turnaround and user-friendly tools to manage the environment in a self-service model, decreasing the need for on-going IT support and maintenance. Data should be versioned, curated and tagged so that users can readily attach confidence levels to the data as they areusing it. “Fit-for-purpose” acceptable-use policies should ensurethat proper controls are in place so that users can mine this data for potential analysis and new insights that add value to with total control while contributing to the overall ecosystem.

On demand and real time analytics are within reach with these new processing paradigms.

3. Move the Data Warehouse to a New Neighborhood

While data warehouses are important tenets of an organization's business operations, they are falling short in delivering an agile, exploratory ideation facility for incepting new business capabilities.

In today’s environment, accounting for every question in one model is impossible. New data sources are emerging just too fast. New questions are sprouting up even faster. A highly engineered environment that only takes the data it needs upfront is going to have difficulty adapting to rapidly changing requirements.

Augmenting the data warehouse with agile analytical exploratory environments allows a business to leverage both environments successfully while mitigating cost and risk.

4. Worry Less & Execute Fast

Gone are the days where every expense needed to be justified over a 5 year depreciation cycle and achieve the approval of the board of directors. Big Data is accessible within the spending limits of most departments. Think of your data lake as an enterprise platform, find a department interested in taking the journey with you and go do. Most organizations see a very quick benefit and ROI by adopting the technology which is proven to be both cost-effective and impactful. Sometime it’s ok to break the rules, this is one of these cases.

5. Take the Best, Leave the Rest Behind

Best of Breed is back. Leverage your current investments while keeping an eye towards the future of data management. It’s big, it’s fast, it’s bold, but most of all it’s smart. You cannot afford to be left behind in the war over who has the most accurate and current information to run their business. 

Read Also

Enhanced Operational Efficiency with Datadriven IT Strategy

Tim Nall, SVP & CIO, Brown-Forman [NYSE: BF.B]

How a Risk Management Information System (RMIS) Adds Efficiency, Security, and Savings to Claims and Risk Management

Stephen J. Ackourey, Director of Risk Management Information Services, Hub International Limited