What a Japanese Auto Maker Can Teach Us about Big Data ROI
Big data is big and getting bigger. Emerging technologies like mobility, social media, and the Internet of Things are generating volumes of data faster than most firms can capture and process them. However, that isn’t stopping companies from trying. A recent IDC forecast projects big data spending to follow a 23 percent compounded annual growth curve culminating in a projected $48.6 billion global expenditure by 2019.
“Big data is becoming bigger and richer by the day”
Capturing big data is the easy part. Analyzing, processing, and gaining significant insights from the digital tsunami presents many barriers that stand between data-centric firms and the business insights they seek. Dissimilar data structures complicate processing workflows. The global talent shortage makes finding qualified analytics professionals difficult and expensive. And massive data volumes make marshaling and Extract, Transform and Load (ETL) as awkward as moving a waterbed mattress.
Like all emerging technologies, big data suffers from low levels of common knowledge and high levels of expectation. As a result, stakeholders increase pressure on CIOs to produce eye-popping returns while keeping costs within their definition of “reasonable.” But if IDC’s projections are accurate, they’ll be fortunate to bring the price tag anywhere south of “astronomical.”
Is big data ROI just an expensive mirage? Or are we just not doing it right?
Applying Timeless Principles to New Data Disciplines
In his landmark book, Lean Thinking, James Womack illustrates how Toyota propelled itself to a position of market leadership in the automobile industry by systematically eliminating muda, the Japanese word for waste. Womack’s answer to muda is lean thinking, the process of defining and implementing activities that provide customer-centric value while removing processes that only consume time and resources.
In the big data world, one of the most glaring examples of muda is the process of data modeling. Before captured data can be turned into predictive analytics, skilled data professionals must prepare a transformation process to bring raw data into a structured form so that analytical tools can work their magic.
But big data is becoming bigger and richer by the day. New data sources continue to pop up as existing ones evolve, complicating the modeling effort and further delaying time to value. It also doesn’t help that only a handful of data scientists on the planet are experienced enough to perform this manual process effectively. By the time the business intelligence tools can produce any meaningful insights, there’s a good chance the time to take action on them has already passed.
Lean Thinking to Remove Big Muda from Big Data
Manual data modeling can be enormously effective for firms that can afford the necessary analytical talent. But innate levels of muda within the process make it cost prohibitive for all but the largest of corporations. Throwing more hardware and people at the problem clearly isn’t the answer.
In decades past, lean practitioners would have classified big data modeling as “Type One Muda” — a process that produces no direct customer value but is nonetheless required to fulfill production systems. In an environment of modern machine learning technology, however, perhaps this human-centric workflow is not as necessary as it appears.
Could we apply a lean principle of value stream enhancement to shore up the inefficiencies in big data modeling? And could we leverage our powerful computing platforms to address this and other analysis-heavy big data process points?
Software Defined Potential for Lean Outcomes
Many firms have already leveraged the benefits of software defined architecture to shorten the time and effort required to provision IT assets. Data-centric firms could benefit from the same techniques and principles to eliminate muda from the big data analysis process.
Some business intelligence vendors have begun to develop marketing analysis packages featuring cognitive computing technology to address this very issue. By emulating the human thought process, they’ve programmed smart algorithms to recognize existing data patterns, evaluate new unstructured sources, and calculate the least costly route to an effective big data model suitable for predictive analytics.
Is the net result as effective as those produced by a platoon of highly trained data scientists? Probably not, but it could most likely produce some actionable insight quickly enough to generate significant ROI at a lower cost. By slashing the time, and effort of today’s manual methods, a software defined solution would decrease analysis costs and bring SMBs one step closer to big data insights that elude them today.
Comparatively speaking, big data is still in its infancy, so all the muda within the current value chain should come as no surprise. Simplifying the modeling cycle is only one example of applying lean thought to eliminate inefficiencies. Until IT settles on a generally accepted set of best practices, there will be many other opportunities to eliminate waste and generate actionable business insights more efficiently.
By applying lean methods to new data disciplines like big data, big firms can significantly reduce costs, smaller firms can gain greater business insight, and everyone can enjoy significantly higher return on big data investment.