Technology to Mitigate the Rising Big Data Cost
Adoption for Open Data Sources
The significant costs of using Big Data—in analytics applications—are in the technical (information technology and data science) and industry/business function subject matter expertise. These costs are rising because demand is very high and supply is low. Technology can mitigate the rising cost of talented personnel as it has in the past: the unit cost of storage continues to decrease and the competition for Big Data/analytics market share drives lower software prices for data integration and ‘on demand’ hosting services for large data volumes. The increasing availability and adoption of ‘open data’ sources, companies can incorporate new sources of data and ideas for free where, historically, data had to be purchased.
Promise of a ‘Step Function’
The combination of machine learning and robotic process automation has the promise of a ‘step function’ improvement in lower-level task automation that can extract significant efficiency benefits across industries. These technologies go beyond the process automation we’ve grown accustomed to through the implementation of process design efficiencies combined with traditional transaction processing software solutions.
Further, autonomous vehicles, drones, augmented/virtual all have the promise to drastically change the way we work in the office and in the field. Importantly for our employees, these technologies can improve safety related to many activities.
The combination of machine learning and robotic process automation has the promise of a ‘step function’ improvement in lower-level task automation
Big Data’s Role
Quietly and with increasing frequency at a consumer level, I’ve grown increasingly accustomed to real-time applications (like Uber) to support daily activities. The volume of data and application response time of smartphone applications makes me more productive at home and at work. In the context of my role at AEP, I can now ask questions without concern for the complexity and volume of data required to answer them. The rapid progress in cognitive computing (that leverages big data) combined with the mass market availability of effective voice interactions will drastically change how we interact with software services (from creating a simple shopping list to automated home control systems that learn your preferences.)
Focus on Business Value
The most important skill remains an appreciation and focus on business value returned by any technology investment. It is also critical for technology leaders to use logical frameworks that help explain technology evolutions.
Leaders need to understand how to orchestrate and leverage combinations of existing, stand-along services into solutions. The technology leader, then, will be an expert in building partnerships with multiple, independent parties.
Our long-term objective is to find any people, process and technology savings to reduce ‘Run’ costs so we can reallocate funds to ‘Grow’ and ‘Transform’ opportunities. Increasingly, we need to work closely with our partners in the operations functions to return business value quickly. A strong relationship between the business partner and IT expert always returns the best results—so we work hard to foster these relationships. We continue to evaluate our IT and operations investment prioritization and project management disciplines to ensure we are applying “just enough” structure and governance to accelerate the ‘demand to delivery’ value stream, manage well and maintain quality results. Further, an emphasis on reducing technology complexity—from the data center to the desktop—makes technology more accessible. Finally, we continually evaluate our investment priorities to ensure we’re funding technology enablers that reduce operating costs or drive competitive differentiation.
Proficiency in Agile
The agile (or even DevOps) development and deployment models are well understood but not necessarily easy to execute consistently. We are working to improve our proficiency in agile as we’ve found our business partners are far more satisfied with this development approach. One challenge with agile is to ensure each project is built within the guidelines of an overall software architecture. Agile has some built-in mechanisms to address this requirement, such as “refactoring” or additional sprints to address “technical debt.” Another obstacle can be lack of understanding or experience with agile with IT or business partners. If agile is not executed well, or if the desire for faster outcomes compromises the foundational disciplines of information technology (e.g., cybersecurity, information management, technology lifecycle management, etc.), organizations will be hindered by a more complex and costly technology ecosystem in the long run.
Evolution of ‘mode 2’
Well, we do still measure our teams on project deadlines… certain fundamentals always apply… As discussed above, agile development is one technique. While agile development is not inherently faster, the constant engagement of the business partner removes their anxiety over the likely success of the final deliverables that sometimes creeps into traditional, months-long waterfall projects. The use of scrum techniques and frequent reviews of backlogs have ensured that the most important work is getting done first. Additionally, the application of lean management techniques for continuous improvement and problem solving has encouraged our ‘front line’ to identify and resolve any bottlenecks and issues. We have also adopted Gartner’s ‘mode 2’ approach to facilitate piloting emerging technologies—from hardware provisioning measured in minutes, not weeks, to application component iterations delivered in days, not months. Successful ‘mode 2’ execution relies on standard platform configurations– so we are working with our business partners to define a limited set. The next step in the evolution of ‘mode 2,’ is to enable our ‘citizen developers’ within a controlled set of technologies and data sets. Finally, we are implementing tools to automate the development, testing and deployment of software. These tools are radically accelerating implementation timeframes while improving the quality of our work products.