Big Data Analytics: Effectiveness, Ethics, and Emerging Trends

Dr. Satyam Priyadarshy, Chief Data Scientist, Halliburton

Big Data is the force for digital transformation of businesses. Many definitions of the Big Data exist in the literature, and the most relevant ones have one thing in common, that Big Data is not just about lots of data. For business context, the definition of Big Data that is most relevant and appropriate is given here. Big Data is about creating value from all the data, by leveraging emerging technologies, and full analytics value, to uncover hidden inefficiencies and take actionable insights, to grow the business through a continuous data-driven innovation. The value creation from Big Data requires Big Data Analytics and is the process of examining all data related to the business to uncover hidden patterns, correlations, market trends, and other information that would be useful for companies to be competitive and innovative. Big Data analytics is becoming increasingly prevalent in many industries. Depending upon the industry, Big Data analytics allows for the following: efficient and effective medical care, optimization of machinery, increased performance of devices, financial trading, personalized customer service, personalized shopping experience, improving law enforcement, and effective eGovernment, etc. Many of these areas include a significant amount of personal data. The Internet, marketing, retail and consumer industries have seen significant value creation through Big Data analytics. There are industries like oil and gas, healthcare, and education that are in early stages of their Big Data journey.

Understanding the role of Big Data analytics in oil and gas industry is most challenging compared to any other industry. Recent reports suggest that the oil and gas industry uses only 1 percent of the data they generate, thus providing an opportunity to create economic value from the data. Big data analytics has been proven to increase production and effectiveness of oil drilling from six to eight percent (Bain, 2014). Big data analytics can also capture more accurate real-time data, improve plant performance and reduce oil-field costs. Big data analytics can allow an advantage in geology interpretation, new oil well delivery, and oil well and field optimization.

  ​Big Data analytics will continue to grow given the increased adoption of emerging technologies like Internet of Things, Augmented Reality and Virtual Reality, etc.  

Improving healthcare and public health is also where big data analytics is highly effective. The computing power of big data analytics can enable the decoding of entire DNA strings in minutes, which will allow to find cures and predict disease patterns. Big data analytics also plays a role in monitoring and predicting the developments of disease outbreaks and epidemics (Marr, 2016). Alongside that, the personal quantification and optimization come in with wearable health technology such as fitness trackers. Big data analytics will give consumers real-time data that will enable them to best suit their personal health care needs.

Learning Big Data analytics is also growing as new platforms of education and learning have become a source of gaining knowledge and interaction data. Learning Big Data allows for any user or company to use these large data sources and personalize and optimize content for their learner and students. The traditional methods of course and teacher performance are quite ineffective and provide very limited insight. The availability of real-time data from learning platforms combined with traditional performance review becomes a 360-degree view of learning and education effectiveness.

As discussed so many industries are taking advantage of Big Data analytics and a treasure trove of accessible data including massive amounts of personal data on the web through direct or indirect access to bring about various automated to semi-automated sensitive decisions that impact the social fabric of the community, nation and the world. These decisions can have a major impact on an individual’s life. For example, Big Data credit scoring algorithm processes over 20000 constantly evolving data points, including a person’s online, mobile, and browsing behavior. Such use of a large number of data points and impact on social aspects of life requires an understanding of Ethics of Big Data analytics.Kavita N. Priyadarshy, Founder & CEO, Sahas LLC

Some of the ethical implications that may come with big data analytics is that these large chunks of data could accidentally be put in the wrong hands and cause someone to retrieve personal information about others, as well as commit different forms of fraud. While the implication is quite risky, it can be easily managed and secured. Some areas where big data is used have its benefits but also have its implications. One example is in law enforcement. Machine learning can allow law enforcement to identify robberies and access phone records to identify suspects in a crime; however, the implication of big data analytics in law enforcement can be the accessing of smartphones without a warrant or identifying suspects by web browsing habits. (Martin, 2015). In the education sector, big data analytics can be beneficial as it may individualize student instruction, take accountability for students in school, and identify students who are at risk for dropping out; however, the implications of it can allow the data to be used for possible admissions discrimination.           

Big Data technologies while providing new challenges for legal and ethical areas are also providing solutions for addressing the same challenges they create. In simplest term, one can mine the data in real-time and see if anyone is abusing the data or the information associated with it. These technologies are used by law enforcement agencies and nation states to safeguard the interest of their citizens. To a smaller degree, these solutions can be employed by industries to protect their assets and people.

Despite the challenges, the adoption of Big Data analytics will continue to increase across all industries, as we believe that Big Data analytics is the force behind digital transformation of all industries in the current global economic environment. In the near future, we should see the extensive use of personalized and precision medicine for treatment of patients, primarily due to increased adoption of Big Data analytics. In the realm of education big data analytics can be highly effective in curating lesson plans based on each student’s learning behaviors. There are three types of learners: auditory, visual, and kinesthetic. Using big data analytics, educators can ensure that every student’s learning experience is based on a personalized method to allow them to grasp complex concepts.

In summary, Big Data analytics continues to be effective in many industries and areas of life, but by no means, its full potential has been exploited by any industry. Big Data analytics will continue to grow given the increased adoption of emerging technologies like Internet of Things, Augmented Reality and Virtual Reality, etc.