Integrating SOA for Handling Big-Data

By CIOReview | Friday, March 17, 2017

Understanding the Handling of Big Data

Big data analytics is the process of examining large amounts of unstructured data to discover hidden patterns and unknown correlations which helps for business intelligence in current era so it can help in better decision making. The runtime analysis and debugging of such data cannot be addressed by traditional monitoring and debugging approaches. To maximize and manage big data, organizations are widely adopting Software Oriented Architecture (SOA) middleware technologies.

The intended outcome of SOA adoption is to improve interoperability both internal and external data, realizing cost savings over time by adopting reusable and open standard IT services, and to align IT services closely with services that the organization provides to its ecosystem. The primary concern while implementing SOA is whether the company needs data-centric SOA or SOA-centric data , heavily influenced by the company’s data management policies.

SOA and Integration Models

Recently, the focus of organizations has shifted from data to SOA. Juggling SOA-data relationship to optimally manage models of big data, cloud data and data hierarchies is one of SOA's most profound challenges. The integration of data plays a crucial role in condition monitoring and active data warehousing. It is classified in horizontal and vertical integration capturing different integration scenarios. Horizontal scenarios deal with application providing complementary functionality, whereas vertical scenarios deal with integrating applications on different abstraction levels.

Horizontal and Vertical Integration for Managing Big Data

The vertically integrated data model links application data services to resources in a more application-specific way, where the customer relationship management or enterprise resource planning or dynamic data authentication application data is separated first at the as-a-Service level. To provide more uniform data integrity and management, management services may be provided as SOA components that operate on various database systems, performing tasks such as deduplication and integrity checks, in database-specific ways. This approach is easily adaptable, but risks compromising SOA as-a-Service principles in how data is accessed, and it may also create issues with consistency in data management.

The application components are largely insulated from data management differences of RDBMS versus big data. The horizontal model is more consistent with SOA principles, since it abstracts data services from SOA components more thoroughly. To make it work, it is necessary to define abstractions for non-relational databases and to deal with any inefficiencies associated with the abstraction process.  The application components are largely insulated from data management differences of RDBMS versus big data. While this approach can't create the simple query model of RDBMS for the reasons already given, it at least replicates the simple model of RDBMS that we presented above.

SOA's three data-centric models:

Data as a Service (DaaS):

Representing how data is made available to SOA components. DaaS depends on the principle that specified, useful data can be supplied to users on demand, irrespective of any organizational or geographical separation between consumers and providers, eliminating redundancy and associated expenditures, allowing data use and/or modification by multiple users via a single update point.

Physical Hierarchies:

Determining how storage and storage hierarchies figure into SOA data access. The physical hierarchy will be decided Depending on the type of big data that is meant to be processed.

Storage Hierarchies:

It relates to Identifying the way that data, data management services and SOA components relate. The size and the value of big data will determine the type of storage by automatically moving data between high-cost and low-cost storage media.

Architecting for a ‘Big Data’ future

Efficiency issues are complex. One essential step is to ensure the overhead associated with this orchestration is minimal as horizontal database model is likely to be implemented through a message service bus like most SOA processes. Big data problems are not the sole domain of multi-national companies as even small to medium size organizations aggregate data, making sense of that data that both scales and meets modern performance demands requires a big-data approach.

With big data at play, a company's net income has more to do with enterprise architecture than ever before. By finding efficient ways to leverage back-end data assets in order to achieve business goals, enterprise architects can ensure that the business case remains clear throughout big data technology efforts.