Real-Time Data
Real-Time Data is an issue worthy of attention, because it is part of the strategic directions of many organisations. Industry reports (Forrester, Boston Retail Partners) indicate that Real-Time Data is the basis and at the same time a precondition for the development of, among others, the entire retail sector. Below, we explain what lies behind this term in theory and practice.
A brief overview of definitions
Techopedia.com defines real-time data as:
data that are passed along to the end user as quickly as they are gathered.
According to HVR (one of the manufacturers of real-time data processing solutions):
real-time data are immediately copied to one or more destinations where they will be stored and made available as soon as they are generated.
The Cambridge Dictionary indicates that real-time data are:
data that are transmitted and/or made available immediately after they are received and/or generated.
Without discussing the above definitions further, it is worth to point that each of them is based on the assumption that, once data are generated and/or saved, they are immediately processed. The purposes and contexts of such processing may differ and, importantly, they are not mutually exclusive.
Implementation of Real-Time Data in an organization
Real-Time Data is the possibility to access always up-to-date data, regardless of where and how they are processed. Implementation is usually based on solutions referred to as Real-Time Data Replication, Real-Time Data Processing, Real-Time Data Integration, e.g.HVR, Oracle GoldenGate. This class of tools enables real-time data replication (with 1–2 sec of delay) between different business and technological IT platforms (figure below, source: hvr-software.com).
An important functionality is the modification of data during replication, e.g. to compile reports, and testing of data consistency between source and target environments with the possibility of automatic correction.
In conclusion, real-time data replication mechanisms enable unlimited access to always up-to-date production data, no matter the geographical location they are generated and processed in. What does this mean for business and IT practice? What is the business and technical potential of this class of solutions?
Business context of using Real-Time Data
Let us assume that reporting of data in organisations is done following two models:
- users obtain data directly from the source systems where the data are processed
- users obtain data from reporting systems fed by data from the source systems in which the data are processed.
The former model provides users with access to always up-to-date data (provided that the real time data condition is fulfilled), but only in the context of individual systems, which is an important limitation when you need to analyse data from various sources. Also very important is the performance and stability of operation of source systems, which often experience failures in this model as a result of the burden of reporting processes, interrupting the continuity of business processes and generating business losses.
Classic centralised reporting systems are fed with data in time intervals, often daily (the latter model). They allow access to all or selected data of the organisation, which addresses the limitations of the first model, but does not provide access to real data (due to feed intervals).
The use of real-time data replication ensures that after every operation in the source system, the data are automatically moved to a “different location”, which will serve as the basis for reporting, while maintaining data consistency with the source system. One of the possible examples of such architecture is presented in the figure below (source: hvr-software.com).
This approach shortens data processing time, reduces the risk of data unavailability as a result of “heavy” failures and time-consuming processing (performed mainly at night), and provides access to always up-to-date data.
For organisations for which access to data refreshed online is not crucial, the benefit is a scalable and efficient architecture and real-time data processing mechanisms that ensure high availability and consistency of data and optimise operational processes associated with data processing. It is worth to note that the market is highly dynamic, and access to real time data may very soon prove not so much helpful for organisations which do not currently need it, but absolutely necessary from the perspective of business growth and maintaining competitiveness.
Real-time data processing enables to track business operations online, and to make decisions and initiate various types of business processes based on the current “business picture”. Not only is it convenient but also often the source of competitive advantage. A few examples:
- Retail: tracking sales and inventory at all points of sale, regardless of their geographic location, enables dynamic management of marketing campaigns – real-time marketing;
- Banking: centralisation and online processing of customer data and all customer-related operations enables to reduce the costs of customer service, increase service quality, personalise products targeted at customers in real time – real-time sales;
- Finance: online transfer of data to fraud systems reduces the level of fraud;
- Medicine: online synchronisation of medical appointment calendars enables doctors to make effective use of their working time;
- Logistics: precise monitoring of the supply chain enables its optimisation, respond quickly to incidents and limit the losses associated with them;
- Transport: real-time synchronisation of flight/route planning systems.
Technical context for using Real Time data
Data management is a challenge for most organisations, due to, among other things, rapid growth of data quantity, large amounts and diversity of data processed, geographical distribution of data processing systems and heterogeneous business and technological system infrastructure. Tools for real-time data replication are an important element of the data processing architecture. In addition to the optimisation of processing operations, they enable, among others, the following:
- performance of seamless database maintenance operations without the need to designate service windows and interrupt business processes;
- data migration between technologically different databases with minimal downtime;
- synchronisation of processing centres in the Active-Active model, thus increasing effective utilisation of the computing power of individual centres, increasing the efficiency and improving the stability of business applications and providing a secure high availability architecture;
- online feed of test environments together with data anonymisation – this problematic issue for many organisations becomes very simple to implement, manage and scale using Real-Time Data Replication mechanisms.
The diversity of environments from and to which data can be replicated is shown below (source: hvr-software.com).