Big Data Sources
The following sources describe the list of data stored in the databases (Figure 1.3).
Media includes social media and intuitive stages, similar to Google, Facebook, Twitter, YouTube, Instagram, just as conventional media like pictures, recordings, sounds, and webcasts that give quantitative and subjective bits of knowledge on each part of client association.
1.7.2 Business Data
The data that are captured by an organization are considered as an asset which is used for taking any crucial decisions, planning strategy, and all associated operations.
Figure 1.3 Big data sources.
22.214.171.124 Customer’s Details
Name, age, mobile number, address, and all other basic details of its customers are needed for any organization for its effective running.
126.96.36.199 Transaction Details
Here the transaction details whenever a customer purchases can be noted and used for any future notifications that can be sent to the customer to increase the sale.
Here the records such as the total number of visits by the customer; number of hits in a day; and all other interactions among the customers, stakeholders, support agents and others will be created so that on any occasion, the data can be used to take precise decisions.
1.7.3 IoT Data
Industries are using different types of sensors already, and the inclusion of the internet has led to many technological developments and reduced the workload of human beings in all aspects of the society. The IoT environment delivers intelligence using various sensors such as humidity, proximity, pressure, water quality, level, gas, IR, ultrasonic, motion detection, accelerometer, gyroscope, and optical sensors. It captures the data continuously even if there are a lot of data that are associated with multimedia such as images, videos, and so on.
The sensor data that are stored in the cloud and will increase in size can be called as big data. These sensors share the collected data with available connected devices and make it smarter for humankind, increasing functionality and effectiveness. An automated car is an example, where all the sensors collect the data from the environment and then send them the database; these data are then sent to all other vehicles.
Big Data System Components
Big data and analytics solutions usually comprise five system functions namely (Govindan et al. 2018)
- 1. Data acquisition
- 2. Data retention 3- Data transport
- 4. Data processing
- 5. Data leverage (Figure 1.4).
Figure 1.4 Big data system components.
1.8.1 Data Acquisition (DAQ)
It is the process of converting the physical entities into digital means, so that they can be accessed and retrieved whenever the user needs them in the future. It involves getting the data and removing the bugs or any other entities before transferring to the data warehouse. It depends on four main V’s such as volume, variety, velocity, and value (Deshpande et al. 2004). It has been used in all places where data are recorded from the environment.
It does three operations such as
1.8.2 Data Retention
It is the process in which any decisions can be taken from the available data, whenever any crucial situation occurs in industry, which is collected by the data acquisition method. Data retention policies consist of security and official or any legal worries against economics to find data formats, cryptography methods, retrieval time, and archival rules (Boyd and Crawdford 2012).
Data retention policies are the standards that are established to follow the norms of retrieving the data in the system. It makes the call of deleting any data which will be no longer required in the system and prioritizing data according to its importance.
1.8.3 Data Transportation
The data that are collected has to be transported to many servers to improve business continuity, load balancing, and replication. It is the process of transmitting data from one location to the other physically or logically. It is the reason for delaying of data or loss of data as well as the speed of transmission of data. This layer is used to ensure whether data has been sent to the destination without any packet loss from the other end.
1.8.4 Data Processing
Processing of data is another mammoth task as the structure of the data is not defined in big data management system. It is the process of collecting and manipulating the data items for useful decision-making in any aspect of the system. It comes under information processing.
The steps that are carried out are
1.8.5 Data Leverage
It is the process of increasing the revenue of the organization by using the existing data in the system. In big data analytics, it uses large number of data files. As a whole, these data are huge in volume and cannot be used meaningfully, but here it can be used to predict and take any decision on any crucial situation.
There are three classifications of data leveraging: