Modeling and Analysis of Hadoop MapReduce Systems for Big Data Using Petri Nets

Figures & data

Figure 1. Flowchart of MapReduce (White 2009)

Figure 2. Basic Petri net model

Table

Algorithm: MapReduce Framework
INPUT:	The data a user wants to analyze.
OUTPUT:	The data analysis results after MapReduce.
PROCEDURE:
Step 1:	Start MapReduce program.
Step 2:	The user inputs the data.
Step 3:	The mapper begins the map function to generate <key, value> pairs.
Step 4:	According to <key, value> pairs, cut the job into many small sub-jobs, and then split to every node.
Step 5:	Part 1 and Part 2 process their sub-jobs.
Step 6:	Merge all data sets from every node to the Reducer and perform the reduce function.
Step 7:	Output the data after being reduced.

Figure 3. MapReduce Petri net model

Figure 4. Screenshot of WoPeD

Table

Algorithm: MapReduce
INPUT:	The data a user wants to analyze.
OUTPUT:	The data analysis results after MapReduce.
PROCEDURE:
Step 1:	Start MapReduce program.
Step 2:	The user inputs the data.
Step 3:	The mapper function begins the map function to generate <key, value> pairs.
Step 4:	According to <key, value> pairs, cut the job into many small sub-jobs, and then split to every node.
Step 5:	The nodes fuzzify the data.
Step 6:	The nodes analyze the data using fuzzy inference.
Step 7:	The nodes defuzzify the data.
Step 8:	Merge all data from every node to the Reducer doing reduce function.
Step 9:	Output the data after being reduced.

Figure 5. Petri net model of parallel MapReduce framework

Figure 6. Internal structure of parallel MapReduce framework

Table

Algorithm: Map
INPUT:	The data that a user wants to analyze
OUTPUT:	Separate sub-jobs distributed to the various nodes
PROCEDURE:
Step 1:	The mapper function begins the input format function to normalize the input data sets.
Step 2:	The formatted data sets are sent to the record reader to generate <key, value> pairs.
Step 3:	The partitioner divides the data sets and distributes them to the nodes in the system.

Table

Algorithm: Reduce
INPUT:	The data resulting from the Map operation
OUTPUT:	The data analysis results yielded by the Reduce function
PROCEDURE:
Step 1:	The nodes execute the shuffle function to compile the defuzzified datasets.
Step 2:	The sorting function rearranges the compiled data sets.
Step 3:	The data are sent to the reduce function for integration and simplification.
Step 4:	The reduced data are sent to the output format function for normalization.
Step 5:	The nodes execute the shuffle function to compile the defuzzified datasets.

Table

Algorithm: File Checker
INPUT:	The data that a user wants to analyze
OUTPUT:	The data check results
PROCEDURE:
Step 1:	The file checker is executed.
Step 2:	The system is searching for the uploaded data that the user wants to analyze.
Step 3:	If the data exist, then the next task begins; if not, then the system returns to its initial state.

Figure 7. File checker

Table

Algorithm: Rule Checker
INPUT:	The analysis rules input by the user
OUTPUT:	The rule analysis results
PROCEDURE:
Step 1:	The rule checker is executed.
Step 2:	Each of the rules input by the user is read in.
Step 3:	The rules are checked for dependence or contradiction.
Step 4:	If neither of these exists, then the next task begins; if dependence or contradictions exist, then the system returns to its initial state.

Figure 8. Rule checker

Table

Algorithm: Big-data analysis system with checking mechanism
INPUT:	The data that a user wants to analyze
OUTPUT:	The data analysis results after MapReduce
PROCEDURE:
Step 1:	Start MapReduce program.
Step 2:	The user inputs the data.
Step 3:	The file checker is executed.
Step 4:	The system is searching for the uploaded data that the user wants to analyze.
Step 5:	If the data exist, then the next task begins; if not, then the system returns to its initial state.
Step 6:	The rule checker is executed.
Step 7:	Each of the rules input by the user is read in.
Step 8:	The rules are checked for dependence or contradiction.
Step 9:	If neither of these exists, then the next task begins; if dependence or contradictions exist, then the system returns to its initial state.
Step 10:	The mapper function begins the input format function to normalize the input data.
Step 11:	The formatted data are sent to the record reader to generate <key, value> pairs.
Step 12:	The partitioner divides the data and distributes them to the nodes in the system.
Step 13:	The nodes fuzzify the data.
Step 14:	The nodes analyze the data using fuzzy inference.
Step 15:	The nodes defuzzify the data.
Step 16:	The nodes execute the shuffle function to compile the defuzzified data.
Step 17:	The sorting function rearranges the compiled data.
Step 18:	The data are sent to the reduce function for integration and simplification.
Step 19:	The reduced data are sent to the output format function for normalization.
Step 20:	The normalized data yield the results.

Figure 9. Big data analysis system by inclusion of checking mechanisms

Figure 10. File error

Figure 11. Rule error

Figure 12. <key, value> pairs error

Table 1. Comparison of the proposed approach with CTL and LTL

Analysis Method	Proposed Petri Nets	Computation Tree Logic (CTL)	Linear Temporal Logic (LTL)
Presentation	Visual or Mathematical Model	Mathematical Model	Mathematical Model
Ease of Simulation	Easy	Difficult	Difficult
Target of Analysis	Verification of reachability in detailed procedures of MapReduce using visual Petri net models and facilitation of tracking and error prevention for system developers	The temporal model of CTL is a dendritic structure in which future directions cannot be confirmed, and any branch could become reality.	In LTL, the future is seen as a sequence of states, so the future is viewed as a path. It is capable of using mathematical models to track the state changes of systems.

White, T. 2009. Hadoop: the definitive guide. USA: O’Reilly Media.

 Google Scholar