Introduction of MapReduce Programming Model – Hadoop Tutorial

8/23/2025

Introduction of MapReduce Programming Model

Go Back

Introduction of MapReduce Programming Model – Hadoop Tutorial

The MapReduce programming model is the core of Hadoop’s data processing framework. It provides a simple yet powerful way to process large-scale data across distributed clusters. With MapReduce, developers can break down complex data operations into smaller, manageable tasks that run in parallel, ensuring efficiency and scalability.

In this Hadoop tutorial, we will explore the basics of the MapReduce programming model, its components, workflow, and examples.

Introduction of MapReduce Programming Model

What is the MapReduce Programming Model?

MapReduce is a programming paradigm designed for processing and generating large datasets. It works by dividing tasks into two main phases:

  • Map Phase – Processes input data and transforms it into key-value pairs.

  • Reduce Phase – Aggregates, summarizes, or filters the mapped data to produce the final output.

This model helps Hadoop handle big data efficiently by running computations in parallel across multiple nodes.

Key Components of MapReduce

  1. Mapper

    • Takes input data and converts it into key-value pairs.

    • Example: Counting words in a text file.

  2. Reducer

    • Aggregates the mapper’s output.

    • Example: Summing up word counts.

  3. Driver Program

    • Controls the job execution, monitors tasks, and coordinates between Mapper and Reducer.

  4. InputSplit & RecordReader

    • Splits the input file into manageable chunks and feeds them into the mapper.

  5. OutputFormat

    • Defines how the final data is stored (e.g., text, sequence files).

MapReduce Workflow

  1. Input data is split into chunks.

  2. The Mapper processes each chunk and outputs key-value pairs.

  3. The framework shuffles and sorts data based on keys.

  4. The Reducer processes values grouped by keys.

  5. Final output is written to HDFS.

Example: Word Count Program

// Mapper Class
public static class TokenizerMapper extends Mapper<Object, Text, Text, IntWritable> {
    private final static IntWritable one = new IntWritable(1);
    private Text word = new Text();

    public void map(Object key, Text value, Context context) throws IOException, InterruptedException {
        StringTokenizer itr = new StringTokenizer(value.toString());
        while (itr.hasMoreTokens()) {
            word.set(itr.nextToken());
            context.write(word, one);
        }
    }
}

// Reducer Class
public static class IntSumReducer extends Reducer<Text, IntWritable, Text, IntWritable> {
    public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
        int sum = 0;
        for (IntWritable val : values) {
            sum += val.get();
        }
        context.write(key, new IntWritable(sum));
    }
}

Advantages of MapReduce

  • Handles large-scale data processing.

  • Provides fault tolerance via task re-execution.

  • Supports parallelism and scalability.

  • Works seamlessly with HDFS for storage.

Conclusion

The MapReduce programming model is a fundamental concept in Hadoop for processing big data in a distributed and scalable way. By understanding Mappers, Reducers, and the job execution workflow, developers can harness the power of Hadoop to perform large-scale data analytics effectively.

Table of content