Spark Modes of Deployment – Cluster mode and Client Mode

Updated:01/20/2021 by Computer Hope

A spark application consists of a driver which run either on the client or on application master node and many executors which run across slave nodes in the cluster.

"Client mode” in Apache Spark -

The behavior of the spark job depends on the “driver” component and here, the”driver” component of spark job will run on the machine from which job is submitted. Hence, this spark mode is basically called “client mode”.
You will be able to see the detailed logs in the terminal.
You will be able to terminate the job from the terminal itself.

“cluster mode” in Apache Spark

Similarly, here “driver” component of spark job will not run on the local machine from which job is submitted. Hence, this spark mode is basically “cluster mode”. In addition, here spark jobs will launch the “driver” component inside the cluster
You won't be able to see the detailed logs in the terminal.
Since driver is not created in the Master itself, you won't be able to terminate the job from the terminal


Below is screenshot of out put of Word Count program in Apache Spark

Spark Modes of Deployment – Cluster mode and Client Mode

spark Modes of Deployment – Cluster mode and Client Mode