16 - Apache Spark First Java Program - Create JavaSparkContext
@backstreetbrogrammer
--------------------------------------------------------------------------------
Chapter 03 - Apache Spark First Java Program - Create JavaSparkContext
--------------------------------------------------------------------------------
We can create a SparkSession object containing all the application configurations.
final var spark = SparkSession.builder()
.appName("SparkFirstProgram")
.master("local[*]")
.getOrCreate();
The appName parameter is a name for the application to show on the cluster UI.
The master is a Spark, Mesos or YARN cluster URL, or a special “local” string to run in local mode.
When running on a cluster, we will not want to hardcode master in the program, but rather launch the application with spark-submit and receive it there.
However, for local testing and unit tests, we can pass “local” to run Spark in-process.
local[*] means run Spark locally with as many worker threads as logical cores on the machine.
Create a JavaSparkContext object which tells Spark how to access a cluster, by passing the SparkContext object to its constructor
JavaSparkContext(SparkContext sc)
// Using SparkSession
final var sc = new JavaSparkContext(spark.sparkContext());
Github: https://github.com/backstreetbrogramm...
Apache Spark for Java Developers Playlist: • Apache Spark for Java Developers
Java Serialization Playlist: • Java Serialization
Dynamic Programming Playlist: • Dynamic Programming
#java #javadevelopers #javaprogramming #apachespark #spark
On this page of the site you can watch the video online 16 - Apache Spark First Java Program - Create JavaSparkContext with a duration of hours minute second in good quality, which was uploaded by the user Rishi Srivastava 01 February 2023, share the link with friends and acquaintances, this video has already been watched 2,684 times on youtube and it was liked by 12 viewers. Enjoy your viewing!
