16 - Apache Spark First Java Program - Create JavaSparkContext
@backstreetbrogrammer
--------------------------------------------------------------------------------
Chapter 03 - Apache Spark First Java Program - Create JavaSparkContext
--------------------------------------------------------------------------------
We can create a SparkSession object containing all the application configurations.
final var spark = SparkSession.builder()
.appName("SparkFirstProgram")
.master("local[*]")
.getOrCreate();
The appName parameter is a name for the application to show on the cluster UI.
The master is a Spark, Mesos or YARN cluster URL, or a special “local” string to run in local mode.
When running on a cluster, we will not want to hardcode master in the program, but rather launch the application with spark-submit and receive it there.
However, for local testing and unit tests, we can pass “local” to run Spark in-process.
local[*] means run Spark locally with as many worker threads as logical cores on the machine.
Create a JavaSparkContext object which tells Spark how to access a cluster, by passing the SparkContext object to its constructor
JavaSparkContext(SparkContext sc)
// Using SparkSession
final var sc = new JavaSparkContext(spark.sparkContext());
Github: https://github.com/backstreetbrogramm...
Apache Spark for Java Developers Playlist: • Apache Spark for Java Developers
Java Serialization Playlist: • Java Serialization
Dynamic Programming Playlist: • Dynamic Programming
#java #javadevelopers #javaprogramming #apachespark #spark
На этой странице сайта вы можете посмотреть видео онлайн 16 - Apache Spark First Java Program - Create JavaSparkContext длительностью часов минут секунд в хорошем качестве, которое загрузил пользователь Rishi Srivastava 01 Февраль 2023, поделитесь ссылкой с друзьями и знакомыми, на youtube это видео уже посмотрели 2,684 раз и оно понравилось 12 зрителям. Приятного просмотра!
