16 - Apache Spark First Java Program - Create JavaSparkContext

Publicado el: 01 febrero 2023
en el canal de: Rishi Srivastava

2,684

‪@backstreetbrogrammer‬

--------------------------------------------------------------------------------
Chapter 03 - Apache Spark First Java Program - Create JavaSparkContext
--------------------------------------------------------------------------------
We can create a SparkSession object containing all the application configurations.

final var spark = SparkSession.builder()
.appName("SparkFirstProgram")
.master("local[*]")
.getOrCreate();

The appName parameter is a name for the application to show on the cluster UI.

The master is a Spark, Mesos or YARN cluster URL, or a special “local” string to run in local mode.
When running on a cluster, we will not want to hardcode master in the program, but rather launch the application with spark-submit and receive it there.
However, for local testing and unit tests, we can pass “local” to run Spark in-process.

local[*] means run Spark locally with as many worker threads as logical cores on the machine.

Create a JavaSparkContext object which tells Spark how to access a cluster, by passing the SparkContext object to its constructor

JavaSparkContext(SparkContext sc)

// Using SparkSession
final var sc = new JavaSparkContext(spark.sparkContext());

Github: https://github.com/backstreetbrogramm...

Apache Spark for Java Developers Playlist:    • Apache Spark for Java Developers
Java Serialization Playlist:    • Java Serialization
Dynamic Programming Playlist:    • Dynamic Programming

#java #javadevelopers #javaprogramming #apachespark #spark

En esta página del sitio puede ver el video en línea 16 - Apache Spark First Java Program - Create JavaSparkContext de Duración hora minuto segunda en buena calidad , que subió el usuario Rishi Srivastava 01 febrero 2023, comparta el enlace con amigos y conocidos, en youtube este video ya ha sido visto 2,684 veces y le gustó 12 a los espectadores. Disfruta viendo!