16 - Apache Spark First Java Program - Create JavaSparkContext

Publicado em: 01 Fevereiro 2023
no canal de: Rishi Srivastava

2,684

‪@backstreetbrogrammer‬

--------------------------------------------------------------------------------
Chapter 03 - Apache Spark First Java Program - Create JavaSparkContext
--------------------------------------------------------------------------------
We can create a SparkSession object containing all the application configurations.

final var spark = SparkSession.builder()
.appName("SparkFirstProgram")
.master("local[*]")
.getOrCreate();

The appName parameter is a name for the application to show on the cluster UI.

The master is a Spark, Mesos or YARN cluster URL, or a special “local” string to run in local mode.
When running on a cluster, we will not want to hardcode master in the program, but rather launch the application with spark-submit and receive it there.
However, for local testing and unit tests, we can pass “local” to run Spark in-process.

local[*] means run Spark locally with as many worker threads as logical cores on the machine.

Create a JavaSparkContext object which tells Spark how to access a cluster, by passing the SparkContext object to its constructor

JavaSparkContext(SparkContext sc)

// Using SparkSession
final var sc = new JavaSparkContext(spark.sparkContext());

Github: https://github.com/backstreetbrogramm...

Apache Spark for Java Developers Playlist:    • Apache Spark for Java Developers
Java Serialization Playlist:    • Java Serialization
Dynamic Programming Playlist:    • Dynamic Programming

#java #javadevelopers #javaprogramming #apachespark #spark

Nesta página do site você pode assistir ao vídeo on-line 16 - Apache Spark First Java Program - Create JavaSparkContext duração hora minuto segundo em boa qualidade , que foi baixado pelo usuário Rishi Srivastava 01 Fevereiro 2023, compartilhe o link com seus amigos e conhecidos, no youtube este vídeo já foi visto 2,684 vezes e gostou 12 espectadores. Boa visualização!

1,090