16 - Apache Spark First Java Program - Create JavaSparkContext
@backstreetbrogrammer
--------------------------------------------------------------------------------
Chapter 03 - Apache Spark First Java Program - Create JavaSparkContext
--------------------------------------------------------------------------------
We can create a SparkSession object containing all the application configurations.
final var spark = SparkSession.builder()
.appName("SparkFirstProgram")
.master("local[*]")
.getOrCreate();
The appName parameter is a name for the application to show on the cluster UI.
The master is a Spark, Mesos or YARN cluster URL, or a special “local” string to run in local mode.
When running on a cluster, we will not want to hardcode master in the program, but rather launch the application with spark-submit and receive it there.
However, for local testing and unit tests, we can pass “local” to run Spark in-process.
local[*] means run Spark locally with as many worker threads as logical cores on the machine.
Create a JavaSparkContext object which tells Spark how to access a cluster, by passing the SparkContext object to its constructor
JavaSparkContext(SparkContext sc)
// Using SparkSession
final var sc = new JavaSparkContext(spark.sparkContext());
Github: https://github.com/backstreetbrogramm...
Apache Spark for Java Developers Playlist: • Apache Spark for Java Developers
Java Serialization Playlist: • Java Serialization
Dynamic Programming Playlist: • Dynamic Programming
#java #javadevelopers #javaprogramming #apachespark #spark
In questa pagina del sito puoi guardare il video online 16 - Apache Spark First Java Program - Create JavaSparkContext della durata di ore minuti seconda in buona qualità , che l'utente ha caricato Rishi Srivastava 01 febbraio 2023, condividi il link con amici e conoscenti, su youtube questo video è già stato visto 2,684 volte e gli è piaciuto 12 spettatori. Buona visione!
