Class: SparkSessionBuilder

eclairjs/sql. SparkSessionBuilder

SparkSessionBuilder for SparkSession.

Constructor

new SparkSessionBuilder()

Source:

Methods

appName(name) → {module:eclairjs/sql.SparkSessionBuilder}

Sets a name for the application, which will be shown in the Spark web UI. If no application name is set, a randomly generated name will be used.
Parameters:
Name Type Description
name string
Since:
  • EclairJS 0.6 Spark 2.0.0
Source:
Returns:
Type
module:eclairjs/sql.SparkSessionBuilder

config(conf, valueopt) → {module:eclairjs/sql.SparkSessionBuilder}

Sets a list of config options based on the given SparkConf. Or sets a config option. Options set using this method are automatically propagated to both SparkConf and SparkSession's own configuration.
Parameters:
Name Type Attributes Description
conf module:eclairjs.SparkConf | string or key
value boolean | number | String <optional>
Since:
  • EclairJS 0.6 Spark 2.0.0
Source:
Returns:
Type
module:eclairjs/sql.SparkSessionBuilder

enableHiveSupport() → {module:eclairjs/sql.SparkSessionBuilder}

Enables Hive support, including connectivity to a persistent Hive metastore, support for Hive serdes, and Hive user-defined functions.
Since:
  • EclairJS 0.6 Spark 2.0.0
Source:
Returns:
Type
module:eclairjs/sql.SparkSessionBuilder

getOrCreate() → {module:eclairjs/sql.SparkSession}

Gets an existing SparkSession or, if there is no existing one, creates a new one based on the options set in this SparkSessionBuilder. This method first checks whether there is a valid thread-local SparkSession, and if yes, return that one. It then checks whether there is a valid global default SparkSession, and if yes, return that one. If no valid global default SparkSession exists, the method creates a new SparkSession and assigns the newly created SparkSession as the global default. In case an existing SparkSession is returned, the config options specified in this SparkSessionBuilder will be applied to the existing SparkSession.
Since:
  • EclairJS 0.6 Spark 2.0.0
Source:
Returns:
Type
module:eclairjs/sql.SparkSession

master(master) → {module:eclairjs/sql.SparkSessionBuilder}

Sets the Spark master URL to connect to, such as "local" to run locally, "local[4]" to run locally with 4 cores, or "spark://master:7077" to run on a Spark standalone cluster.
Parameters:
Name Type Description
master string
Since:
  • EclairJS 0.6 Spark 2.0.0
Source:
Returns:
Type
module:eclairjs/sql.SparkSessionBuilder