Class: Broadcast

eclairjs/broadcast. Broadcast

A broadcast variable. Broadcast variables allow the programmer to keep a read-only variable cached on each machine rather than shipping a copy of it with tasks. They can be used, for example, to give every node a copy of a large input dataset in an efficient manner. Spark also attempts to distribute broadcast variables using efficient broadcast algorithms to reduce communication cost. Broadcast variables are created from a variable `v` by calling module:eclairjs.SparkContext#broadcast. The broadcast variable is a wrapper around `v`, and its value can be accessed by calling the `value` method. The interpreter session below shows this: After the broadcast variable is created, it should be used instead of the value `v` in any functions run on the cluster so that `v` is not shipped to the nodes more than once. In addition, the object `v` should not be modified after it is broadcast in order to ensure that all nodes get the same value of the broadcast variable (e.g. if the variable is shipped to a new node later).

Constructor

new Broadcast()

Source:
Example
var b = sparkContext.broadcast([1,2]);
 var bc = b.value(); // bc = [1,2]

Methods

destroy()

Destroy all data and metadata related to this broadcast variable. Use this with caution; once a broadcast variable has been destroyed, it cannot be used again. This method blocks until destroy has completed
Source:

toString() → {string}

Source:
Returns:
Type
string

unpersist(blockingopt)

Delete cached copies of this broadcast on the executors. If the broadcast is used after this is called, it will need to be re-sent to each executor.
Parameters:
Name Type Attributes Description
blocking boolean <optional>
 Whether to block until unpersisting has completed
Source:

value() → {object}

Get the broadcasted value.
Source:
Returns:
Type
object