Class: Accumulator

eclairjs. Accumulator

new Accumulator(initialValue, param, name)

A simpler value of Accumulable where the result type being accumulated is the same as the types of elements being merged, i.e. variables that are only "added" to through an associative operation and can therefore be efficiently supported in parallel. They can be used to implement counters (as in MapReduce) or sums. EclairJS supports accumulators of numeric value types. An accumulator is created from an initial value `v` by calling [[SparkContext#accumulator]]. Tasks running on the cluster can then add to it using the [[Accumulable#add]]. However, they cannot read its value. Only the driver program can read the accumulator's value, using its value method.
Parameters:
Name Type Description
initialValue number
param module:eclairjs.AccumulableParam
name string human-readable name for use in Spark's web UI
Source:
Example
var accum = sparkContext.accumulator(0);
   sparkContext.parallelize([1, 2, 3, 4])
               .foreach(function(x, accum) {
					accum.add(x);
				});
   print(accum.value()); // displays 10

Extends

Methods

add(term)

Add more data to this accumulator / accumulable
Parameters:
Name Type Description
term object the data to add
Inherited From:
Source:

localValue() → {object}

Get the current value of this accumulator from within a task. This is NOT the global value of the accumulator. To get the global value after a completed operation on the dataset, call `value`. The typical use of this method is to directly mutate the local value, eg., to add an element to a Set.
Inherited From:
Source:
Returns:
Type
object

merge(term)

Merge two accumulable objects together Normally, a user will not want to use this version, but will instead call `add`.
Parameters:
Name Type Description
term object the other `R` that will get merged with this
Inherited From:
Source:

setValue(newValue)

Set the accumulator's value; only allowed on master
Parameters:
Name Type Description
newValue object
Inherited From:
Source:

toString() → {string}

Inherited From:
Source:
Returns:
Type
string

value() → {object}

Access the accumulator's current value; only allowed on master.
Inherited From:
Source:
Returns:
Type
object