Class: Accumulator

eclairjs.Accumulator

new Accumulator(initialValue, param, name)

A simpler value of Accumulable where the result type being accumulated is the same as the types of elements being merged, i.e. variables that are only "added" to through an associative operation and can therefore be efficiently supported in parallel. They can be used to implement counters (as in MapReduce) or sums. EclairJS supports accumulators of numeric value types. An accumulator is created from an initial value `v` by calling [[SparkContext#accumulator]]. Tasks running on the cluster can then add to it using the [[Accumulable#add]]. However, they cannot read its value. Only the driver program can read the accumulator's value, using its value method.
Parameters:
Name Type Description
initialValue number
param module:eclairjs.AccumulableParam
name string human-readable name for use in Spark's web UI
Source:
Example
var accum = sparkContext.accumulator(0);
   sparkContext.parallelize([1, 2, 3, 4])
               .foreach(function(x, accum) {
					accum.add(x);
				});
   print(accum.value()); // displays 10

Extends

  • Accumulable