Class: functions

eclairjs/sql. functions

Spark SQL functions.

Constructor

new functions()

Source:

Methods

(static) abs(column) → {module:eclairjs/sql.Column}

Computes the absolute value.
Parameters:
Name Type Description
column module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) acos(col) → {module:eclairjs/sql.Column}

Computes the cosine inverse of the given value; the returned angle is in the range 0.0 through pi.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) add_months(startDate, numMonths) → {module:eclairjs/sql.Column}

Returns the date that is numMonths after startDate.
Parameters:
Name Type Description
startDate module:eclairjs/sql.Column
numMonths integer
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) approxCountDistinct(column, rsdopt) → {module:eclairjs/sql.Column}

Aggregate function: returns the approximate number of distinct items in a group.
Parameters:
Name Type Attributes Description
column module:eclairjs/sql.Column | string object or column name as a string
rsd float <optional>
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) array(columnExpr,) → {module:eclairjs/sql.Column}

Creates a new array column. The input columns must all have the same data type.
Parameters:
Name Type Description
columnExpr, module:eclairjs/sql.Column | string ...columnExpr or columnName, ...columnName
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) array_contains(column, value) → {module:eclairjs/sql.Column}

Returns true if the array contain the value
Parameters:
Name Type Description
column module:eclairjs/sql.Column
value string | number
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) asc(columnName) → {module:eclairjs/sql.Column}

Returns a sort expression based on ascending order of the column.
Parameters:
Name Type Description
columnName string
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column
Example
// Sort by dept in ascending order, and then age in descending order.
  df.sort(functions.asc("dept"), functions.desc("age"))

(static) ascii(e) → {module:eclairjs/sql.Column}

Computes the numeric value of the first character of the string column, and returns the result as a int column.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) asin(col) → {module:eclairjs/sql.Column}

Computes the sine inverse of the given value; the returned angle is in the range -pi/2 through pi/2.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) atan(col) → {module:eclairjs/sql.Column}

Computes the tangent inverse of the given value.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) atan2(left, right) → {module:eclairjs/sql.Column}

Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
Parameters:
Name Type Description
left module:eclairjs/sql.Column | string | float
right module:eclairjs/sql.Column | string | float
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column
Example
var col1 = new Column("age");
var col2 = new Column("expense");
var result = functions.atan2(col1, col2);
 // or
 result = functions.atan2(col1, "name");
 // or
 result = functions.atan2("age", col2);
 // or
 result = functions.atan2("age", "expense");
 // or
 result = functions.atan2(col1, 2.0);
 // or
 result = functions.atan2("age", 2.0);
 // or
 result = functions.atan2(2.0, col2);
 // or
 result = functions.atan2(2.0, "expense");

(static) avg(column) → {module:eclairjs/sql.Column}

Aggregate function: returns the average of the values in a group.
Parameters:
Name Type Description
column module:eclairjs/sql.Column | string object or column name as a string
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) base64(e) → {module:eclairjs/sql.Column}

Computes the BASE64 encoding of a binary column and returns it as a string column. This is the reverse of unbase64.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) bin(col) → {module:eclairjs/sql.Column}

An expression that returns the string representation of the binary value of the given long column. For example, bin("12") returns "1100".
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) bitwiseNOT(e) → {module:eclairjs/sql.Column}

Computes bitwise NOT.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) broadcast(df) → {module:eclairjs/sql.DataFrame}

Marks a DataFrame as small enough for use in broadcast joins. The following example marks the right DataFrame for broadcast hash join using `joinKey`.
Parameters:
Name Type Description
df module:eclairjs/sql.DataFrame
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.DataFrame
Example
// left and right are DataFrames
  left.join(broadcast(right), "joinKey")

(static) callUDF(udfName, cols) → {module:eclairjs/sql.Column}

Call an user-defined function. Example:
Parameters:
Name Type Description
udfName string
cols Array.<module:eclairjs/sql.Column>
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column
Example
var Column = require(EclairJS_Globals.NAMESPACE + '/sql/Column');
var functions = require(EclairJS_Globals.NAMESPACE + '/sql/functions');

sqlContext.udf().register("udfTest", function(col1, col2) {
         return col1 + col2;
 }, DataTypes.StringType);

 var result = df.select(functions.callUDF("udfTest", [new Column("col1"), new Column("col2")]));

(static) cbrt(col) → {module:eclairjs/sql.Column}

Computes the cube-root of the given value.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) ceil(col) → {module:eclairjs/sql.Column}

Computes the ceiling of the given value.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) coalesce(column,) → {module:eclairjs/sql.Column}

Returns the first column that is not null, or null if all inputs are null. For example, `coalesce(a, b, c)` will return a if a is not null, or b if a is null and b is not null, or c if both a and b are null but c is not null.
Parameters:
Name Type Description
column, module:eclairjs/sql.Column ...column
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) col(colName) → {module:eclairjs/sql.Column}

Returns a Column based on the given column name.
Parameters:
Name Type Description
colName string
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) column(colName) → {module:eclairjs/sql.Column}

Returns a [[Column]] based on the given column name. Alias of col.
Parameters:
Name Type Description
colName string
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) concat(columnExpr,) → {module:eclairjs/sql.Column}

Concatenates multiple input string columns together into a single string column.
Parameters:
Name Type Description
columnExpr, module:eclairjs/sql.Column | string ...columnExpr or columnName, ...columnName
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) concat_ws(sep, columnExpr,) → {module:eclairjs/sql.Column}

Concatenates multiple input string columns together into a single string column, using the given separator.
Parameters:
Name Type Description
sep string
columnExpr, module:eclairjs/sql.Column | string ...columnExpr or columnName, ...columnName
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) conv(num, fromBase) → {module:eclairjs/sql.Column}

Convert a number in a string column from one base to another.
Parameters:
Name Type Description
num module:eclairjs/sql.Column
fromBase integer * @param {integer} toBase
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) cos(col) → {module:eclairjs/sql.Column}

Computes the cosine of the given value.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) cosh(col) → {module:eclairjs/sql.Column}

Computes the hyperbolic cosine of the given value.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) count(column) → {module:eclairjs/sql.Column}

Aggregate function: returns the number of items in a group.
Parameters:
Name Type Description
column module:eclairjs/sql.Column | string object or column name as a string
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) countDistinct(columnExpr,) → {module:eclairjs/sql.Column}

Aggregate function: returns the number of distinct items in a group.
Parameters:
Name Type Description
columnExpr, module:eclairjs/sql.Column | string ...columnExpr or columnName, ...columnName
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) crc32(e) → {module:eclairjs/sql.Column}

Calculates the cyclic redundancy check value (CRC32) of a binary column and returns the value as a bigint.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) cumeDist() → {module:eclairjs/sql.Column}

Window function: returns the cumulative distribution of values within a window partition, i.e. the fraction of rows that are below the current row.
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column
Example
N = total number of rows in the partition
  cumeDist(x) = number of values before (and including) x / N



This is equivalent to the CUME_DIST function in SQL.

(static) current_date() → {module:eclairjs/sql.Column}

Returns the current date as a date column.
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) current_timestamp() → {module:eclairjs/sql.Column}

Returns the current timestamp as a timestamp column.
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) date_add(start, days) → {module:eclairjs/sql.Column}

Returns the date that is `days` days after `start`
Parameters:
Name Type Description
start module:eclairjs/sql.Column
days integer
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) date_format(dateExpr, format) → {module:eclairjs/sql.Column}

Converts a date/timestamp/string to a value of string in the format specified by the date format given by the second argument. A pattern could be for instance `dd.MM.yyyy` and could return a string like '18.03.1993'. All pattern letters of SimpleDateFormat can be used. NOTE: Use when ever possible specialized functions like year. These benefit from a specialized implementation.
Parameters:
Name Type Description
dateExpr module:eclairjs/sql.Column
format string
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) date_sub(start, days) → {module:eclairjs/sql.Column}

Returns the date that is `days` days before `start`
Parameters:
Name Type Description
start module:eclairjs/sql.Column
days integer
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) datediff(end, start) → {module:eclairjs/sql.Column}

Returns the number of days from `start` to `end`.
Parameters:
Name Type Description
end module:eclairjs/sql.Column
start module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) dayofmonth(e) → {module:eclairjs/sql.Column}

Extracts the day of the month as an integer from a given date/timestamp/string.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) dayofyear(e) → {module:eclairjs/sql.Column}

Extracts the day of the year as an integer from a given date/timestamp/string.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) decode(value, charset) → {module:eclairjs/sql.Column}

Computes the first argument into a string from a binary using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16'). If either argument is null, the result will also be null.
Parameters:
Name Type Description
value module:eclairjs/sql.Column
charset string one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16'
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) denseRank() → {module:eclairjs/sql.Column}

Window function: returns the rank of rows within a window partition, without any gaps. The difference between rank and denseRank is that denseRank leaves no gaps in ranking sequence when there are ties. That is, if you were ranking a competition using denseRank and had three people tie for second place, you would say that all three were in second place and that the next person came in third. This is equivalent to the DENSE_RANK function in SQL.
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) desc(columnName) → {module:eclairjs/sql.Column}

Returns a sort expression based on the descending order of the column.
Parameters:
Name Type Description
columnName string
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column
Example
// Sort by dept in ascending order, and then age in descending order.
  df.sort(functions.asc("dept"), functions.desc("age"))

(static) encode(value, charset) → {module:eclairjs/sql.Column}

Computes the first argument into a binary from a string using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16'). If either argument is null, the result will also be null.
Parameters:
Name Type Description
value module:eclairjs/sql.Column
charset string one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16'
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) exp(col) → {module:eclairjs/sql.Column}

Computes the exponential of the given value.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) explode(e) → {module:eclairjs/sql.Column}

Creates a new row for each element in the given array or map column.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) expm1(col) → {module:eclairjs/sql.Column}

Computes the exponential of the given value minus one.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) expr(expr) → {module:eclairjs/sql.Column}

Parses the expression string into the column that it represents, similar to DataFrame.selectExpr
Parameters:
Name Type Description
expr string
Source:
Returns:
Type
module:eclairjs/sql.Column
Example
// get the number of words of each length
  df.groupBy(functions.expr("length(word)")).count()

(static) factorial(e) → {module:eclairjs/sql.Column}

Computes the factorial of the given value.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) first(column) → {module:eclairjs/sql.Column}

Aggregate function: returns the first value in a group.
Parameters:
Name Type Description
column module:eclairjs/sql.Column | string object or column name as a string
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) floor(col) → {module:eclairjs/sql.Column}

Computes the floor of the given value.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) format_number(x, d) → {module:eclairjs/sql.Column}

Formats numeric column x to a format like '#,###,###.##', rounded to d decimal places, and returns the result as a string column. If d is 0, the result has no decimal point or fractional part. If d < 0, the result will be null.
Parameters:
Name Type Description
x module:eclairjs/sql.Column
d integer rounded to d decimal places
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) format_string(format,, columnExpr,) → {module:eclairjs/sql.Column}

Formats the arguments in printf-style and returns the result as a string column.
Parameters:
Name Type Description
format, string printf-style
columnExpr, module:eclairjs/sql.Column | string ...columnExpr or columnName, ...columnName
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) from_unixtime(ut, fopt) → {module:eclairjs/sql.Column}

Converts the number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment in the current system time zone in the given format.
Parameters:
Name Type Attributes Description
ut module:eclairjs/sql.Column
f string <optional>
 data format example: "yyyy-MM-dd"
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) from_utc_timestamp(ts, tz) → {module:eclairjs/sql.Column}

Assumes given timestamp is UTC and converts to given timezone.
Parameters:
Name Type Description
ts module:eclairjs/sql.Column
tz string
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) getLength(e) → {module:eclairjs/sql.Column}

Computes the length of a given string or binary column.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) greatest(columnExpr,) → {module:eclairjs/sql.Column}

Returns the greatest value of the list of values, skipping null values. This function takes at least 2 parameters. It will return null if all parameters are null.
Parameters:
Name Type Description
columnExpr, module:eclairjs/sql.Column | string ...columnExpr or columnName, ...columnName
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) hex(column) → {module:eclairjs/sql.Column}

Computes hex value of the given column.
Parameters:
Name Type Description
column module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) hour(e) → {module:eclairjs/sql.Column}

Extracts the hours as an integer from a given date/timestamp/string.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) hypot(left, right) → {module:eclairjs/sql.Column}

Computes `sqrt(a^2^ + b^2^)` without intermediate overflow or underflow.
Parameters:
Name Type Description
left module:eclairjs/sql.Column | string | float
right module:eclairjs/sql.Column | string | float
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column
Example
var col1 = new Column("age");
var col2 = new Column("expense");
var result = functions.hypot(col1, col2);
 // or
 result = functions.hypot(col1, "name");
 // or
 result = functions.hypot("age", col2);
 // or
 result = functions.hypot("age", "expense");
 // or
 result = functions.hypot(col1, 2.0);
 // or
 result = functions.hypot("age", 2.0);
 // or
 result = functions.hypot(2.0, col2);
 // or
 result = functions.hypot(2.0, "expense");

(static) initcap(e) → {module:eclairjs/sql.Column}

Returns a new string column by converting the first letter of each word to uppercase. Words are delimited by whitespace. For example, "hello world" will become "Hello World".
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) inputFileName() → {module:eclairjs/sql.Column}

Creates a string column for the file name of the current Spark task.
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) instr(str, substring) → {module:eclairjs/sql.Column}

Locate the position of the first occurrence of substr column in the given string. Returns null if either of the arguments are null. NOTE: The position is not zero based, but 1 based index, returns 0 if substr could not be found in str.
Parameters:
Name Type Description
str module:eclairjs/sql.Column
substring string
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) isNaN(column) → {module:eclairjs/sql.Column}

Return true iff the column is NaN.
Parameters:
Name Type Description
column module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) lag(column, offset, defaultValue) → {module:eclairjs/sql.Column}

Window function: returns the value that is `offset` rows before the current row, and `null` or defaultValue if there is less than `offset` rows before the current row. For example, an `offset` of one will return the previous row at any given point in the window partition. This is equivalent to the LAG function in SQL.
Parameters:
Name Type Description
column module:eclairjs/sql.Column | string
offset integer
defaultValue object
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) last(column) → {module:eclairjs/sql.Column}

Aggregate function: returns the last value in a group.
Parameters:
Name Type Description
column module:eclairjs/sql.Column | string object or column name as a string
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) last_day(e) → {module:eclairjs/sql.Column}

Given a date column, returns the last day of the month which the given date belongs to. For example, input "2015-07-27" returns "2015-07-31" since July 31 is the last day of the month in July 2015.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) lead(column, offset, defaultValue) → {module:eclairjs/sql.Column}

Window function: returns the value that is `offset` rows after the current row, and `null` or defaultValue if there is less than `offset` rows after the current row. For example, an `offset` of one will return the next row at any given point in the window partition. This is equivalent to the LEAD function in SQL.
Parameters:
Name Type Description
column module:eclairjs/sql.Column | string
offset integer
defaultValue object
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) least(columnExpr,) → {module:eclairjs/sql.Column}

Returns the least value of the list of values, skipping null values. This function takes at least 2 parameters. It will return null iff all parameters are null.
Parameters:
Name Type Description
columnExpr, module:eclairjs/sql.Column | string ...columnExpr or columnName, ...columnName
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) levenshtein(l, r) → {module:eclairjs/sql.Column}

Computes the Levenshtein distance of the two given string columns.
Parameters:
Name Type Description
l module:eclairjs/sql.Column
r module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) lit(literal) → {module:eclairjs/sql.Column}

Creates a Column of literal value. The passed in object is returned directly if it is already a Column. Otherwise, a new Column is created to represent the literal value.
Parameters:
Name Type Description
literal object
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) locate(substr, str, posopt) → {module:eclairjs/sql.Column}

Locate the position of the first occurrence of substr. NOTE: The position is not zero based, but 1 based index, returns 0 if substr could not be found in str.
Parameters:
Name Type Attributes Description
substr string
str module:eclairjs/sql.Column
pos integer <optional>
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) log(col, baseopt) → {module:eclairjs/sql.Column}

Computes the natural logarithm of the given column.
Parameters:
Name Type Attributes Description
col module:eclairjs/sql.Column | string
base float <optional>
 Returns the first argument-base logarithm for the column
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) log1p(col) → {module:eclairjs/sql.Column}

Computes the natural logarithm of the given value plus one.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) log2(col) → {module:eclairjs/sql.Column}

Computes the logarithm of the given column in base 2.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) log10(col) → {module:eclairjs/sql.Column}

Computes the logarithm of the given value in base 10.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) lower(e) → {module:eclairjs/sql.Column}

Converts a string column to lower case.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) lpad(str, len, pad) → {module:eclairjs/sql.Column}

Left-pad the string column with
Parameters:
Name Type Description
str module:eclairjs/sql.Column
len integer
pad string
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) ltrim(e) → {module:eclairjs/sql.Column}

Trim the spaces from left end for the specified string value.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) max(column) → {module:eclairjs/sql.Column}

Aggregate function: returns the maximum value of the expression in a group.
Parameters:
Name Type Description
column module:eclairjs/sql.Column | string object or column name as a string
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) md5(e) → {module:eclairjs/sql.Column}

Calculates the MD5 digest of a binary column and returns the value as a 32 character hex string.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) mean(column) → {module:eclairjs/sql.Column}

Aggregate function: returns the average of the values in a group. Alias for avg.
Parameters:
Name Type Description
column module:eclairjs/sql.Column | string object or column name as a string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) min(column) → {module:eclairjs/sql.Column}

Aggregate function: returns the minimum value of the expression in a group.
Parameters:
Name Type Description
column module:eclairjs/sql.Column | string object or column name as a string
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) minute(e) → {module:eclairjs/sql.Column}

Extracts the minutes as an integer from a given date/timestamp/string.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) monotonicallyIncreasingId() → {module:eclairjs/sql.Column}

A column expression that generates monotonically increasing 64-bit integers. The generated ID is guaranteed to be monotonically increasing and unique, but not consecutive. The current implementation puts the partition ID in the upper 31 bits, and the record number within each partition in the lower 33 bits. The assumption is that the data frame has less than 1 billion partitions, and each partition has less than 8 billion records. As an example, consider a DataFrame with two partitions, each with 3 records. This expression would return the following IDs: 0, 1, 2, 8589934592 (1L << 33), 8589934593, 8589934594.
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) month(e) → {module:eclairjs/sql.Column}

Extracts the month as an integer from a given date/timestamp/string.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) months_between(date1, date2) → {module:eclairjs/sql.Column}

Extracts the months between date1 and date2
Parameters:
Name Type Description
date1 module:eclairjs/sql.Column
date2 module:eclairjs/sql.Column
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) nanvl(col1, col2) → {module:eclairjs/sql.Column}

Returns col1 if it is not NaN, or col2 if col1 is NaN. Both inputs should be floating point columns (DoubleType or FloatType).
Parameters:
Name Type Description
col1 module:eclairjs/sql.Column
col2 module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) negate(e) → {module:eclairjs/sql.Column}

Unary minus, i.e. negate the expression.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column
Example
df.select(functions.negate(df.col("amount")) );

(static) next_day(date, dayOfWeek,) → {module:eclairjs/sql.Column}

Given a date column, returns the first date which is later than the value of the date column that is on the specified day of the week. For example, `next_day('2015-07-27', "Sunday")` returns 2015-08-02 because that is the first Sunday after 2015-07-27. Day of the week parameter is case insensitive, and accepts: "Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun".
Parameters:
Name Type Description
date module:eclairjs/sql.Column
dayOfWeek, string Day of the week parameter is case insensitive, and accepts: "Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun".
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) not(e) → {module:eclairjs/sql.Column}

Inversion of boolean expression, i.e. NOT.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column
Example
df.filter( functions.not(df.col("isActive")) );

(static) ntile(n) → {module:eclairjs/sql.Column}

Window function: returns the ntile group id (from 1 to `n` inclusive) in an ordered window partition. Fow example, if `n` is 4, the first quarter of the rows will get value 1, the second quarter will get 2, the third quarter will get 3, and the last quarter will get 4. This is equivalent to the NTILE function in SQL.
Parameters:
Name Type Description
n integer
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) percentRank() → {module:eclairjs/sql.Column}

Window function: returns the relative rank (i.e. percentile) of rows within a window partition. This is computed by:
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column
Example
(rank of row in its partition - 1) / (number of rows in the partition - 1)


This is equivalent to the PERCENT_RANK function in SQL.

(static) pmod(dividend, divisor) → {module:eclairjs/sql.Column}

Returns the positive value of dividend mod divisor.
Parameters:
Name Type Description
dividend module:eclairjs/sql.Column
divisor module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) pow(left, right) → {module:eclairjs/sql.Column}

Returns the value of the first argument raised to the power of the second argument.
Parameters:
Name Type Description
left module:eclairjs/sql.Column | string | float
right module:eclairjs/sql.Column | string | float
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column
Example
var col1 = new Column("age");
var col2 = new Column("expense");
var result = functions.atan2(col1, col2);
 // or
 result = functions.pow(col1, "name");
 // or
 result = functions.pow("age", col2);
 // or
 result = functions.pow("age", "expense");
 // or
 result = functions.pow(col1, 2.0);
 // or
 result = functions.pow("age", 2.0);
 // or
 result = functions.pow(2.0, col2);
 // or
 result = functions.pow(2.0, "expense");

(static) quarter(e) → {module:eclairjs/sql.Column}

Extracts the quarter as an integer from a given date/timestamp/string.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) rand(seedopt) → {module:eclairjs/sql.Column}

Generate a random column with i.i.d. samples from U[0.0, 1.0].
Parameters:
Name Type Attributes Description
seed integer <optional>
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) randn(seedopt) → {module:eclairjs/sql.Column}

Generate a column with i.i.d. samples from the standard normal distribution.
Parameters:
Name Type Attributes Description
seed integer <optional>
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) rank() → {module:eclairjs/sql.Column}

Window function: returns the rank of rows within a window partition. The difference between rank and denseRank is that denseRank leaves no gaps in ranking sequence when there are ties. That is, if you were ranking a competition using denseRank and had three people tie for second place, you would say that all three were in second place and that the next person came in third. This is equivalent to the RANK function in SQL.
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) regexp_extract(e, regex, groupIdx) → {module:eclairjs/sql.Column}

Extract a specific(idx) group identified by a regex, from the specified string column.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
regex string
groupIdx integer
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) regexp_replace(e, pattern, replacement) → {module:eclairjs/sql.Column}

Replace all substrings of the specified string value that match regexp with rep.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
pattern string
replacement string
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) repeat(str, n) → {module:eclairjs/sql.Column}

Repeats a string column n times, and returns it as a new string column.
Parameters:
Name Type Description
str module:eclairjs/sql.Column
n integer
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) reverse(str) → {module:eclairjs/sql.Column}

Reverses the string column and returns it as a new string column.
Parameters:
Name Type Description
str module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) rint(col) → {module:eclairjs/sql.Column}

Returns the double value that is closest in value to the argument and is equal to a mathematical integer.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) round(e) → {module:eclairjs/sql.Column}

Returns the value of the column `e` rounded to 0 decimal places.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) rowNumber() → {module:eclairjs/sql.Column}

Window function: returns a sequential number starting at 1 within a window partition. This is equivalent to the ROW_NUMBER function in SQL.
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) rpad(e, len, pad) → {module:eclairjs/sql.Column}

Right-padded with pad to a length of len.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
len integer
pad string
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) rtrim(e) → {module:eclairjs/sql.Column}

Trim the spaces from right end for the specified string value.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) second(e) → {module:eclairjs/sql.Column}

Extracts the seconds as an integer from a given date/timestamp/string.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) sha1(e) → {module:eclairjs/sql.Column}

Calculates the SHA-1 digest of a binary column and returns the value as a 40 character hex string.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) sha2(e, numBits) → {module:eclairjs/sql.Column}

Calculates the SHA-2 family of hash functions of a binary column and returns the value as a hex string.
Parameters:
Name Type Description
e module:eclairjs/sql.Column column to compute SHA-2 on.
numBits number one of 224, 256, 384, or 512.
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) shiftLeft(e, numBits) → {module:eclairjs/sql.Column}

Shift the the given value numBits left. If the given value is a long value, this function will return a long value else it will return an integer value.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
numBits integer
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) shiftRight(e, numBits) → {module:eclairjs/sql.Column}

Shift the the given value numBits right. If the given value is a long value, it will return a long value else it will return an integer value.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
numBits integer
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) shiftRightUnsigned(e, numBits) → {module:eclairjs/sql.Column}

Unsigned shift the the given value numBits right. If the given value is a long value, it will return a long value else it will return an integer value.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
numBits integer
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) signum(col) → {module:eclairjs/sql.Column}

Computes the signum of the given value.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) sin(col) → {module:eclairjs/sql.Column}

Computes the sine of the given value.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) sinh(col) → {module:eclairjs/sql.Column}

Computes the hyperbolic sine of the given value.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) size(e) → {module:eclairjs/sql.Column}

Returns length of array or map.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) sort_array(e, ascopt) → {module:eclairjs/sql.Column}

Sorts the input array for the given column in ascending / descending order, according to the natural ordering of the array elements.
Parameters:
Name Type Attributes Description
e module:eclairjs/sql.Column
asc boolean <optional>
 defaults to true
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) soundex(e) → {module:eclairjs/sql.Column}

* Return the soundex code for the specified expression.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) sparkPartitionId() → {module:eclairjs/sql.Column}

Partition ID of the Spark task. Note that this is indeterministic because it depends on data partitioning and task scheduling.
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) split(str, pattern) → {module:eclairjs/sql.Column}

Splits str around pattern (pattern is a regular expression). NOTE: pattern is a string represent the regular expression.
Parameters:
Name Type Description
str module:eclairjs/sql.Column
pattern string
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) sqrt(col) → {module:eclairjs/sql.Column}

Computes the square root of the specified float value.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) struct(columnExpr,) → {module:eclairjs/sql.Column}

Creates a new struct column. If the input column is a column in a DataFrame, or a derived column expression that is named (i.e. aliased), its name would be remained as the StructField's name, otherwise, the newly generated StructField's name would be auto generated as col${index + 1}, i.e. col1, col2, col3, ...
Parameters:
Name Type Description
columnExpr, module:eclairjs/sql.Column | string ...columnExpr or columnName, ...columnName
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) substring(str, pos, len) → {module:eclairjs/sql.Column}

Substring starts at `pos` and is of length `len` when str is String type or returns the slice of byte array that starts at `pos` in byte and is of length `len` when str is Binary type
Parameters:
Name Type Description
str module:eclairjs/sql.Column
pos integer
len integer
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) substring_index(str, delim, count) → {module:eclairjs/sql.Column}

Returns the substring from string str before count occurrences of the delimiter delim. If count is positive, everything the left of the final delimiter (counting from left) is returned. If count is negative, every to the right of the final delimiter (counting from the right) is returned. substring_index performs a case-sensitive match when searching for delim.
Parameters:
Name Type Description
str module:eclairjs/sql.Column
delim string
count integer
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) sum(column) → {module:eclairjs/sql.Column}

Aggregate function: returns the sum of all values in the expression.
Parameters:
Name Type Description
column module:eclairjs/sql.Column | string object or column name as a string
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) sumDistinct(column) → {module:eclairjs/sql.Column}

Aggregate function: returns the sum of distinct values in the expression.
Parameters:
Name Type Description
column module:eclairjs/sql.Column | string object or column name as a string
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) tan(col) → {module:eclairjs/sql.Column}

Computes the tangent of the given value.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) tanh(col) → {module:eclairjs/sql.Column}

Computes the hyperbolic tangent of the given value.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) to_date(e) → {module:eclairjs/sql.Column}

Converts the column into DateType.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) to_utc_timestamp(ts, tz) → {module:eclairjs/sql.Column}

Assumes given timestamp is in given timezone and converts to UTC.
Parameters:
Name Type Description
ts module:eclairjs/sql.Column
tz string
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) toDegrees(col) → {module:eclairjs/sql.Column}

Converts an angle measured in radians to an approximately equivalent angle measured in degrees.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) toRadians(col) → {module:eclairjs/sql.Column}

Converts an angle measured in degrees to an approximately equivalent angle measured in radians.
Parameters:
Name Type Description
col module:eclairjs/sql.Column | string
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) translate(src, matchingString, replaceString) → {module:eclairjs/sql.Column}

Translate any character in the src by a character in replaceString. The characters in replaceString is corresponding to the characters in matchingString. The translate will happen when any character in the string matching with the character in the matchingString.
Parameters:
Name Type Description
src module:eclairjs/sql.Column
matchingString string
replaceString string
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) trim(e) → {module:eclairjs/sql.Column}

Trim the spaces from both ends for the specified string column.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) trunc(date, format) → {module:eclairjs/sql.Column}

Returns date truncated to the unit specified by the format.
Parameters:
Name Type Description
date module:eclairjs/sql.Column
format string : 'year', 'yyyy', 'yy' for truncate by year, or 'month', 'mon', 'mm' for truncate by month
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) unbase64(e) → {module:eclairjs/sql.Column}

Decodes a BASE64 encoded string column and returns it as a binary column. This is the reverse of base64.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) unhex(column) → {module:eclairjs/sql.Column}

Inverse of hex. Interprets each pair of characters as a hexadecimal number and converts to the byte representation of number.
Parameters:
Name Type Description
column module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) unix_timestamp(sopt, popt) → {module:eclairjs/sql.Column}

Gets current Unix timestamp in seconds. if not arguments are specified Convert time string with given pattern (see [http://docs.oracle.com/javase/tutorial/i18n/format/simpleDateFormat.html]) to Unix time stamp (in seconds), return null if fail.
Parameters:
Name Type Attributes Description
s module:eclairjs/sql.Column <optional>
 converts time string in format yyyy-MM-dd HH:mm:ss to Unix timestamp (in seconds), using the default timezone and the default locale, return null if fail.
p string <optional>
 convert time string with given pattern (see [http://docs.oracle.com/javase/tutorial/i18n/format/simpleDateFormat.html]) to Unix time stamp (in seconds), return null if fail.
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) upper(e) → {module:eclairjs/sql.Column}

Converts a string column to upper case.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.3.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) weekofyear(e) → {module:eclairjs/sql.Column}

Extracts the week number as an integer from a given date/timestamp/string.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) when() → {module:eclairjs/sql.Column}

Evaluates a list of conditions and returns one of multiple possible result expressions. If otherwise is not defined at the end, null is returned for unmatched conditions.
Since:
  • EclairJS 0.1 Spark 1.4.0
Source:
Returns:
Type
module:eclairjs/sql.Column
Example
// Example: encoding gender string column into integer.
  people.select(functions.when(people.col("gender").equalTo("male"), 0)
    .when(people.col("gender").equalTo("female"), 1)
    .otherwise(2))

(static) window(timeColumn, windowDuration, slideDurationopt, startTimeopt) → {module:eclairjs/sql.Column}

Generates tumbling time windows given a timestamp specifying column.
Parameters:
Name Type Attributes Description
timeColumn module:eclairjs/sql.Column
windowDuration string
slideDuration string <optional>
 Bucketize rows into one or more time windows given a timestamp specifying column.
startTime string <optional>
 Bucketize rows into one or more time windows given a timestamp specifying column.
Source:
Returns:
Type
module:eclairjs/sql.Column

(static) year(e) → {module:eclairjs/sql.Column}

Extracts the year as an integer from a given date/timestamp/string.
Parameters:
Name Type Description
e module:eclairjs/sql.Column
Since:
  • EclairJS 0.1 Spark 1.5.0
Source:
Returns:
Type
module:eclairjs/sql.Column