Top Description Inners Constructors Methods
com.google.common.math

public final Class Quantiles

extends Object
Class Inheritance
Annotations
@J2ktIncompatible
@GwtIncompatible
@ElementTypesAreNonnullByDefault
Static Imports
com.google.common.base.Preconditions.checkArgument, java.lang.Double.NEGATIVE_INFINITY, .Double.NaN, .Double.POSITIVE_INFINITY, java.util.Arrays.sort, .Collections.unmodifiableMap

Provides a fluent API for calculating quantiles.

Examples

To compute the median:

double myMedian = median().compute(myDataset);
where median() has been statically imported.

To compute the 99th percentile:

double myPercentile99 = percentiles().index(99).compute(myDataset);
where percentiles() has been statically imported.

To compute median and the 90th and 99th percentiles:

Map<Integer, Double> myPercentiles =
    percentiles().indexes(50, 90, 99).compute(myDataset);
where percentiles() has been statically imported: myPercentiles maps the keys 50, 90, and 99, to their corresponding quantile values.

To compute quartiles, use quartiles() instead of percentiles(). To compute arbitrary q-quantiles, use scale(q).

These examples all take a copy of your dataset. If you have a double array, you are okay with it being arbitrarily reordered, and you want to avoid that copy, you can use computeInPlace instead of compute.

Definition and notes on interpolation

The definition of the kth q-quantile of N values is as follows: define x = k * (N - 1) / q; if x is an integer, the result is the value which would appear at index x in the sorted dataset (unless there are NaN values, see below); otherwise, the result is the average of the values which would appear at the indexes floor(x) and ceil(x) weighted by (1-frac(x)) and frac(x) respectively. This is the same definition as used by Excel and by S, it is the Type 7 definition in R, and it is described by wikipedia as providing "Linear interpolation of the modes for the order statistics for the uniform distribution on [0,1]."

Handling of non-finite values

If any values in the input are NaN then all values returned are NaN. (This is the one occasion when the behaviour is not the same as you'd get from sorting with Arrays.sort(double[]) or Collections.sort(List<Double>) and selecting the required value(s). Those methods would sort NaN as if it is greater than any other value and place them at the end of the dataset, even after POSITIVE_INFINITY.)

Otherwise, NEGATIVE_INFINITY and POSITIVE_INFINITY sort to the beginning and the end of the dataset, as you would expect.

If required to do a weighted average between an infinity and a finite value, or between an infinite value and itself, the infinite value is returned. If required to do a weighted average between NEGATIVE_INFINITY and POSITIVE_INFINITY, NaN is returned (note that this will only happen if the dataset contains no finite values).

Performance

The average time complexity of the computation is O(N) in the size of the dataset. There is a worst case time complexity of O(N^2). You are extremely unlikely to hit this quadratic case on randomly ordered data (the probability decreases faster than exponentially in N), but if you are passing in unsanitized user data then a malicious user could force it. A light shuffle of the data using an unpredictable seed should normally be enough to thwart this attack.

The time taken to compute multiple quantiles on the same dataset using indexes is generally less than the total time taken to compute each of them separately, and sometimes much less. For example, on a large enough dataset, computing the 90th and 99th percentiles together takes about 55% as long as computing them separately.

When calling ScaleAndIndex#compute (in either form), the memory requirement is 8*N bytes for the copy of the dataset plus an overhead which is independent of N (but depends on the quantiles being computed). When calling computeInPlace (in either form), only the overhead is required. The number of object allocations is independent of N in both cases.

Author
Pete Gillin
Since
20.0

Nested and Inner Type Summary

Modifier and TypeClass and Description
public static class
Quantiles.Scale

Describes the point in a fluent API chain where only the scale (i.e. the q in q-quantiles) has been specified.

public static class
Quantiles.ScaleAndIndex

Describes the point in a fluent API chain where the scale and a single quantile index (i.e. the q and the k in the kth q-quantile) have been specified.

public static class
Quantiles.ScaleAndIndexes

Describes the point in a fluent API chain where the scale and a multiple quantile indexes (i.e. the q and a set of values for the k in the kth q-quantile) have been specified.

Constructor Summary

AccessConstructor and Description
public

Method Summary

Modifier and TypeMethod and Description
private static void
checkIndex(int index, int scale)

private static int
chooseNextSelection(int[] allRequired, int requiredFrom, int requiredTo, int from, int to)

Chooses the next selection to do from the required selections.

private static boolean
containsNaN(double... dataset)

Returns whether any of the values in dataset are NaN.

private static double
interpolate(double lower, double upper, double remainder, double scale)

Returns a value a fraction (remainder / scale) of the way between lower and upper.

private static double[]
intsToDoubles(int[] ints)

private static double[]
longsToDoubles(long[] longs)

public static Quantiles.ScaleAndIndex
median()

Specifies the computation of a median (i.e. the 1st 2-quantile).

private static void
movePivotToStartOfSlice(double[] array, int from, int to)

Selects the pivot to use, namely the median of the values at from, to, and halfway between the two (rounded down), from array, and ensure (by swapping elements if necessary) that that pivot value appears at the start of the slice i.e. at from.

private static int
partition(double[] array, int from, int to)

Performs a partition operation on the slice of array with elements in the range [from, to].

public static Quantiles.Scale
percentiles()

Specifies the computation of percentiles (i.e. 100-quantiles).

public static Quantiles.Scale
quartiles()

Specifies the computation of quartiles (i.e. 4-quantiles).

public static Quantiles.Scale
scale(int
the scale for the quantiles to be calculated, i.e. the q of the q-quantiles, which must be positive
scale
)

Specifies the computation of q-quantiles.

private static void
selectAllInPlace(int[] allRequired, int requiredFrom, int requiredTo, double[] array, int from, int to)

Performs an in-place selection, like selectInPlace, to select all the indexes allRequired[i] for i in the range [requiredFrom, requiredTo].

private static void
selectInPlace(int required, double[] array, int from, int to)

Performs an in-place selection to find the element which would appear at a given index in a dataset if it were sorted.

private static void
swap(double[] array, int i, int j)

Swaps the values at i and j in array.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Constructor Detail

Quantilesback to summary
public Quantiles()

Method Detail

checkIndexback to summary
private static void checkIndex(int index, int scale)
chooseNextSelectionback to summary
private static int chooseNextSelection(int[] allRequired, int requiredFrom, int requiredTo, int from, int to)

Chooses the next selection to do from the required selections. It is required that the array allRequired is sorted and that allRequired[i] are in the range [from, to] for all i in the range [requiredFrom, requiredTo]. The value returned by this method is the i in that range such that allRequired[i] is as close as possible to the center of the range [from, to]. Choosing the value closest to the center of the range first is the most efficient strategy because it minimizes the size of the subranges from which the remaining selections must be done.

containsNaNback to summary
private static boolean containsNaN(double... dataset)

Returns whether any of the values in dataset are NaN.

interpolateback to summary
private static double interpolate(double lower, double upper, double remainder, double scale)

Returns a value a fraction (remainder / scale) of the way between lower and upper. Assumes that lower <= upper. Correctly handles infinities (but not NaN).

intsToDoublesback to summary
private static double[] intsToDoubles(int[] ints)
longsToDoublesback to summary
private static double[] longsToDoubles(long[] longs)
medianback to summary
public static Quantiles.ScaleAndIndex median()

Specifies the computation of a median (i.e. the 1st 2-quantile).

movePivotToStartOfSliceback to summary
private static void movePivotToStartOfSlice(double[] array, int from, int to)

Selects the pivot to use, namely the median of the values at from, to, and halfway between the two (rounded down), from array, and ensure (by swapping elements if necessary) that that pivot value appears at the start of the slice i.e. at from. Expects that from is strictly less than to.

partitionback to summary
private static int partition(double[] array, int from, int to)

Performs a partition operation on the slice of array with elements in the range [from, to]. Uses the median of from, to, and the midpoint between them as a pivot. Returns the index which the slice is partitioned around, i.e. if it returns ret then we know that the values with indexes in [from, ret) are less than or equal to the value at ret and the values with indexes in (ret, to] are greater than or equal to that.

percentilesback to summary
public static Quantiles.Scale percentiles()

Specifies the computation of percentiles (i.e. 100-quantiles).

quartilesback to summary
public static Quantiles.Scale quartiles()

Specifies the computation of quartiles (i.e. 4-quantiles).

scaleback to summary
public static Quantiles.Scale scale(int scale)

Specifies the computation of q-quantiles.

Parameters
scale:int

the scale for the quantiles to be calculated, i.e. the q of the q-quantiles, which must be positive

selectAllInPlaceback to summary
private static void selectAllInPlace(int[] allRequired, int requiredFrom, int requiredTo, double[] array, int from, int to)

Performs an in-place selection, like selectInPlace, to select all the indexes allRequired[i] for i in the range [requiredFrom, requiredTo]. These indexes must be sorted in the array and must all be in the range [from, to].

selectInPlaceback to summary
private static void selectInPlace(int required, double[] array, int from, int to)

Performs an in-place selection to find the element which would appear at a given index in a dataset if it were sorted. The following preconditions should hold:

  • required, from, and to should all be indexes into array;
  • required should be in the range [from, to];
  • all the values with indexes in the range [0, from) should be less than or equal to all the values with indexes in the range [from, to];
  • all the values with indexes in the range (to, array.length - 1] should be greater than or equal to all the values with indexes in the range [from, to].
This method will reorder the values with indexes in the range [from, to] such that all the values with indexes in the range [from, required) are less than or equal to the value with index required, and all the values with indexes in the range (required, to] are greater than or equal to that value. Therefore, the value at required is the value which would appear at that index in the sorted dataset.
swapback to summary
private static void swap(double[] array, int i, int j)

Swaps the values at i and j in array.

com.google.common.math back to summary

public final Class Quantiles.Scale

extends Object
Class Inheritance

Describes the point in a fluent API chain where only the scale (i.e. the q in q-quantiles) has been specified.
Since
20.0

Field Summary

Modifier and TypeField and Description
private final int

Constructor Summary

AccessConstructor and Description
private
Scale(int scale)

Method Summary

Modifier and TypeMethod and Description
public Quantiles.ScaleAndIndex
index(int
the quantile index, which must be in the inclusive range [0, q] for q-quantiles
index
)

Specifies a single quantile index to be calculated, i.e. the k in the kth q-quantile.

public Quantiles.ScaleAndIndexes
indexes(int...
the quantile indexes, each of which must be in the inclusive range [0, q] for q-quantiles; the order of the indexes is unimportant, duplicates will be ignored, and the set will be snapshotted when this method is called
indexes
)

Specifies multiple quantile indexes to be calculated, each index being the k in the kth q-quantile.

public Quantiles.ScaleAndIndexes
indexes(Collection<Integer>
the quantile indexes, each of which must be in the inclusive range [0, q] for q-quantiles; the order of the indexes is unimportant, duplicates will be ignored, and the set will be snapshotted when this method is called
indexes
)

Specifies multiple quantile indexes to be calculated, each index being the k in the kth q-quantile.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

scaleback to summary
private final int scale

Constructor Detail

Scaleback to summary
private Scale(int scale)

Method Detail

indexback to summary
public Quantiles.ScaleAndIndex index(int index)

Specifies a single quantile index to be calculated, i.e. the k in the kth q-quantile.

Parameters
index:int

the quantile index, which must be in the inclusive range [0, q] for q-quantiles

indexesback to summary
public Quantiles.ScaleAndIndexes indexes(int... indexes)

Specifies multiple quantile indexes to be calculated, each index being the k in the kth q-quantile.

Parameters
indexes:int[]

the quantile indexes, each of which must be in the inclusive range [0, q] for q-quantiles; the order of the indexes is unimportant, duplicates will be ignored, and the set will be snapshotted when this method is called

Exceptions
IllegalArgumentException:
if indexes is empty
indexesback to summary
public Quantiles.ScaleAndIndexes indexes(Collection<Integer> indexes)

Specifies multiple quantile indexes to be calculated, each index being the k in the kth q-quantile.

Parameters
indexes:Collection<Integer>

the quantile indexes, each of which must be in the inclusive range [0, q] for q-quantiles; the order of the indexes is unimportant, duplicates will be ignored, and the set will be snapshotted when this method is called

Exceptions
IllegalArgumentException:
if indexes is empty
com.google.common.math back to summary

public final Class Quantiles.ScaleAndIndex

extends Object
Class Inheritance

Describes the point in a fluent API chain where the scale and a single quantile index (i.e. the q and the k in the kth q-quantile) have been specified.
Since
20.0

Field Summary

Modifier and TypeField and Description
private final int
private final int

Constructor Summary

AccessConstructor and Description
private
ScaleAndIndex(int scale, int index)

Method Summary

Modifier and TypeMethod and Description
public double

Returns:

the quantile value
compute
(Collection<? extends Number>
the dataset to do the calculation on, which must be non-empty, which will be cast to doubles (with any associated lost of precision), and which will not be mutated by this call (it is copied instead)
dataset
)

Computes the quantile value of the given dataset.

public double

Returns:

the quantile value
compute
(double...
the dataset to do the calculation on, which must be non-empty, which will not be mutated by this call (it is copied instead)
dataset
)

Computes the quantile value of the given dataset.

public double

Returns:

the quantile value
compute
(long...
the dataset to do the calculation on, which must be non-empty, which will be cast to doubles (with any associated lost of precision), and which will not be mutated by this call (it is copied instead)
dataset
)

Computes the quantile value of the given dataset.

public double

Returns:

the quantile value
compute
(int...
the dataset to do the calculation on, which must be non-empty, which will be cast to doubles, and which will not be mutated by this call (it is copied instead)
dataset
)

Computes the quantile value of the given dataset.

public double

Returns:

the quantile value
computeInPlace
(double...
the dataset to do the calculation on, which must be non-empty, and which will be arbitrarily reordered by this method call
dataset
)

Computes the quantile value of the given dataset, performing the computation in-place.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

indexback to summary
private final int index
scaleback to summary
private final int scale

Constructor Detail

ScaleAndIndexback to summary
private ScaleAndIndex(int scale, int index)

Method Detail

computeback to summary
public double compute(Collection<? extends Number> dataset)

Computes the quantile value of the given dataset.

Parameters
dataset:Collection<? extends Number>

the dataset to do the calculation on, which must be non-empty, which will be cast to doubles (with any associated lost of precision), and which will not be mutated by this call (it is copied instead)

Returns:double

the quantile value

computeback to summary
public double compute(double... dataset)

Computes the quantile value of the given dataset.

Parameters
dataset:double[]

the dataset to do the calculation on, which must be non-empty, which will not be mutated by this call (it is copied instead)

Returns:double

the quantile value

computeback to summary
public double compute(long... dataset)

Computes the quantile value of the given dataset.

Parameters
dataset:long[]

the dataset to do the calculation on, which must be non-empty, which will be cast to doubles (with any associated lost of precision), and which will not be mutated by this call (it is copied instead)

Returns:double

the quantile value

computeback to summary
public double compute(int... dataset)

Computes the quantile value of the given dataset.

Parameters
dataset:int[]

the dataset to do the calculation on, which must be non-empty, which will be cast to doubles, and which will not be mutated by this call (it is copied instead)

Returns:double

the quantile value

computeInPlaceback to summary
public double computeInPlace(double... dataset)

Computes the quantile value of the given dataset, performing the computation in-place.

Parameters
dataset:double[]

the dataset to do the calculation on, which must be non-empty, and which will be arbitrarily reordered by this method call

Returns:double

the quantile value

com.google.common.math back to summary

public final Class Quantiles.ScaleAndIndexes

extends Object
Class Inheritance

Describes the point in a fluent API chain where the scale and a multiple quantile indexes (i.e. the q and a set of values for the k in the kth q-quantile) have been specified.
Since
20.0

Field Summary

Modifier and TypeField and Description
private final int[]
private final int

Constructor Summary

AccessConstructor and Description
private
ScaleAndIndexes(int scale, int[] indexes)

Method Summary

Modifier and TypeMethod and Description
public Map<Integer, Double>

Returns:

an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order they were passed to the indexes method.
compute
(Collection<? extends Number>
the dataset to do the calculation on, which must be non-empty, which will be cast to doubles (with any associated lost of precision), and which will not be mutated by this call (it is copied instead)
dataset
)

Computes the quantile values of the given dataset.

public Map<Integer, Double>

Returns:

an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order they were passed to the indexes method.
compute
(double...
the dataset to do the calculation on, which must be non-empty, which will not be mutated by this call (it is copied instead)
dataset
)

Computes the quantile values of the given dataset.

public Map<Integer, Double>

Returns:

an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order they were passed to the indexes method.
compute
(long...
the dataset to do the calculation on, which must be non-empty, which will be cast to doubles (with any associated lost of precision), and which will not be mutated by this call (it is copied instead)
dataset
)

Computes the quantile values of the given dataset.

public Map<Integer, Double>

Returns:

an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order they were passed to the indexes method.
compute
(int...
the dataset to do the calculation on, which must be non-empty, which will be cast to doubles, and which will not be mutated by this call (it is copied instead)
dataset
)

Computes the quantile values of the given dataset.

public Map<Integer, Double>

Returns:

an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order that the indexes were passed to the indexes method.
computeInPlace
(double...
the dataset to do the calculation on, which must be non-empty, and which will be arbitrarily reordered by this method call
dataset
)

Computes the quantile values of the given dataset, performing the computation in-place.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

indexesback to summary
private final int[] indexes
scaleback to summary
private final int scale

Constructor Detail

ScaleAndIndexesback to summary
private ScaleAndIndexes(int scale, int[] indexes)

Method Detail

computeback to summary
public Map<Integer, Double> compute(Collection<? extends Number> dataset)

Computes the quantile values of the given dataset.

Parameters
dataset:Collection<? extends Number>

the dataset to do the calculation on, which must be non-empty, which will be cast to doubles (with any associated lost of precision), and which will not be mutated by this call (it is copied instead)

Returns:Map<Integer, Double>

an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order they were passed to the indexes method.

computeback to summary
public Map<Integer, Double> compute(double... dataset)

Computes the quantile values of the given dataset.

Parameters
dataset:double[]

the dataset to do the calculation on, which must be non-empty, which will not be mutated by this call (it is copied instead)

Returns:Map<Integer, Double>

an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order they were passed to the indexes method.

computeback to summary
public Map<Integer, Double> compute(long... dataset)

Computes the quantile values of the given dataset.

Parameters
dataset:long[]

the dataset to do the calculation on, which must be non-empty, which will be cast to doubles (with any associated lost of precision), and which will not be mutated by this call (it is copied instead)

Returns:Map<Integer, Double>

an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order they were passed to the indexes method.

computeback to summary
public Map<Integer, Double> compute(int... dataset)

Computes the quantile values of the given dataset.

Parameters
dataset:int[]

the dataset to do the calculation on, which must be non-empty, which will be cast to doubles, and which will not be mutated by this call (it is copied instead)

Returns:Map<Integer, Double>

an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order they were passed to the indexes method.

computeInPlaceback to summary
public Map<Integer, Double> computeInPlace(double... dataset)

Computes the quantile values of the given dataset, performing the computation in-place.

Parameters
dataset:double[]

the dataset to do the calculation on, which must be non-empty, and which will be arbitrarily reordered by this method call

Returns:Map<Integer, Double>

an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order that the indexes were passed to the indexes method.