To compute the median:
double myMedian = median().compute(myDataset);
where median() has been statically imported.
To compute the 99th percentile:
double myPercentile99 = percentiles().index(99).compute(myDataset);
where percentiles() has been statically imported.
To compute median and the 90th and 99th percentiles:
Map<Integer, Double> myPercentiles =
percentiles().indexes(50, 90, 99).compute(myDataset);
where percentiles() has been statically imported: myPercentiles maps the keys
50, 90, and 99, to their corresponding quantile values.
To compute quartiles, use quartiles() instead of percentiles(). To compute
arbitrary q-quantiles, use scale(q).
These examples all take a copy of your dataset. If you have a double array, you are okay with
it being arbitrarily reordered, and you want to avoid that copy, you can use computeInPlace instead of compute.
The definition of the kth q-quantile of N values is as follows: define x = k * (N - 1) / q; if
x is an integer, the result is the value which would appear at index x in the sorted dataset
(unless there are NaN values, see below); otherwise, the result is the average
of the values which would appear at the indexes floor(x) and ceil(x) weighted by (1-frac(x)) and
frac(x) respectively. This is the same definition as used by Excel and by S, it is the Type 7
definition in R, and it is
described by
wikipedia as providing "Linear interpolation of the modes for the order statistics for the
uniform distribution on [0,1]."
If any values in the input are NaN then all values returned are NaN. (This is the one occasion when the behaviour is not the same as you'd get from
sorting with Arrays. or Collections. and selecting
the required value(s). Those methods would sort NaN as if it is greater than
any other value and place them at the end of the dataset, even after POSITIVE_INFINITY.)
Otherwise, NEGATIVE_INFINITY and POSITIVE_INFINITY sort to the beginning and the end of the dataset, as
you would expect.
If required to do a weighted average between an infinity and a finite value, or between an
infinite value and itself, the infinite value is returned. If required to do a weighted average
between NEGATIVE_INFINITY and POSITIVE_INFINITY, NaN is returned (note that this will only happen if the
dataset contains no finite values).
The average time complexity of the computation is O(N) in the size of the dataset. There is a worst case time complexity of O(N^2). You are extremely unlikely to hit this quadratic case on randomly ordered data (the probability decreases faster than exponentially in N), but if you are passing in unsanitized user data then a malicious user could force it. A light shuffle of the data using an unpredictable seed should normally be enough to thwart this attack.
The time taken to compute multiple quantiles on the same dataset using indexes is generally less than the total time taken to compute each of them separately, and
sometimes much less. For example, on a large enough dataset, computing the 90th and 99th
percentiles together takes about 55% as long as computing them separately.
When calling ScaleAndIndex#compute (in either
form), the memory requirement is 8*N bytes for the copy of the dataset plus an overhead which is
independent of N (but depends on the quantiles being computed). When calling computeInPlace (in either form), only the overhead is required. The number of object allocations is independent of
N in both cases.
| Modifier and Type | Class and Description |
|---|---|
| public static class | Quantiles.
Describes the point in a fluent API chain where only the scale (i.e. the q in q-quantiles) has been specified. |
| public static class | Quantiles.
Describes the point in a fluent API chain where the scale and a single quantile index (i.e. the q and the k in the kth q-quantile) have been specified. |
| public static class | Quantiles.
Describes the point in a fluent API chain where the scale and a multiple quantile indexes (i.e. the q and a set of values for the k in the kth q-quantile) have been specified. |
| Access | Constructor and Description |
|---|---|
| public |
| Modifier and Type | Method and Description |
|---|---|
| private static void | |
| private static int | chooseNextSelection(int[] allRequired, int requiredFrom, int requiredTo, int from, int to)
Chooses the next selection to do from the required selections. |
| private static boolean | |
| private static double | interpolate(double lower, double upper, double remainder, double scale)
Returns a value a fraction |
| private static double[] | |
| private static double[] | |
| public static Quantiles. | |
| private static void | movePivotToStartOfSlice(double[] array, int from, int to)
Selects the pivot to use, namely the median of the values at |
| private static int | partition(double[] array, int from, int to)
Performs a partition operation on the slice of |
| public static Quantiles. | |
| public static Quantiles. | |
| public static Quantiles. | scale(int
the scale for the quantiles to be calculated, i.e. the q of the q-quantiles, which
must be positive scale)Specifies the computation of q-quantiles. |
| private static void | selectAllInPlace(int[] allRequired, int requiredFrom, int requiredTo, double[] array, int from, int to)
Performs an in-place selection, like |
| private static void | selectInPlace(int required, double[] array, int from, int to)
Performs an in-place selection to find the element which would appear at a given index in a dataset if it were sorted. |
| private static void |
| Quantiles | back to summary |
|---|---|
| public Quantiles() | |
| checkIndex | back to summary |
|---|---|
| private static void checkIndex(int index, int scale) | |
| chooseNextSelection | back to summary |
|---|---|
| private static int chooseNextSelection(int[] allRequired, int requiredFrom, int requiredTo, int from, int to) Chooses the next selection to do from the required selections. It is required that the array
| |
| containsNaN | back to summary |
|---|---|
| private static boolean containsNaN(double... dataset) Returns whether any of the values in | |
| interpolate | back to summary |
|---|---|
| private static double interpolate(double lower, double upper, double remainder, double scale) Returns a value a fraction | |
| intsToDoubles | back to summary |
|---|---|
| private static double[] intsToDoubles(int[] ints) | |
| longsToDoubles | back to summary |
|---|---|
| private static double[] longsToDoubles(long[] longs) | |
| median | back to summary |
|---|---|
| public static Quantiles. Specifies the computation of a median (i.e. the 1st 2-quantile). | |
| movePivotToStartOfSlice | back to summary |
|---|---|
| private static void movePivotToStartOfSlice(double[] array, int from, int to) Selects the pivot to use, namely the median of the values at | |
| partition | back to summary |
|---|---|
| private static int partition(double[] array, int from, int to) Performs a partition operation on the slice of | |
| percentiles | back to summary |
|---|---|
| public static Quantiles. Specifies the computation of percentiles (i.e. 100-quantiles). | |
| quartiles | back to summary |
|---|---|
| public static Quantiles. Specifies the computation of quartiles (i.e. 4-quantiles). | |
| scale | back to summary |
|---|---|
| public static Quantiles. Specifies the computation of q-quantiles.
| |
| selectAllInPlace | back to summary |
|---|---|
| private static void selectAllInPlace(int[] allRequired, int requiredFrom, int requiredTo, double[] array, int from, int to) Performs an in-place selection, like | |
| selectInPlace | back to summary |
|---|---|
| private static void selectInPlace(int required, double[] array, int from, int to) Performs an in-place selection to find the element which would appear at a given index in a dataset if it were sorted. The following preconditions should hold:
from, to] such
that all the values with indexes in the range [from, required) are less than or
equal to the value with index required, and all the values with indexes in the range
(required, to] are greater than or equal to that value. Therefore, the value at
required is the value which would appear at that index in the sorted dataset.
| |
| swap | back to summary |
|---|---|
| private static void swap(double[] array, int i, int j) Swaps the values at | |
| Modifier and Type | Field and Description |
|---|---|
| private final int |
| Access | Constructor and Description |
|---|---|
| private |
| Modifier and Type | Method and Description |
|---|---|
| public Quantiles. | index(int
the quantile index, which must be in the inclusive range [0, q] for q-quantiles index)Specifies a single quantile index to be calculated, i.e. the k in the kth q-quantile. |
| public Quantiles. | indexes(int...
the quantile indexes, each of which must be in the inclusive range [0, q] for
q-quantiles; the order of the indexes is unimportant, duplicates will be ignored, and the
set will be snapshotted when this method is called indexes)Specifies multiple quantile indexes to be calculated, each index being the k in the kth q-quantile. |
| public Quantiles. | indexes(Collection<Integer>
the quantile indexes, each of which must be in the inclusive range [0, q] for
q-quantiles; the order of the indexes is unimportant, duplicates will be ignored, and the
set will be snapshotted when this method is called indexes)Specifies multiple quantile indexes to be calculated, each index being the k in the kth q-quantile. |
| scale | back to summary |
|---|---|
| private final int scale | |
| Scale | back to summary |
|---|---|
| private Scale(int scale) | |
| index | back to summary |
|---|---|
| public Quantiles. Specifies a single quantile index to be calculated, i.e. the k in the kth q-quantile.
| |
| indexes | back to summary |
|---|---|
| public Quantiles. Specifies multiple quantile indexes to be calculated, each index being the k in the kth q-quantile.
| |
| indexes | back to summary |
|---|---|
| public Quantiles. Specifies multiple quantile indexes to be calculated, each index being the k in the kth q-quantile.
| |
| Access | Constructor and Description |
|---|---|
| private |
| Modifier and Type | Method and Description |
|---|---|
| public double | Returns: the quantile valuethe dataset to do the calculation on, which must be non-empty, which will be
cast to doubles (with any associated lost of precision), and which will not be mutated by
this call (it is copied instead) dataset)Computes the quantile value of the given dataset. |
| public double | Returns: the quantile valuethe dataset to do the calculation on, which must be non-empty, which will not
be mutated by this call (it is copied instead) dataset)Computes the quantile value of the given dataset. |
| public double | Returns: the quantile valuethe dataset to do the calculation on, which must be non-empty, which will be
cast to doubles (with any associated lost of precision), and which will not be mutated by
this call (it is copied instead) dataset)Computes the quantile value of the given dataset. |
| public double | Returns: the quantile valuethe dataset to do the calculation on, which must be non-empty, which will be
cast to doubles, and which will not be mutated by this call (it is copied instead) dataset)Computes the quantile value of the given dataset. |
| public double | Returns: the quantile valuethe dataset to do the calculation on, which must be non-empty, and which will
be arbitrarily reordered by this method call dataset)Computes the quantile value of the given dataset, performing the computation in-place. |
| index | back to summary |
|---|---|
| private final int index | |
| scale | back to summary |
|---|---|
| private final int scale | |
| ScaleAndIndex | back to summary |
|---|---|
| private ScaleAndIndex(int scale, int index) | |
| compute | back to summary |
|---|---|
| public double compute(Collection<? extends Number> dataset) Computes the quantile value of the given dataset.
| |
| compute | back to summary |
|---|---|
| public double compute(double... dataset) Computes the quantile value of the given dataset.
| |
| compute | back to summary |
|---|---|
| public double compute(long... dataset) Computes the quantile value of the given dataset.
| |
| compute | back to summary |
|---|---|
| public double compute(int... dataset) Computes the quantile value of the given dataset.
| |
| computeInPlace | back to summary |
|---|---|
| public double computeInPlace(double... dataset) Computes the quantile value of the given dataset, performing the computation in-place.
| |
| Modifier and Type | Field and Description |
|---|---|
| private final int[] | |
| private final int |
| Access | Constructor and Description |
|---|---|
| private |
| Modifier and Type | Method and Description |
|---|---|
| public Map | Returns: an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order they were passed to theindexes method.the dataset to do the calculation on, which must be non-empty, which will be
cast to doubles (with any associated lost of precision), and which will not be mutated by
this call (it is copied instead) dataset)Computes the quantile values of the given dataset. |
| public Map | Returns: an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order they were passed to theindexes method.the dataset to do the calculation on, which must be non-empty, which will not
be mutated by this call (it is copied instead) dataset)Computes the quantile values of the given dataset. |
| public Map | Returns: an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order they were passed to theindexes method.the dataset to do the calculation on, which must be non-empty, which will be
cast to doubles (with any associated lost of precision), and which will not be mutated by
this call (it is copied instead) dataset)Computes the quantile values of the given dataset. |
| public Map | Returns: an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order they were passed to theindexes method.the dataset to do the calculation on, which must be non-empty, which will be
cast to doubles, and which will not be mutated by this call (it is copied instead) dataset)Computes the quantile values of the given dataset. |
| public Map | Returns: an unmodifiable, ordered map of results: the keys will be the specified quantile indexes, and the values the corresponding quantile values. When iterating, entries in the map are ordered by quantile index in the same order that the indexes were passed to theindexes method.the dataset to do the calculation on, which must be non-empty, and which will
be arbitrarily reordered by this method call dataset)Computes the quantile values of the given dataset, performing the computation in-place. |
| indexes | back to summary |
|---|---|
| private final int[] indexes | |
| scale | back to summary |
|---|---|
| private final int scale | |
| ScaleAndIndexes | back to summary |
|---|---|
| private ScaleAndIndexes(int scale, int[] indexes) | |
| compute | back to summary |
|---|---|
| public Map Computes the quantile values of the given dataset.
| |
| compute | back to summary |
|---|---|
| public Map Computes the quantile values of the given dataset.
| |
| compute | back to summary |
|---|---|
| public Map Computes the quantile values of the given dataset.
| |
| compute | back to summary |
|---|---|
| public Map Computes the quantile values of the given dataset.
| |
| computeInPlace | back to summary |
|---|---|
| public Map Computes the quantile values of the given dataset, performing the computation in-place.
| |