Top Description Fields Constructors Methods
org.apache.commons.math3.stat.inference

public Class KolmogorovSmirnovTest

extends Object
Class Inheritance
Imports
java.math.BigDecimal, java.util.Arrays, .HashSet, org.apache.commons.math3.distribution.EnumeratedRealDistribution, .RealDistribution, .UniformRealDistribution, org.apache.commons.math3.exception.InsufficientDataException, .MathArithmeticException, .MathInternalError, .NullArgumentException, .NumberIsTooLargeException, .OutOfRangeException, .TooManyIterationsException, org.apache.commons.math3.exception.util.LocalizedFormats, org.apache.commons.math3.fraction.BigFraction, .BigFractionField, .FractionConversionException, org.apache.commons.math3.linear.Array2DRowFieldMatrix, .FieldMatrix, .MatrixUtils, .RealMatrix, org.apache.commons.math3.random.JDKRandomGenerator, .RandomGenerator, .Well19937c, org.apache.commons.math3.util.CombinatoricsUtils, .FastMath, .MathArrays, .MathUtils

Implementation of the Kolmogorov-Smirnov (K-S) test for equality of continuous distributions.

The K-S test uses a statistic based on the maximum deviation of the empirical distribution of sample data points from the distribution expected under the null hypothesis. For one-sample tests evaluating the null hypothesis that a set of sample data points follow a given distribution, the test statistic is \(D_n=\sup_x |F_n(x)-F(x)|\), where \(F\) is the expected distribution and \(F_n\) is the empirical distribution of the \(n\) sample data points. The distribution of \(D_n\) is estimated using a method based on [1] with certain quick decisions for extreme values given in [2].

Two-sample tests are also supported, evaluating the null hypothesis that the two samples x and y come from the same underlying distribution. In this case, the test statistic is \(D_{n,m}=\sup_t | F_n(t)-F_m(t)|\) where \(n\) is the length of x, \(m\) is the length of y, \(F_n\) is the empirical distribution that puts mass \(1/n\) at each of the values in x and \(F_m\) is the empirical distribution of the y values. The default 2-sample test method, kolmogorovSmirnovTest(double[], double[]) works as follows:

If the product of the sample sizes is less than 10000 and the sample data contains ties, random jitter is added to the sample data to break ties before applying the algorithm above. Alternatively, the bootstrap(double[], double[], int, boolean) method, modeled after ks.boot in the R Matching package [3], can be used if ties are known to be present in the data.

In the two-sample case, \(D_{n,m}\) has a discrete distribution. This makes the p-value associated with the null hypothesis \(H_0 : D_{n,m} \ge d \) differ from \(H_0 : D_{n,m} > d \) by the mass of the observed value \(d\). To distinguish these, the two-sample tests use a boolean strict parameter. This parameter is ignored for large samples.

The methods used by the 2-sample default implementation are also exposed directly:

References:


Note that [1] contains an error in computing h, refer to MATH-437 for details.

Since
3.3

Field Summary

Modifier and TypeField and Description
protected static final double
protected static final int
LARGE_SAMPLE_PRODUCT

When product of sample sizes exceeds this value, 2-sample K-S test uses asymptotic distribution to compute the p-value.

protected static final int
MAXIMUM_PARTIAL_SUM_COUNT

Bound on the number of partial sums in ksSum(double, double, int)

protected static final int
MONTE_CARLO_ITERATIONS

Deprecated
Default number of iterations used by monteCarloP(double, int, int, boolean, int).
protected static final double
PG_SUM_RELATIVE_ERROR

Convergence criterion for the sums in #pelzGood(double, double, int)}

private final RandomGenerator
rng

Random data generator used by monteCarloP(double, int, int, boolean, int)

protected static final int
SMALL_SAMPLE_PRODUCT

Deprecated
No longer used.

Constructor Summary

AccessConstructor and Description
public
KolmogorovSmirnovTest()

Construct a KolmogorovSmirnovTest instance with a default random data generator.

public
KolmogorovSmirnovTest(RandomGenerator
random data generator used by monteCarloP(double, int, int, boolean, int)
rng
)
Deprecated

Construct a KolmogorovSmirnovTest with the provided random data generator.

Method Summary

Modifier and TypeMethod and Description
public double

Returns:

approximate probability that a randomly selected m-n partition of m + n generates \(D_{n,m}\) greater than d
approximateP
(double
D-statistic value
d
,
int
first sample size
n
,
int
second sample size
m
)

Uses the Kolmogorov-Smirnov distribution to approximate \(P(D_{n,m} > d)\) where \(D_{n,m}\) is the 2-sample Kolmogorov-Smirnov statistic.

public double

Returns:

estimated p-value
bootstrap
(double[]
first sample
x
,
double[]
second sample
y
,
int
number of bootstrap resampling iterations
iterations
,
boolean
whether or not the null hypothesis is expressed as a strict inequality
strict
)

Estimates the p-value of a two-sample Kolmogorov-Smirnov test evaluating the null hypothesis that x and y are samples drawn from the same probability distribution.

public double

Returns:

estimated p-value
bootstrap
(double[]
first sample
x
,
double[]
second sample
y
,
int
number of bootstrap resampling iterations
iterations
)

Computes bootstrap(x, y, iterations, true).

private static int

Returns:

C(i,j) for given m, n, c
c
(int
first path parameter
i
,
int
second path paramter
j
,
int
first sample size
m
,
int
second sample size
n
,
long cmn, boolean
whether or not the null hypothesis uses strict inequality
strict
)

The function C(i, j) defined in [4] (class javadoc), formula (5.5).

private static long

Returns:

the integral d-statistic in the range [0, n*m]
calculateIntegralD
(double
a d-statistic in the range [0, 1]
d
,
int
first sample size
n
,
int
second sample size
m
,
boolean
whether the returned value divided by (n*m) is allowed to be equal to d
strict
)

Given a d-statistic in the range [0, 1] and the two sample sizes n and m, an integral d-statistic in the range [0, n*m] is calculated, that can be used for comparison with other integral d-statistics.

public double

Returns:

\(P(D_n < d)\)
cdf
(double
statistic
d
,
int
sample size
n
)

Calculates \(P(D_n < d)\) using the method described in [1] with quick decisions for extreme values given in [2] (see above).

public double

Returns:

\(P(D_n < d)\)
cdf
(double
statistic
d
,
int
sample size
n
,
boolean
whether the probability should be calculated exact using org.apache.commons.math3.fraction.BigFraction everywhere at the expense of very slow execution time, or if double should be used convenient places to gain speed. Almost never choose true in real applications unless you are very sure; true is almost solely for verification purposes.
exact
)

Calculates P(D_n < d) using method described in [1] with quick decisions for extreme values given in [2] (see above).

public double

Returns:

\(P(D_n < d)\)
cdfExact
(double
statistic
d
,
int
sample size
n
)

Calculates P(D_n < d).

private void
checkArray(double[]
array to test
array
)

Verifies that array has length at least 2.

private FieldMatrix<BigFraction>

Returns:

H matrix
createExactH
(double
statistic
d
,
int
sample size
n
)

Creates H of size m x m as described in [1] (see above).

private RealMatrix

Returns:

H matrix
createRoundedH
(double
statistic
d
,
int
sample size
n
)

Creates H of size m x m as described in [1] (see above) using double-precision.

private double

Returns:

the two-sided probability of \(P(D_n < d)\)
exactK
(double
statistic
d
,
int
sample size
n
)

Calculates the exact value of P(D_n < d) using the method described in [1] (reference in class javadoc above) and org.apache.commons.math3.fraction.BigFraction (see above).

public double

Returns:

probability that a randomly selected m-n partition of m + n generates \(D_{n,m}\) greater than (resp. greater than or equal to) d
exactP
(double
D-statistic value
d
,
int
first sample size
n
,
int
second sample size
m
,
boolean
whether or not the probability to compute is expressed as a strict inequality
strict
)

Computes \(P(D_{n,m} > d)\) if strict is true; otherwise \(P(D_{n,m} \ge d)\), where \(D_{n,m}\) is the 2-sample Kolmogorov-Smirnov statistic.

pack-priv static void
fillBooleanArrayRandomlyWithFixedNumberTrueValues(final boolean[]
boolean array
b
,
final int
number of true values the boolean array should finally have
numberOfTrueValues
,
final RandomGenerator
random data generator
rng
)

Fills a boolean array randomly with a fixed number of true values.

private static void
fixTies(double[]
first sample
x
,
double[]
second sample
y
)

If there are no ties in the combined dataset formed from x and y, this method is a no-op.

private static boolean

Returns:

true if x and y together contain ties
hasTies
(double[]
first sample
x
,
double[]
second sample
y
)

Returns true iff there are ties in the combined sample formed from x and y.

private long

Returns:

test statistic \(n m D_{n,m}\) used to evaluate the null hypothesis that x and y represent samples from the same underlying distribution
integralKolmogorovSmirnovStatistic
(double[]
first sample
x
,
double[]
second sample
y
)

Computes the two-sample Kolmogorov-Smirnov test statistic, \(D_{n,m}=\sup_x |F_n(x)-F_m(x)|\) where \(n\) is the length of x, \(m\) is the length of y, \(F_n\) is the empirical distribution that puts mass \(1/n\) at each of the values in x and \(F_m\) is the empirical distribution of the y values.

private double

Returns:

proportion of randomly generated m-n partitions of m + n that result in \(D_{n,m}\) greater than or equal to d/(n*m))
integralMonteCarloP
(final long
integral D-statistic
d
,
final int
first sample size
n
,
final int
second sample size
m
,
final int
number of random partitions to generate
iterations
)

Uses Monte Carlo simulation to approximate \(P(D_{n,m} >= d/(n*m))\) where \(D_{n,m}\) is the 2-sample Kolmogorov-Smirnov statistic.

private static void
jitter(double[]
input/output data array - entries overwritten by the method
data
,
RealDistribution
probability distribution to sample for jitter values
dist
)

Adds random jitter to data using deviates sampled from dist.

public double

Returns:

Kolmogorov-Smirnov statistic \(D_n\)
kolmogorovSmirnovStatistic
(RealDistribution
reference distribution
distribution
,
double[]
sample being evaluated
data
)

Computes the one-sample Kolmogorov-Smirnov test statistic, \(D_n=\sup_x |F_n(x)-F(x)|\) where \(F\) is the distribution (cdf) function associated with distribution, \(n\) is the length of data and \(F_n\) is the empirical distribution that puts mass \(1/n\) at each of the values in data.

public double

Returns:

test statistic \(D_{n,m}\) used to evaluate the null hypothesis that x and y represent samples from the same underlying distribution
kolmogorovSmirnovStatistic
(double[]
first sample
x
,
double[]
second sample
y
)

Computes the two-sample Kolmogorov-Smirnov test statistic, \(D_{n,m}=\sup_x |F_n(x)-F_m(x)|\) where \(n\) is the length of x, \(m\) is the length of y, \(F_n\) is the empirical distribution that puts mass \(1/n\) at each of the values in x and \(F_m\) is the empirical distribution of the y values.

public double

Returns:

the p-value associated with the null hypothesis that data is a sample from distribution
kolmogorovSmirnovTest
(RealDistribution
reference distribution
distribution
,
double[]
sample being being evaluated
data
,
boolean
whether or not to force exact computation of the p-value
exact
)

Computes the p-value, or observed significance level, of a one-sample Kolmogorov-Smirnov test evaluating the null hypothesis that data conforms to distribution.

public double

Returns:

p-value associated with the null hypothesis that x and y represent samples from the same distribution
kolmogorovSmirnovTest
(double[]
first sample dataset
x
,
double[]
second sample dataset
y
,
boolean
whether or not the probability to compute is expressed as a strict inequality (ignored for large samples)
strict
)

Computes the p-value, or observed significance level, of a two-sample Kolmogorov-Smirnov test evaluating the null hypothesis that x and y are samples drawn from the same probability distribution.

public double

Returns:

p-value associated with the null hypothesis that x and y represent samples from the same distribution
kolmogorovSmirnovTest
(double[]
first sample dataset
x
,
double[]
second sample dataset
y
)

Computes the p-value, or observed significance level, of a two-sample Kolmogorov-Smirnov test evaluating the null hypothesis that x and y are samples drawn from the same probability distribution.

public double

Returns:

the p-value associated with the null hypothesis that data is a sample from distribution
kolmogorovSmirnovTest
(RealDistribution
reference distribution
distribution
,
double[]
sample being being evaluated
data
)

Computes the p-value, or observed significance level, of a one-sample Kolmogorov-Smirnov test evaluating the null hypothesis that data conforms to distribution.

public boolean

Returns:

true iff the null hypothesis that data is a sample from distribution can be rejected with confidence 1 - alpha
kolmogorovSmirnovTest
(RealDistribution
reference distribution
distribution
,
double[]
sample being being evaluated
data
,
double
significance level of the test
alpha
)

Performs a Kolmogorov-Smirnov test evaluating the null hypothesis that data conforms to distribution.

public double

Returns:

Kolmogorov sum evaluated at t
ksSum
(double
argument
t
,
double
Cauchy criterion for partial sums
tolerance
,
int
maximum number of partial sums to compute
maxIterations
)

Computes \( 1 + 2 \sum_{i=1}^\infty (-1)^i e^{-2 i^2 t^2} \) stopping when successive partial sums are within tolerance of one another, or when maxIterations partial sums have been computed.

public double

Returns:

proportion of randomly generated m-n partitions of m + n that result in \(D_{n,m}\) greater than (resp. greater than or equal to) d
monteCarloP
(final double
D-statistic value
d
,
final int
first sample size
n
,
final int
second sample size
m
,
final boolean
whether or not the probability to compute is expressed as a strict inequality
strict
,
final int
number of random partitions to generate
iterations
)

Uses Monte Carlo simulation to approximate \(P(D_{n,m} > d)\) where \(D_{n,m}\) is the 2-sample Kolmogorov-Smirnov statistic.

private static double

Returns:

number or paths to (i, j) from (0,0) representing D-values as large as c for given m, n
n
(int
first path parameter
i
,
int
second path parameter
j
,
int
first sample size
m
,
int
second sample size
n
,
long cnm, boolean
whether or not the null hypothesis uses strict inequality
strict
)

The function N(i, j) defined in [4] (class javadoc).

public double

Returns:

\(P(D_n < d)\)
pelzGood
(double
value of d-statistic (x in [2])
d
,
int
sample size
n
)

Computes the Pelz-Good approximation for \(P(D_n < d)\) as described in [2] in the class javadoc.

private double

Returns:

\(P(D_n < d)\)
roundedK
(double
statistic
d
,
int
sample size
n
)

Calculates P(D_n < d) using method described in [1] and doubles (see above).

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

KS_SUM_CAUCHY_CRITERIONback to summary
protected static final double KS_SUM_CAUCHY_CRITERION

Convergence criterion for ksSum(double, double, int)

LARGE_SAMPLE_PRODUCTback to summary
protected static final int LARGE_SAMPLE_PRODUCT

When product of sample sizes exceeds this value, 2-sample K-S test uses asymptotic distribution to compute the p-value.

MAXIMUM_PARTIAL_SUM_COUNTback to summary
protected static final int MAXIMUM_PARTIAL_SUM_COUNT

Bound on the number of partial sums in ksSum(double, double, int)

MONTE_CARLO_ITERATIONSback to summary
protected static final int MONTE_CARLO_ITERATIONS

Deprecated

Default number of iterations used by monteCarloP(double, int, int, boolean, int). Deprecated as of version 3.6, as this method is no longer needed.

Annotations
@Deprecated
PG_SUM_RELATIVE_ERRORback to summary
protected static final double PG_SUM_RELATIVE_ERROR

Convergence criterion for the sums in #pelzGood(double, double, int)}

rngback to summary
private final RandomGenerator rng

Random data generator used by monteCarloP(double, int, int, boolean, int)

SMALL_SAMPLE_PRODUCTback to summary
protected static final int SMALL_SAMPLE_PRODUCT

Deprecated

No longer used.

Annotations
@Deprecated

Constructor Detail

KolmogorovSmirnovTestback to summary
public KolmogorovSmirnovTest()

Construct a KolmogorovSmirnovTest instance with a default random data generator.

KolmogorovSmirnovTestback to summary
public KolmogorovSmirnovTest(RandomGenerator rng)

Deprecated

Construct a KolmogorovSmirnovTest with the provided random data generator. The #monteCarloP(double, int, int, boolean, int) that uses the generator supplied to this constructor is deprecated as of version 3.6.

Parameters
rng:RandomGenerator

random data generator used by monteCarloP(double, int, int, boolean, int)

Annotations
@Deprecated

Method Detail

approximatePback to summary
public double approximateP(double d, int n, int m)

Uses the Kolmogorov-Smirnov distribution to approximate \(P(D_{n,m} > d)\) where \(D_{n,m}\) is the 2-sample Kolmogorov-Smirnov statistic. See kolmogorovSmirnovStatistic(double[], double[]) for the definition of \(D_{n,m}\).

Specifically, what is returned is \(1 - k(d \sqrt{mn / (m + n)})\) where \(k(t) = 1 + 2 \sum_{i=1}^\infty (-1)^i e^{-2 i^2 t^2}\). See ksSum(double, double, int) for details on how convergence of the sum is determined. This implementation passes ksSum 1.0E-20 as tolerance and 100000 as maxIterations.

Parameters
d:double

D-statistic value

n:int

first sample size

m:int

second sample size

Returns:double

approximate probability that a randomly selected m-n partition of m + n generates \(D_{n,m}\) greater than d

bootstrapback to summary
public double bootstrap(double[] x, double[] y, int iterations, boolean strict)

Estimates the p-value of a two-sample Kolmogorov-Smirnov test evaluating the null hypothesis that x and y are samples drawn from the same probability distribution. This method estimates the p-value by repeatedly sampling sets of size x.length and y.length from the empirical distribution of the combined sample. When strict is true, this is equivalent to the algorithm implemented in the R function ks.boot, described in

Jasjeet S. Sekhon. 2011. 'Multivariate and Propensity Score Matching
Software with Automated Balance Optimization: The Matching package for R.'
Journal of Statistical Software, 42(7): 1-52.
Parameters
x:double[]

first sample

y:double[]

second sample

iterations:int

number of bootstrap resampling iterations

strict:boolean

whether or not the null hypothesis is expressed as a strict inequality

Returns:double

estimated p-value

bootstrapback to summary
public double bootstrap(double[] x, double[] y, int iterations)

Computes bootstrap(x, y, iterations, true). This is equivalent to ks.boot(x,y, nboots=iterations) using the R Matching package function. See #bootstrap(double[], double[], int, boolean).

Parameters
x:double[]

first sample

y:double[]

second sample

iterations:int

number of bootstrap resampling iterations

Returns:double

estimated p-value

cback to summary
private static int c(int i, int j, int m, int n, long cmn, boolean strict)

The function C(i, j) defined in [4] (class javadoc), formula (5.5). defined to return 1 if |i/n - j/m| <= c; 0 otherwise. Here c is scaled up and recoded as a long to avoid rounding errors in comparison tests, so what is actually tested is |im - jn| <= cmn.

Parameters
i:int

first path parameter

j:int

second path paramter

m:int

first sample size

n:int

second sample size

cmn:long

integral D-statistic (see calculateIntegralD(double, int, int, boolean))

strict:boolean

whether or not the null hypothesis uses strict inequality

Returns:int

C(i,j) for given m, n, c

calculateIntegralDback to summary
private static long calculateIntegralD(double d, int n, int m, boolean strict)

Given a d-statistic in the range [0, 1] and the two sample sizes n and m, an integral d-statistic in the range [0, n*m] is calculated, that can be used for comparison with other integral d-statistics. Depending whether strict is true or not, the returned value divided by (n*m) is greater than (resp greater than or equal to) the given d value (allowing some tolerance).

Parameters
d:double

a d-statistic in the range [0, 1]

n:int

first sample size

m:int

second sample size

strict:boolean

whether the returned value divided by (n*m) is allowed to be equal to d

Returns:long

the integral d-statistic in the range [0, n*m]

cdfback to summary
public double cdf(double d, int n) throws MathArithmeticException

Calculates \(P(D_n < d)\) using the method described in [1] with quick decisions for extreme values given in [2] (see above). The result is not exact as with cdfExact(double, int) because calculations are based on double rather than org.apache.commons.math3.fraction.BigFraction.

Parameters
d:double

statistic

n:int

sample size

Returns:double

\(P(D_n < d)\)

Exceptions
MathArithmeticException:
if algorithm fails to convert h to a org.apache.commons.math3.fraction.BigFraction in expressing d as \((k - h) / m\) for integer k, m and \(0 \le h < 1\)
cdfback to summary
public double cdf(double d, int n, boolean exact) throws MathArithmeticException

Calculates P(D_n < d) using method described in [1] with quick decisions for extreme values given in [2] (see above).

Parameters
d:double

statistic

n:int

sample size

exact:boolean

whether the probability should be calculated exact using org.apache.commons.math3.fraction.BigFraction everywhere at the expense of very slow execution time, or if double should be used convenient places to gain speed. Almost never choose true in real applications unless you are very sure; true is almost solely for verification purposes.

Returns:double

\(P(D_n < d)\)

Exceptions
MathArithmeticException:
if algorithm fails to convert h to a org.apache.commons.math3.fraction.BigFraction in expressing d as \((k - h) / m\) for integer k, m and \(0 \le h < 1\).
cdfExactback to summary
public double cdfExact(double d, int n) throws MathArithmeticException

Calculates P(D_n < d). The result is exact in the sense that BigFraction/BigReal is used everywhere at the expense of very slow execution time. Almost never choose this in real applications unless you are very sure; this is almost solely for verification purposes. Normally, you would choose cdf(double, int). See the class javadoc for definitions and algorithm description.

Parameters
d:double

statistic

n:int

sample size

Returns:double

\(P(D_n < d)\)

Exceptions
MathArithmeticException:
if the algorithm fails to convert h to a org.apache.commons.math3.fraction.BigFraction in expressing d as \((k - h) / m\) for integer k, m and \(0 \le h < 1\)
checkArrayback to summary
private void checkArray(double[] array)

Verifies that array has length at least 2.

Parameters
array:double[]

array to test

Exceptions
NullArgumentException:
if array is null
InsufficientDataException:
if array is too short
createExactHback to summary
private FieldMatrix<BigFraction> createExactH(double d, int n) throws NumberIsTooLargeException, FractionConversionException

Creates H of size m x m as described in [1] (see above).

Parameters
d:double

statistic

n:int

sample size

Returns:FieldMatrix<BigFraction>

H matrix

Exceptions
NumberIsTooLargeException:
if fractional part is greater than 1
FractionConversionException:
if algorithm fails to convert h to a org.apache.commons.math3.fraction.BigFraction in expressing d as \((k - h) / m\) for integer k, m and \(0 <= h < 1\).
createRoundedHback to summary
private RealMatrix createRoundedH(double d, int n) throws NumberIsTooLargeException

Creates H of size m x m as described in [1] (see above) using double-precision.

Parameters
d:double

statistic

n:int

sample size

Returns:RealMatrix

H matrix

Exceptions
NumberIsTooLargeException:
if fractional part is greater than 1
exactKback to summary
private double exactK(double d, int n) throws MathArithmeticException

Calculates the exact value of P(D_n < d) using the method described in [1] (reference in class javadoc above) and org.apache.commons.math3.fraction.BigFraction (see above).

Parameters
d:double

statistic

n:int

sample size

Returns:double

the two-sided probability of \(P(D_n < d)\)

Exceptions
MathArithmeticException:
if algorithm fails to convert h to a org.apache.commons.math3.fraction.BigFraction in expressing d as \((k - h) / m\) for integer k, m and \(0 \le h < 1\).
exactPback to summary
public double exactP(double d, int n, int m, boolean strict)

Computes \(P(D_{n,m} > d)\) if strict is true; otherwise \(P(D_{n,m} \ge d)\), where \(D_{n,m}\) is the 2-sample Kolmogorov-Smirnov statistic. See kolmogorovSmirnovStatistic(double[], double[]) for the definition of \(D_{n,m}\).

The returned probability is exact, implemented by unwinding the recursive function definitions presented in [4] (class javadoc).

Parameters
d:double

D-statistic value

n:int

first sample size

m:int

second sample size

strict:boolean

whether or not the probability to compute is expressed as a strict inequality

Returns:double

probability that a randomly selected m-n partition of m + n generates \(D_{n,m}\) greater than (resp. greater than or equal to) d

fillBooleanArrayRandomlyWithFixedNumberTrueValuesback to summary
pack-priv static void fillBooleanArrayRandomlyWithFixedNumberTrueValues(final boolean[] b, final int numberOfTrueValues, final RandomGenerator rng)

Fills a boolean array randomly with a fixed number of true values. The method uses a simplified version of the Fisher-Yates shuffle algorithm. By processing first the true values followed by the remaining false values less random numbers need to be generated. The method is optimized for the case that the number of true values is larger than or equal to the number of false values.

Parameters
b:boolean[]

boolean array

numberOfTrueValues:int

number of true values the boolean array should finally have

rng:RandomGenerator

random data generator

fixTiesback to summary
private static void fixTies(double[] x, double[] y)

If there are no ties in the combined dataset formed from x and y, this method is a no-op. If there are ties, a uniform random deviate in (-minDelta / 2, minDelta / 2) - {0} is added to each value in x and y, where minDelta is the minimum difference between unequal values in the combined sample. A fixed seed is used to generate the jitter, so repeated activations with the same input arrays result in the same values.

Note

if there are ties in the data, this method overwrites the data in x and y with the jittered values.

Parameters
x:double[]

first sample

y:double[]

second sample

hasTiesback to summary
private static boolean hasTies(double[] x, double[] y)

Returns true iff there are ties in the combined sample formed from x and y.

Parameters
x:double[]

first sample

y:double[]

second sample

Returns:boolean

true if x and y together contain ties

integralKolmogorovSmirnovStatisticback to summary
private long integralKolmogorovSmirnovStatistic(double[] x, double[] y)

Computes the two-sample Kolmogorov-Smirnov test statistic, \(D_{n,m}=\sup_x |F_n(x)-F_m(x)|\) where \(n\) is the length of x, \(m\) is the length of y, \(F_n\) is the empirical distribution that puts mass \(1/n\) at each of the values in x and \(F_m\) is the empirical distribution of the y values. Finally \(n m D_{n,m}\) is returned as long value.

Parameters
x:double[]

first sample

y:double[]

second sample

Returns:long

test statistic \(n m D_{n,m}\) used to evaluate the null hypothesis that x and y represent samples from the same underlying distribution

Exceptions
InsufficientDataException:
if either x or y does not have length at least 2
NullArgumentException:
if either x or y is null
integralMonteCarloPback to summary
private double integralMonteCarloP(final long d, final int n, final int m, final int iterations)

Uses Monte Carlo simulation to approximate \(P(D_{n,m} >= d/(n*m))\) where \(D_{n,m}\) is the 2-sample Kolmogorov-Smirnov statistic.

Here d is the D-statistic represented as long value. The real D-statistic is obtained by dividing d by n*m. See also monteCarloP(double, int, int, boolean, int).

Parameters
d:long

integral D-statistic

n:int

first sample size

m:int

second sample size

iterations:int

number of random partitions to generate

Returns:double

proportion of randomly generated m-n partitions of m + n that result in \(D_{n,m}\) greater than or equal to d/(n*m))

jitterback to summary
private static void jitter(double[] data, RealDistribution dist)

Adds random jitter to data using deviates sampled from dist.

Note that jitter is applied in-place - i.e., the array values are overwritten with the result of applying jitter.

Parameters
data:double[]

input/output data array - entries overwritten by the method

dist:RealDistribution

probability distribution to sample for jitter values

Exceptions
NullPointerException:
if either of the parameters is null
kolmogorovSmirnovStatisticback to summary
public double kolmogorovSmirnovStatistic(RealDistribution distribution, double[] data)

Computes the one-sample Kolmogorov-Smirnov test statistic, \(D_n=\sup_x |F_n(x)-F(x)|\) where \(F\) is the distribution (cdf) function associated with distribution, \(n\) is the length of data and \(F_n\) is the empirical distribution that puts mass \(1/n\) at each of the values in data.

Parameters
distribution:RealDistribution

reference distribution

data:double[]

sample being evaluated

Returns:double

Kolmogorov-Smirnov statistic \(D_n\)

Exceptions
InsufficientDataException:
if data does not have length at least 2
NullArgumentException:
if data is null
kolmogorovSmirnovStatisticback to summary
public double kolmogorovSmirnovStatistic(double[] x, double[] y)

Computes the two-sample Kolmogorov-Smirnov test statistic, \(D_{n,m}=\sup_x |F_n(x)-F_m(x)|\) where \(n\) is the length of x, \(m\) is the length of y, \(F_n\) is the empirical distribution that puts mass \(1/n\) at each of the values in x and \(F_m\) is the empirical distribution of the y values.

Parameters
x:double[]

first sample

y:double[]

second sample

Returns:double

test statistic \(D_{n,m}\) used to evaluate the null hypothesis that x and y represent samples from the same underlying distribution

Exceptions
InsufficientDataException:
if either x or y does not have length at least 2
NullArgumentException:
if either x or y is null
kolmogorovSmirnovTestback to summary
public double kolmogorovSmirnovTest(RealDistribution distribution, double[] data, boolean exact)

Computes the p-value, or observed significance level, of a one-sample Kolmogorov-Smirnov test evaluating the null hypothesis that data conforms to distribution. If exact is true, the distribution used to compute the p-value is computed using extended precision. See cdfExact(double, int).

Parameters
distribution:RealDistribution

reference distribution

data:double[]

sample being being evaluated

exact:boolean

whether or not to force exact computation of the p-value

Returns:double

the p-value associated with the null hypothesis that data is a sample from distribution

Exceptions
InsufficientDataException:
if data does not have length at least 2
NullArgumentException:
if data is null
kolmogorovSmirnovTestback to summary
public double kolmogorovSmirnovTest(double[] x, double[] y, boolean strict)

Computes the p-value, or observed significance level, of a two-sample Kolmogorov-Smirnov test evaluating the null hypothesis that x and y are samples drawn from the same probability distribution. Specifically, what is returned is an estimate of the probability that the kolmogorovSmirnovStatistic(double[], double[]) associated with a randomly selected partition of the combined sample into subsamples of sizes x.length and y.length will strictly exceed (if strict is true) or be at least as large as strict = false) as kolmogorovSmirnovStatistic(x, y).

  • For small samples (where the product of the sample sizes is less than 10000), the exact p-value is computed using the method presented in [4], implemented in exactP(double, int, int, boolean).
  • When the product of the sample sizes exceeds 10000, the asymptotic distribution of \(D_{n,m}\) is used. See approximateP(double, int, int) for details on the approximation.

If x.length * y.length < 10000 and the combined set of values in x and y contains ties, random jitter is added to x and y to break ties before computing \(D_{n,m}\) and the p-value. The jitter is uniformly distributed on (-minDelta / 2, minDelta / 2) where minDelta is the smallest pairwise difference between values in the combined sample.

If ties are known to be present in the data, bootstrap(double[], double[], int, boolean) may be used as an alternative method for estimating the p-value.

Parameters
x:double[]

first sample dataset

y:double[]

second sample dataset

strict:boolean

whether or not the probability to compute is expressed as a strict inequality (ignored for large samples)

Returns:double

p-value associated with the null hypothesis that x and y represent samples from the same distribution

Exceptions
InsufficientDataException:
if either x or y does not have length at least 2
NullArgumentException:
if either x or y is null
See Also
bootstrap(double[], double[], int, boolean)
kolmogorovSmirnovTestback to summary
public double kolmogorovSmirnovTest(double[] x, double[] y)

Computes the p-value, or observed significance level, of a two-sample Kolmogorov-Smirnov test evaluating the null hypothesis that x and y are samples drawn from the same probability distribution. Assumes the strict form of the inequality used to compute the p-value. See kolmogorovSmirnovTest(RealDistribution, double[], boolean).

Parameters
x:double[]

first sample dataset

y:double[]

second sample dataset

Returns:double

p-value associated with the null hypothesis that x and y represent samples from the same distribution

Exceptions
InsufficientDataException:
if either x or y does not have length at least 2
NullArgumentException:
if either x or y is null
kolmogorovSmirnovTestback to summary
public double kolmogorovSmirnovTest(RealDistribution distribution, double[] data)

Computes the p-value, or observed significance level, of a one-sample Kolmogorov-Smirnov test evaluating the null hypothesis that data conforms to distribution.

Parameters
distribution:RealDistribution

reference distribution

data:double[]

sample being being evaluated

Returns:double

the p-value associated with the null hypothesis that data is a sample from distribution

Exceptions
InsufficientDataException:
if data does not have length at least 2
NullArgumentException:
if data is null
kolmogorovSmirnovTestback to summary
public boolean kolmogorovSmirnovTest(RealDistribution distribution, double[] data, double alpha)

Performs a Kolmogorov-Smirnov test evaluating the null hypothesis that data conforms to distribution.

Parameters
distribution:RealDistribution

reference distribution

data:double[]

sample being being evaluated

alpha:double

significance level of the test

Returns:boolean

true iff the null hypothesis that data is a sample from distribution can be rejected with confidence 1 - alpha

Exceptions
InsufficientDataException:
if data does not have length at least 2
NullArgumentException:
if data is null
ksSumback to summary
public double ksSum(double t, double tolerance, int maxIterations)

Computes \( 1 + 2 \sum_{i=1}^\infty (-1)^i e^{-2 i^2 t^2} \) stopping when successive partial sums are within tolerance of one another, or when maxIterations partial sums have been computed. If the sum does not converge before maxIterations iterations a TooManyIterationsException is thrown.

Parameters
t:double

argument

tolerance:double

Cauchy criterion for partial sums

maxIterations:int

maximum number of partial sums to compute

Returns:double

Kolmogorov sum evaluated at t

Exceptions
TooManyIterationsException:
if the series does not converge
monteCarloPback to summary
public double monteCarloP(final double d, final int n, final int m, final boolean strict, final int iterations)

Uses Monte Carlo simulation to approximate \(P(D_{n,m} > d)\) where \(D_{n,m}\) is the 2-sample Kolmogorov-Smirnov statistic. See kolmogorovSmirnovStatistic(double[], double[]) for the definition of \(D_{n,m}\).

The simulation generates iterations random partitions of m + n into an n set and an m set, computing \(D_{n,m}\) for each partition and returning the proportion of values that are greater than d, or greater than or equal to d if strict is false.

Parameters
d:double

D-statistic value

n:int

first sample size

m:int

second sample size

strict:boolean

whether or not the probability to compute is expressed as a strict inequality

iterations:int

number of random partitions to generate

Returns:double

proportion of randomly generated m-n partitions of m + n that result in \(D_{n,m}\) greater than (resp. greater than or equal to) d

nback to summary
private static double n(int i, int j, int m, int n, long cnm, boolean strict)

The function N(i, j) defined in [4] (class javadoc). Returns the number of paths over the lattice {(i,j) : 0 <= i <= n, 0 <= j <= m} from (0,0) to (i,j) satisfying C(h,k, m, n, c) = 1 for each (h,k) on the path. The return value is integral, but subject to overflow, so it is maintained and returned as a double.

Parameters
i:int

first path parameter

j:int

second path parameter

m:int

first sample size

n:int

second sample size

cnm:long

integral D-statistic (see calculateIntegralD(double, int, int, boolean))

strict:boolean

whether or not the null hypothesis uses strict inequality

Returns:double

number or paths to (i, j) from (0,0) representing D-values as large as c for given m, n

pelzGoodback to summary
public double pelzGood(double d, int n)

Computes the Pelz-Good approximation for \(P(D_n < d)\) as described in [2] in the class javadoc.

Parameters
d:double

value of d-statistic (x in [2])

n:int

sample size

Returns:double

\(P(D_n < d)\)

Since
3.4
roundedKback to summary
private double roundedK(double d, int n)

Calculates P(D_n < d) using method described in [1] and doubles (see above).

Parameters
d:double

statistic

n:int

sample size

Returns:double

\(P(D_n < d)\)