The K-S test uses a statistic based on the maximum deviation of the empirical distribution of sample data points from the distribution expected under the null hypothesis. For one-sample tests evaluating the null hypothesis that a set of sample data points follow a given distribution, the test statistic is \(D_n=\sup_x |F_n(x)-F(x)|\), where \(F\) is the expected distribution and \(F_n\) is the empirical distribution of the \(n\) sample data points. The distribution of \(D_n\) is estimated using a method based on [1] with certain quick decisions for extreme values given in [2].
Two-sample tests are also supported, evaluating the null hypothesis that the two samples
x and y come from the same underlying distribution. In this case, the test
statistic is \(D_{n,m}=\sup_t | F_n(t)-F_m(t)|\) where \(n\) is the length of x, \(m\) is
the length of y, \(F_n\) is the empirical distribution that puts mass \(1/n\) at each of
the values in x and \(F_m\) is the empirical distribution of the y values. The
default 2-sample test method, kolmogorovSmirnovTest(double[], double[]) works as
follows:
approximateP(double, int, int) for details on
the approximation.
If the product of the sample sizes is less than 10000 and the sample
data contains ties, random jitter is added to the sample data to break ties before applying
the algorithm above. Alternatively, the bootstrap(double[], double[], int, boolean)
method, modeled after ks.boot
in the R Matching package [3], can be used if ties are known to be present in the data.
In the two-sample case, \(D_{n,m}\) has a discrete distribution. This makes the p-value
associated with the null hypothesis \(H_0 : D_{n,m} \ge d \) differ from \(H_0 : D_{n,m} > d \)
by the mass of the observed value \(d\). To distinguish these, the two-sample tests use a boolean
strict parameter. This parameter is ignored for large samples.
The methods used by the 2-sample default implementation are also exposed directly:
exactP(double, int, int, boolean) computes exact 2-sample p-valuesapproximateP(double, int, int) uses the asymptotic distribution The boolean
arguments in the first two methods allow the probability used to estimate the p-value to be
expressed using strict or non-strict inequality. See
kolmogorovSmirnovTest(double[], double[], boolean).References:
| Modifier and Type | Field and Description |
|---|---|
| protected static final double | KS_SUM_CAUCHY_CRITERION
Convergence criterion for |
| protected static final int | LARGE_SAMPLE_PRODUCT
When product of sample sizes exceeds this value, 2-sample K-S test uses asymptotic distribution to compute the p-value. |
| protected static final int | MAXIMUM_PARTIAL_SUM_COUNT
Bound on the number of partial sums in |
| protected static final int | MONTE_CARLO_ITERATIONS
Deprecated
Default number of iterations used by monteCarloP(double, int, int, boolean, int).
|
| protected static final double | PG_SUM_RELATIVE_ERROR
Convergence criterion for the sums in #pelzGood(double, double, int)} |
| private final RandomGenerator | rng
Random data generator used by |
| protected static final int | SMALL_SAMPLE_PRODUCT
Deprecated
No longer used.
|
| Access | Constructor and Description |
|---|---|
| public | KolmogorovSmirnovTest()
Construct a KolmogorovSmirnovTest instance with a default random data generator. |
| public | KolmogorovSmirnovTest(RandomGenerator
random data generator used by rng)monteCarloP(double, int, int, boolean, int)
Deprecated
Construct a KolmogorovSmirnovTest with the provided random data generator. |
| Modifier and Type | Method and Description |
|---|---|
| public double | Returns: approximate probability that a randomly selected m-n partition of m + n generates \(D_{n,m}\) greater thandD-statistic value d, int first sample size n, int second sample size m)Uses the Kolmogorov-Smirnov distribution to approximate \(P(D_{n,m} > d)\) where \(D_{n,m}\) is the 2-sample Kolmogorov-Smirnov statistic. |
| public double | Returns: estimated p-valuefirst sample x, double[] second sample y, int number of bootstrap resampling iterations iterations, boolean whether or not the null hypothesis is expressed as a strict inequality strict)Estimates the p-value of a two-sample
Kolmogorov-Smirnov test
evaluating the null hypothesis that |
| public double | Returns: estimated p-valuefirst sample x, double[] second sample y, int number of bootstrap resampling iterations iterations)Computes |
| private static int | Returns: C(i,j) for given m, n, cfirst path parameter i, int second path paramter j, int first sample size m, int second sample size n, long integral D-statistic (see cmn, boolean calculateIntegralD(double, int, int, boolean))whether or not the null hypothesis uses strict inequality strict)The function C(i, j) defined in [4] (class javadoc), formula (5.5). |
| private static long | Returns: the integral d-statistic in the range [0, n*m]a d-statistic in the range [0, 1] d, int first sample size n, int second sample size m, boolean whether the returned value divided by (n*m) is allowed to be equal to d strict)Given a d-statistic in the range [0, 1] and the two sample sizes n and m, an integral d-statistic in the range [0, n*m] is calculated, that can be used for comparison with other integral d-statistics. |
| public double | Returns: \(P(D_n < d)\)statistic d, int sample size n)Calculates \(P(D_n < d)\) using the method described in [1] with quick decisions for extreme values given in [2] (see above). |
| public double | Returns: \(P(D_n < d)\)statistic d, int sample size n, boolean whether the probability should be calculated exact using
exact)org. everywhere at the expense of
very slow execution time, or if double should be used convenient places to
gain speed. Almost never choose true in real applications unless you are very
sure; true is almost solely for verification purposes.Calculates |
| public double | |
| private void | |
| private FieldMatrix | Returns: H matrixstatistic d, int sample size n)Creates |
| private RealMatrix | Returns: H matrixstatistic d, int sample size n)Creates |
| private double | Returns: the two-sided probability of \(P(D_n < d)\)statistic d, int sample size n)Calculates the exact value of |
| public double | Returns: probability that a randomly selected m-n partition of m + n generates \(D_{n,m}\) greater than (resp. greater than or equal to)dD-statistic value d, int first sample size n, int second sample size m, boolean whether or not the probability to compute is expressed as a strict inequality strict)Computes \(P(D_{n,m} > d)\) if |
| pack-priv static void | fillBooleanArrayRandomlyWithFixedNumberTrueValues(final boolean[]
boolean array b, final int number of numberOfTrueValues, final RandomGenerator true values the boolean array should finally haverandom data generator rng)Fills a boolean array randomly with a fixed number of |
| private static void | fixTies(double[]
first sample x, double[] second sample y)If there are no ties in the combined dataset formed from x and y, this method is a no-op. |
| private static boolean | Returns: true if x and y together contain tiesfirst sample x, double[] second sample y)Returns true iff there are ties in the combined sample formed from x and y. |
| private long | Returns: test statistic \(n m D_{n,m}\) used to evaluate the null hypothesis thatx and
y represent samples from the same underlying distributionfirst sample x, double[] second sample y)Computes the two-sample Kolmogorov-Smirnov test statistic, \(D_{n,m}=\sup_x |F_n(x)-F_m(x)|\)
where \(n\) is the length of |
| private double | Returns: proportion of randomly generated m-n partitions of m + n that result in \(D_{n,m}\) greater than or equal tod/(n*m))integral D-statistic d, final int first sample size n, final int second sample size m, final int number of random partitions to generate iterations)Uses Monte Carlo simulation to approximate \(P(D_{n,m} >= d/(n*m))\) where \(D_{n,m}\) is the 2-sample Kolmogorov-Smirnov statistic. |
| private static void | jitter(double[]
input/output data array - entries overwritten by the method data, RealDistribution probability distribution to sample for jitter values dist)Adds random jitter to |
| public double | Returns: Kolmogorov-Smirnov statistic \(D_n\)reference distribution distribution, double[] sample being evaluated data)Computes the one-sample Kolmogorov-Smirnov test statistic, \(D_n=\sup_x |F_n(x)-F(x)|\) where
\(F\) is the distribution (cdf) function associated with |
| public double | Returns: test statistic \(D_{n,m}\) used to evaluate the null hypothesis thatx and
y represent samples from the same underlying distributionfirst sample x, double[] second sample y)Computes the two-sample Kolmogorov-Smirnov test statistic, \(D_{n,m}=\sup_x |F_n(x)-F_m(x)|\)
where \(n\) is the length of |
| public double | Returns: the p-value associated with the null hypothesis thatdata is a sample from
distributionreference distribution distribution, double[] sample being being evaluated data, boolean whether or not to force exact computation of the p-value exact)Computes the p-value, or observed significance level, of a one-sample Kolmogorov-Smirnov test
evaluating the null hypothesis that |
| public double | Returns: p-value associated with the null hypothesis thatx and y represent
samples from the same distributionfirst sample dataset x, double[] second sample dataset y, boolean whether or not the probability to compute is expressed as a strict inequality
(ignored for large samples) strict)Computes the p-value, or observed significance level, of a two-sample Kolmogorov-Smirnov test
evaluating the null hypothesis that |
| public double | Returns: p-value associated with the null hypothesis thatx and y represent
samples from the same distributionfirst sample dataset x, double[] second sample dataset y)Computes the p-value, or observed significance level, of a two-sample Kolmogorov-Smirnov test
evaluating the null hypothesis that |
| public double | Returns: the p-value associated with the null hypothesis thatdata is a sample from
distributionreference distribution distribution, double[] sample being being evaluated data)Computes the p-value, or observed significance level, of a one-sample Kolmogorov-Smirnov test
evaluating the null hypothesis that |
| public boolean | Returns: true iff the null hypothesis thatdata is a sample from distribution
can be rejected with confidence 1 - alphareference distribution distribution, double[] sample being being evaluated data, double significance level of the test alpha)Performs a Kolmogorov-Smirnov
test evaluating the null hypothesis that |
| public double | Returns: Kolmogorov sum evaluated at targument t, double Cauchy criterion for partial sums tolerance, int maximum number of partial sums to compute maxIterations)Computes \( 1 + 2 \sum_{i=1}^\infty (-1)^i e^{-2 i^2 t^2} \) stopping when successive partial
sums are within |
| public double | Returns: proportion of randomly generated m-n partitions of m + n that result in \(D_{n,m}\) greater than (resp. greater than or equal to)dD-statistic value d, final int first sample size n, final int second sample size m, final boolean whether or not the probability to compute is expressed as a strict inequality strict, final int number of random partitions to generate iterations)Uses Monte Carlo simulation to approximate \(P(D_{n,m} > d)\) where \(D_{n,m}\) is the 2-sample Kolmogorov-Smirnov statistic. |
| private static double | Returns: number or paths to (i, j) from (0,0) representing D-values as large as c for given m, nfirst path parameter i, int second path parameter j, int first sample size m, int second sample size n, long integral D-statistic (see cnm, boolean calculateIntegralD(double, int, int, boolean))whether or not the null hypothesis uses strict inequality strict)The function N(i, j) defined in [4] (class javadoc). |
| public double | Returns: \(P(D_n < d)\)value of d-statistic (x in [2]) d, int sample size n)Computes the Pelz-Good approximation for \(P(D_n < d)\) as described in [2] in the class javadoc. |
| private double | Returns: \(P(D_n < d)\)statistic d, int sample size n)Calculates |
| KS_SUM_CAUCHY_CRITERION | back to summary |
|---|---|
| protected static final double KS_SUM_CAUCHY_CRITERION Convergence criterion for | |
| LARGE_SAMPLE_PRODUCT | back to summary |
|---|---|
| protected static final int LARGE_SAMPLE_PRODUCT When product of sample sizes exceeds this value, 2-sample K-S test uses asymptotic distribution to compute the p-value. | |
| MAXIMUM_PARTIAL_SUM_COUNT | back to summary |
|---|---|
| protected static final int MAXIMUM_PARTIAL_SUM_COUNT Bound on the number of partial sums in | |
| MONTE_CARLO_ITERATIONS | back to summary |
|---|---|
| protected static final int MONTE_CARLO_ITERATIONS
Deprecated
Default number of iterations used by | |
| PG_SUM_RELATIVE_ERROR | back to summary |
|---|---|
| protected static final double PG_SUM_RELATIVE_ERROR Convergence criterion for the sums in #pelzGood(double, double, int)} | |
| rng | back to summary |
|---|---|
| private final RandomGenerator rng Random data generator used by | |
| SMALL_SAMPLE_PRODUCT | back to summary |
|---|---|
| protected static final int SMALL_SAMPLE_PRODUCT
Deprecated
No longer used. | |
| KolmogorovSmirnovTest | back to summary |
|---|---|
| public KolmogorovSmirnovTest() Construct a KolmogorovSmirnovTest instance with a default random data generator. | |
| KolmogorovSmirnovTest | back to summary |
|---|---|
| public KolmogorovSmirnovTest(RandomGenerator rng)
Deprecated
Construct a KolmogorovSmirnovTest with the provided random data generator. The #monteCarloP(double, int, int, boolean, int) that uses the generator supplied to this constructor is deprecated as of version 3.6.
| |
| approximateP | back to summary |
|---|---|
| public double approximateP(double d, int n, int m) Uses the Kolmogorov-Smirnov distribution to approximate \(P(D_{n,m} > d)\) where \(D_{n,m}\)
is the 2-sample Kolmogorov-Smirnov statistic. See
Specifically, what is returned is \(1 - k(d \sqrt{mn / (m + n)})\) where \(k(t) = 1 + 2
\sum_{i=1}^\infty (-1)^i e^{-2 i^2 t^2}\). See
| |
| bootstrap | back to summary |
|---|---|
| public double bootstrap(double[] x, double[] y, int iterations, boolean strict) Estimates the p-value of a two-sample
Kolmogorov-Smirnov test
evaluating the null hypothesis that Jasjeet S. Sekhon. 2011. 'Multivariate and Propensity Score Matching Software with Automated Balance Optimization: The Matching package for R.' Journal of Statistical Software, 42(7): 1-52.
| |
| bootstrap | back to summary |
|---|---|
| public double bootstrap(double[] x, double[] y, int iterations) Computes
| |
| c | back to summary |
|---|---|
| private static int c(int i, int j, int m, int n, long cmn, boolean strict) The function C(i, j) defined in [4] (class javadoc), formula (5.5). defined to return 1 if |i/n - j/m| <= c; 0 otherwise. Here c is scaled up and recoded as a long to avoid rounding errors in comparison tests, so what is actually tested is |im - jn| <= cmn.
| |
| calculateIntegralD | back to summary |
|---|---|
| private static long calculateIntegralD(double d, int n, int m, boolean strict) Given a d-statistic in the range [0, 1] and the two sample sizes n and m,
an integral d-statistic in the range [0, n*m] is calculated, that can be used for
comparison with other integral d-statistics. Depending whether
| |
| cdf | back to summary |
|---|---|
| public double cdf(double d, int n) throws MathArithmeticException Calculates \(P(D_n < d)\) using the method described in [1] with quick decisions for extreme
values given in [2] (see above). The result is not exact as with
| |
| cdf | back to summary |
|---|---|
| public double cdf(double d, int n, boolean exact) throws MathArithmeticException Calculates
| |
| cdfExact | back to summary |
|---|---|
| public double cdfExact(double d, int n) throws MathArithmeticException Calculates
| |
| checkArray | back to summary |
|---|---|
| private void checkArray(double[] array) Verifies that
| |
| createExactH | back to summary |
|---|---|
| private FieldMatrix Creates
| |
| createRoundedH | back to summary |
|---|---|
| private RealMatrix createRoundedH(double d, int n) throws NumberIsTooLargeException Creates
| |
| exactK | back to summary |
|---|---|
| private double exactK(double d, int n) throws MathArithmeticException Calculates the exact value of
| |
| exactP | back to summary |
|---|---|
| public double exactP(double d, int n, int m, boolean strict) Computes \(P(D_{n,m} > d)\) if The returned probability is exact, implemented by unwinding the recursive function definitions presented in [4] (class javadoc).
| |
| fillBooleanArrayRandomlyWithFixedNumberTrueValues | back to summary |
|---|---|
| pack-priv static void fillBooleanArrayRandomlyWithFixedNumberTrueValues(final boolean[] b, final int numberOfTrueValues, final RandomGenerator rng) Fills a boolean array randomly with a fixed number of
| |
| fixTies | back to summary |
|---|---|
| private static void fixTies(double[] x, double[] y) If there are no ties in the combined dataset formed from x and y, this method is a no-op. If there are ties, a uniform random deviate in (-minDelta / 2, minDelta / 2) - {0} is added to each value in x and y, where minDelta is the minimum difference between unequal values in the combined sample. A fixed seed is used to generate the jitter, so repeated activations with the same input arrays result in the same values. Note if there are ties in the data, this method overwrites the data in x and y with the jittered values.
| |
| hasTies | back to summary |
|---|---|
| private static boolean hasTies(double[] x, double[] y) Returns true iff there are ties in the combined sample formed from x and y.
| |
| integralKolmogorovSmirnovStatistic | back to summary |
|---|---|
| private long integralKolmogorovSmirnovStatistic(double[] x, double[] y) Computes the two-sample Kolmogorov-Smirnov test statistic, \(D_{n,m}=\sup_x |F_n(x)-F_m(x)|\)
where \(n\) is the length of
| |
| integralMonteCarloP | back to summary |
|---|---|
| private double integralMonteCarloP(final long d, final int n, final int m, final int iterations) Uses Monte Carlo simulation to approximate \(P(D_{n,m} >= d/(n*m))\) where \(D_{n,m}\) is the 2-sample Kolmogorov-Smirnov statistic.
Here d is the D-statistic represented as long value.
The real D-statistic is obtained by dividing d by n*m.
See also
| |
| jitter | back to summary |
|---|---|
| private static void jitter(double[] data, RealDistribution dist) Adds random jitter to Note that jitter is applied in-place - i.e., the array values are overwritten with the result of applying jitter.
| |
| kolmogorovSmirnovStatistic | back to summary |
|---|---|
| public double kolmogorovSmirnovStatistic(RealDistribution distribution, double[] data) Computes the one-sample Kolmogorov-Smirnov test statistic, \(D_n=\sup_x |F_n(x)-F(x)|\) where
\(F\) is the distribution (cdf) function associated with
| |
| kolmogorovSmirnovStatistic | back to summary |
|---|---|
| public double kolmogorovSmirnovStatistic(double[] x, double[] y) Computes the two-sample Kolmogorov-Smirnov test statistic, \(D_{n,m}=\sup_x |F_n(x)-F_m(x)|\)
where \(n\) is the length of
| |
| kolmogorovSmirnovTest | back to summary |
|---|---|
| public double kolmogorovSmirnovTest(RealDistribution distribution, double[] data, boolean exact) Computes the p-value, or observed significance level, of a one-sample Kolmogorov-Smirnov test
evaluating the null hypothesis that
| |
| kolmogorovSmirnovTest | back to summary |
|---|---|
| public double kolmogorovSmirnovTest(double[] x, double[] y, boolean strict) Computes the p-value, or observed significance level, of a two-sample Kolmogorov-Smirnov test
evaluating the null hypothesis that
If
If ties are known to be present in the data,
| |
| kolmogorovSmirnovTest | back to summary |
|---|---|
| public double kolmogorovSmirnovTest(double[] x, double[] y) Computes the p-value, or observed significance level, of a two-sample Kolmogorov-Smirnov test
evaluating the null hypothesis that
| |
| kolmogorovSmirnovTest | back to summary |
|---|---|
| public double kolmogorovSmirnovTest(RealDistribution distribution, double[] data) Computes the p-value, or observed significance level, of a one-sample Kolmogorov-Smirnov test
evaluating the null hypothesis that
| |
| kolmogorovSmirnovTest | back to summary |
|---|---|
| public boolean kolmogorovSmirnovTest(RealDistribution distribution, double[] data, double alpha) Performs a Kolmogorov-Smirnov
test evaluating the null hypothesis that
| |
| ksSum | back to summary |
|---|---|
| public double ksSum(double t, double tolerance, int maxIterations) Computes \( 1 + 2 \sum_{i=1}^\infty (-1)^i e^{-2 i^2 t^2} \) stopping when successive partial
sums are within
| |
| monteCarloP | back to summary |
|---|---|
| public double monteCarloP(final double d, final int n, final int m, final boolean strict, final int iterations) Uses Monte Carlo simulation to approximate \(P(D_{n,m} > d)\) where \(D_{n,m}\) is the
2-sample Kolmogorov-Smirnov statistic. See
The simulation generates
| |
| n | back to summary |
|---|---|
| private static double n(int i, int j, int m, int n, long cnm, boolean strict) The function N(i, j) defined in [4] (class javadoc). Returns the number of paths over the lattice {(i,j) : 0 <= i <= n, 0 <= j <= m} from (0,0) to (i,j) satisfying C(h,k, m, n, c) = 1 for each (h,k) on the path. The return value is integral, but subject to overflow, so it is maintained and returned as a double.
| |
| pelzGood | back to summary |
|---|---|
| public double pelzGood(double d, int n) Computes the Pelz-Good approximation for \(P(D_n < d)\) as described in [2] in the class javadoc.
| |
| roundedK | back to summary |
|---|---|
| private double roundedK(double d, int n) Calculates
| |