Top Description Constructors Methods
org.apache.commons.math3.stat.inference

public Class ChiSquareTest

extends Object
Class Inheritance
Imports
org.apache.commons.math3.distribution.ChiSquaredDistribution, org.apache.commons.math3.exception.DimensionMismatchException, .MaxCountExceededException, .NotPositiveException, .NotStrictlyPositiveException, .NullArgumentException, .OutOfRangeException, .ZeroException, org.apache.commons.math3.exception.util.LocalizedFormats, org.apache.commons.math3.util.FastMath, .MathArrays

Implements Chi-Square test statistics.

This implementation handles both known and unknown distributions.

Two samples tests can be used when the distribution is unknown a priori but provided by one sample, or when the hypothesis under test is that the two samples come from the same underlying distribution.

Constructor Summary

AccessConstructor and Description
public
ChiSquareTest()

Construct a ChiSquareTest

Method Summary

Modifier and TypeMethod and Description
private void
checkArray(final long[][]
input 2-way table to check
in
)

Checks to make sure that the input long[][] array is rectangular, has at least 2 rows and 2 columns, and has all non-negative entries.

public double

Returns:

chiSquare test statistic
chiSquare
(final double[]
array of expected frequency counts
expected
,
final long[]
array of observed frequency counts
observed
)

Computes the Chi-Square statistic comparing observed and expected frequency counts.

public double

Returns:

chiSquare test statistic
chiSquare
(final long[][]
array representation of 2-way table
counts
)

Computes the Chi-Square statistic associated with a chi-square test of independence based on the input counts array, viewed as a two-way table.

public double

Returns:

chiSquare test statistic
chiSquareDataSetsComparison
(long[]
array of observed frequency counts of the first data set
observed1
,
long[]
array of observed frequency counts of the second data set
observed2
)

Computes a Chi-Square two sample test statistic comparing bin frequency counts in observed1 and observed2.

public double

Returns:

p-value
chiSquareTest
(final double[]
array of expected frequency counts
expected
,
final long[]
array of observed frequency counts
observed
)

Returns the observed significance level, or p-value, associated with a Chi-square goodness of fit test comparing the observed frequency counts to those in the expected array.

public boolean

Returns:

true iff null hypothesis can be rejected with confidence 1 - alpha
chiSquareTest
(final double[]
array of expected frequency counts
expected
,
final long[]
array of observed frequency counts
observed
,
final double
significance level of the test
alpha
)

Performs a Chi-square goodness of fit test evaluating the null hypothesis that the observed counts conform to the frequency distribution described by the expected counts, with significance level alpha.

public double

Returns:

p-value
chiSquareTest
(final long[][]
array representation of 2-way table
counts
)

Returns the observed significance level, or p-value, associated with a chi-square test of independence based on the input counts array, viewed as a two-way table.

public boolean

Returns:

true iff null hypothesis can be rejected with confidence 1 - alpha
chiSquareTest
(final long[][]
array representation of 2-way table
counts
,
final double
significance level of the test
alpha
)

Performs a chi-square test of independence evaluating the null hypothesis that the classifications represented by the counts in the columns of the input 2-way table are independent of the rows, with significance level alpha.

public double

Returns:

p-value
chiSquareTestDataSetsComparison
(long[]
array of observed frequency counts of the first data set
observed1
,
long[]
array of observed frequency counts of the second data set
observed2
)

Returns the observed significance level, or p-value, associated with a Chi-Square two sample test comparing bin frequency counts in observed1 and observed2.

public boolean

Returns:

true iff null hypothesis can be rejected with confidence 1 - alpha
chiSquareTestDataSetsComparison
(final long[]
array of observed frequency counts of the first data set
observed1
,
final long[]
array of observed frequency counts of the second data set
observed2
,
final double
significance level of the test
alpha
)

Performs a Chi-Square two sample test comparing two binned data sets.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Constructor Detail

ChiSquareTestback to summary
public ChiSquareTest()

Construct a ChiSquareTest

Method Detail

checkArrayback to summary
private void checkArray(final long[][] in) throws NullArgumentException, DimensionMismatchException, NotPositiveException

Checks to make sure that the input long[][] array is rectangular, has at least 2 rows and 2 columns, and has all non-negative entries.

Parameters
in:long[][]

input 2-way table to check

Exceptions
NullArgumentException:
if the array is null
DimensionMismatchException:
if the array is not valid
NotPositiveException:
if the array contains any negative entries
chiSquareback to summary
public double chiSquare(final double[] expected, final long[] observed) throws NotPositiveException, NotStrictlyPositiveException, DimensionMismatchException

Computes the Chi-Square statistic comparing observed and expected frequency counts.

This statistic can be used to perform a Chi-Square test evaluating the null hypothesis that the observed counts follow the expected distribution.

Preconditions:

  • Expected counts must all be positive.
  • Observed counts must all be ≥ 0.
  • The observed and expected arrays must have the same length and their common length must be at least 2.

If any of the preconditions are not met, an IllegalArgumentException is thrown.

Note

This implementation rescales the expected array if necessary to ensure that the sum of the expected and observed counts are equal.

Parameters
expected:double[]

array of expected frequency counts

observed:long[]

array of observed frequency counts

Returns:double

chiSquare test statistic

Exceptions
NotPositiveException:
if observed has negative entries
NotStrictlyPositiveException:
if expected has entries that are not strictly positive
DimensionMismatchException:
if the arrays length is less than 2
chiSquareback to summary
public double chiSquare(final long[][] counts) throws NullArgumentException, NotPositiveException, DimensionMismatchException

Computes the Chi-Square statistic associated with a chi-square test of independence based on the input counts array, viewed as a two-way table.

The rows of the 2-way table are count[0], ... , count[count.length - 1]

Preconditions:

  • All counts must be ≥ 0.
  • The count array must be rectangular (i.e. all count[i] subarrays must have the same length).
  • The 2-way table represented by counts must have at least 2 columns and at least 2 rows.

If any of the preconditions are not met, an IllegalArgumentException is thrown.

Parameters
counts:long[][]

array representation of 2-way table

Returns:double

chiSquare test statistic

Exceptions
NullArgumentException:
if the array is null
NotPositiveException:
if counts has negative entries
DimensionMismatchException:
if the array is not rectangular
chiSquareDataSetsComparisonback to summary
public double chiSquareDataSetsComparison(long[] observed1, long[] observed2) throws DimensionMismatchException, NotPositiveException, ZeroException

Computes a Chi-Square two sample test statistic comparing bin frequency counts in observed1 and observed2. The sums of frequency counts in the two samples are not required to be the same. The formula used to compute the test statistic is

∑[(K * observed1[i] - observed2[i]/K)2 / (observed1[i] + observed2[i])] where
K = &sqrt;[&sum(observed2 / ∑(observed1)]

This statistic can be used to perform a Chi-Square test evaluating the null hypothesis that both observed counts follow the same distribution.

Preconditions:

  • Observed counts must be non-negative.
  • Observed counts for a specific bin must not both be zero.
  • Observed counts for a specific sample must not all be 0.
  • The arrays observed1 and observed2 must have the same length and their common length must be at least 2.

If any of the preconditions are not met, an IllegalArgumentException is thrown.

Parameters
observed1:long[]

array of observed frequency counts of the first data set

observed2:long[]

array of observed frequency counts of the second data set

Returns:double

chiSquare test statistic

Exceptions
DimensionMismatchException:
the the length of the arrays does not match
NotPositiveException:
if any entries in observed1 or observed2 are negative
ZeroException:
if either all counts of observed1 or observed2 are zero, or if the count at some index is zero for both arrays
Since
1.2
chiSquareTestback to summary
public double chiSquareTest(final double[] expected, final long[] observed) throws NotPositiveException, NotStrictlyPositiveException, DimensionMismatchException, MaxCountExceededException

Returns the observed significance level, or p-value, associated with a Chi-square goodness of fit test comparing the observed frequency counts to those in the expected array.

The number returned is the smallest significance level at which one can reject the null hypothesis that the observed counts conform to the frequency distribution described by the expected counts.

Preconditions:

  • Expected counts must all be positive.
  • Observed counts must all be ≥ 0.
  • The observed and expected arrays must have the same length and their common length must be at least 2.

If any of the preconditions are not met, an IllegalArgumentException is thrown.

Note

This implementation rescales the expected array if necessary to ensure that the sum of the expected and observed counts are equal.

Parameters
expected:double[]

array of expected frequency counts

observed:long[]

array of observed frequency counts

Returns:double

p-value

Exceptions
NotPositiveException:
if observed has negative entries
NotStrictlyPositiveException:
if expected has entries that are not strictly positive
DimensionMismatchException:
if the arrays length is less than 2
MaxCountExceededException:
if an error occurs computing the p-value
chiSquareTestback to summary
public boolean chiSquareTest(final double[] expected, final long[] observed, final double alpha) throws NotPositiveException, NotStrictlyPositiveException, DimensionMismatchException, OutOfRangeException, MaxCountExceededException

Performs a Chi-square goodness of fit test evaluating the null hypothesis that the observed counts conform to the frequency distribution described by the expected counts, with significance level alpha. Returns true iff the null hypothesis can be rejected with 100 * (1 - alpha) percent confidence.

Example:
To test the hypothesis that observed follows expected at the 99% level, use

chiSquareTest(expected, observed, 0.01)

Preconditions:

  • Expected counts must all be positive.
  • Observed counts must all be ≥ 0.
  • The observed and expected arrays must have the same length and their common length must be at least 2.
  • 0 < alpha < 0.5

If any of the preconditions are not met, an IllegalArgumentException is thrown.

Note

This implementation rescales the expected array if necessary to ensure that the sum of the expected and observed counts are equal.

Parameters
expected:double[]

array of expected frequency counts

observed:long[]

array of observed frequency counts

alpha:double

significance level of the test

Returns:boolean

true iff null hypothesis can be rejected with confidence 1 - alpha

Exceptions
NotPositiveException:
if observed has negative entries
NotStrictlyPositiveException:
if expected has entries that are not strictly positive
DimensionMismatchException:
if the arrays length is less than 2
OutOfRangeException:
if alpha is not in the range (0, 0.5]
MaxCountExceededException:
if an error occurs computing the p-value
chiSquareTestback to summary
public double chiSquareTest(final long[][] counts) throws NullArgumentException, DimensionMismatchException, NotPositiveException, MaxCountExceededException

Returns the observed significance level, or p-value, associated with a chi-square test of independence based on the input counts array, viewed as a two-way table.

The rows of the 2-way table are count[0], ... , count[count.length - 1]

Preconditions:

  • All counts must be ≥ 0.
  • The count array must be rectangular (i.e. all count[i] subarrays must have the same length).
  • The 2-way table represented by counts must have at least 2 columns and at least 2 rows.

If any of the preconditions are not met, an IllegalArgumentException is thrown.

Parameters
counts:long[][]

array representation of 2-way table

Returns:double

p-value

Exceptions
NullArgumentException:
if the array is null
DimensionMismatchException:
if the array is not rectangular
NotPositiveException:
if counts has negative entries
MaxCountExceededException:
if an error occurs computing the p-value
chiSquareTestback to summary
public boolean chiSquareTest(final long[][] counts, final double alpha) throws NullArgumentException, DimensionMismatchException, NotPositiveException, OutOfRangeException, MaxCountExceededException

Performs a chi-square test of independence evaluating the null hypothesis that the classifications represented by the counts in the columns of the input 2-way table are independent of the rows, with significance level alpha. Returns true iff the null hypothesis can be rejected with 100 * (1 - alpha) percent confidence.

The rows of the 2-way table are count[0], ... , count[count.length - 1]

Example:
To test the null hypothesis that the counts in count[0], ... , count[count.length - 1] all correspond to the same underlying probability distribution at the 99% level, use

chiSquareTest(counts, 0.01)

Preconditions:

  • All counts must be ≥ 0.
  • The count array must be rectangular (i.e. all count[i] subarrays must have the same length).
  • The 2-way table represented by counts must have at least 2 columns and at least 2 rows.

If any of the preconditions are not met, an IllegalArgumentException is thrown.

Parameters
counts:long[][]

array representation of 2-way table

alpha:double

significance level of the test

Returns:boolean

true iff null hypothesis can be rejected with confidence 1 - alpha

Exceptions
NullArgumentException:
if the array is null
DimensionMismatchException:
if the array is not rectangular
NotPositiveException:
if counts has any negative entries
OutOfRangeException:
if alpha is not in the range (0, 0.5]
MaxCountExceededException:
if an error occurs computing the p-value
chiSquareTestDataSetsComparisonback to summary
public double chiSquareTestDataSetsComparison(long[] observed1, long[] observed2) throws DimensionMismatchException, NotPositiveException, ZeroException, MaxCountExceededException

Returns the observed significance level, or p-value, associated with a Chi-Square two sample test comparing bin frequency counts in observed1 and observed2.

The number returned is the smallest significance level at which one can reject the null hypothesis that the observed counts conform to the same distribution.

See chiSquareDataSetsComparison(long[], long[]) for details on the formula used to compute the test statistic. The degrees of of freedom used to perform the test is one less than the common length of the input observed count arrays.

Preconditions:
  • Observed counts must be non-negative.
  • Observed counts for a specific bin must not both be zero.
  • Observed counts for a specific sample must not all be 0.
  • The arrays observed1 and observed2 must have the same length and their common length must be at least 2.

If any of the preconditions are not met, an IllegalArgumentException is thrown.

Parameters
observed1:long[]

array of observed frequency counts of the first data set

observed2:long[]

array of observed frequency counts of the second data set

Returns:double

p-value

Exceptions
DimensionMismatchException:
the the length of the arrays does not match
NotPositiveException:
if any entries in observed1 or observed2 are negative
ZeroException:
if either all counts of observed1 or observed2 are zero, or if the count at the same index is zero for both arrays
MaxCountExceededException:
if an error occurs computing the p-value
Since
1.2
chiSquareTestDataSetsComparisonback to summary
public boolean chiSquareTestDataSetsComparison(final long[] observed1, final long[] observed2, final double alpha) throws DimensionMismatchException, NotPositiveException, ZeroException, OutOfRangeException, MaxCountExceededException

Performs a Chi-Square two sample test comparing two binned data sets. The test evaluates the null hypothesis that the two lists of observed counts conform to the same frequency distribution, with significance level alpha. Returns true iff the null hypothesis can be rejected with 100 * (1 - alpha) percent confidence.

See chiSquareDataSetsComparison(long[], long[]) for details on the formula used to compute the Chisquare statistic used in the test. The degrees of of freedom used to perform the test is one less than the common length of the input observed count arrays.

Preconditions:
  • Observed counts must be non-negative.
  • Observed counts for a specific bin must not both be zero.
  • Observed counts for a specific sample must not all be 0.
  • The arrays observed1 and observed2 must have the same length and their common length must be at least 2.
  • 0 < alpha < 0.5

If any of the preconditions are not met, an IllegalArgumentException is thrown.

Parameters
observed1:long[]

array of observed frequency counts of the first data set

observed2:long[]

array of observed frequency counts of the second data set

alpha:double

significance level of the test

Returns:boolean

true iff null hypothesis can be rejected with confidence 1 - alpha

Exceptions
DimensionMismatchException:
the the length of the arrays does not match
NotPositiveException:
if any entries in observed1 or observed2 are negative
ZeroException:
if either all counts of observed1 or observed2 are zero, or if the count at the same index is zero for both arrays
OutOfRangeException:
if alpha is not in the range (0, 0.5]
MaxCountExceededException:
if an error occurs performing the test
Since
1.2