Top Description Fields Constructors Methods
org.apache.commons.math3.stat.regression

public Class OLSMultipleLinearRegression

extends AbstractMultipleLinearRegression
Class Inheritance
Imports
org.apache.commons.math3.exception.MathIllegalArgumentException, org.apache.commons.math3.linear.Array2DRowRealMatrix, .LUDecomposition, .QRDecomposition, .RealMatrix, .RealVector, org.apache.commons.math3.stat.StatUtils, org.apache.commons.math3.stat.descriptive.moment.SecondMoment

Implements ordinary least squares (OLS) to estimate the parameters of a multiple linear regression model.

The regression coefficients, b, satisfy the normal equations: 

 XT X b = XT y

To solve the normal equations, this implementation uses QR decomposition of the X matrix. (See QRDecomposition for details on the decomposition algorithm.) The X matrix, also known as the design matrix, has rows corresponding to sample observations and columns corresponding to independent variables. When the model is estimated using an intercept term (i.e. when isNoIntercept is false as it is by default), the X matrix includes an initial column identically equal to 1. We solve the normal equations as follows: 

 XTX b = XT y
(QR)T (QR) b = (QR)Ty
RT (QTQ) R b = RT QT y
RT R b = RT QT y
(RT)-1 RT R b = (RT)-1 RT QT y
R b = QT y

Given Q and R, the last equation is solved by back-substitution.

Since
2.0

Field Summary

Modifier and TypeField and Description
private QRDecomposition
qr

Cached QR decomposition of X matrix

private final double
threshold

Singularity threshold for QR decomposition

Constructor Summary

AccessConstructor and Description
public
OLSMultipleLinearRegression()

Create an empty OLSMultipleLinearRegression instance.

public
OLSMultipleLinearRegression(final double
the singularity threshold
threshold)

Create an empty OLSMultipleLinearRegression instance, using the given singularity threshold for the QR decomposition.

Method Summary

Modifier and TypeMethod and Description
public double

Returns:

adjusted R-Squared statistic
calculateAdjustedRSquared()

Returns the adjusted R-squared statistic, defined by the formula 

R2adj = 1 - [SSR (n - 1)] / [SSTO (n - p)]
protected RealVector

Returns:

beta
calculateBeta()

Implements abstract org.apache.commons.math3.stat.regression.AbstractMultipleLinearRegression.calculateBeta.

Calculates the regression coefficients using OLS.

protected RealMatrix

Returns:

The beta variance-covariance matrix
calculateBetaVariance()

Implements abstract org.apache.commons.math3.stat.regression.AbstractMultipleLinearRegression.calculateBetaVariance.

Calculates the variance-covariance matrix of the regression parameters.

public RealMatrix

Returns:

the hat matrix
calculateHat()

Compute the "hat" matrix.

public double

Returns:

residual sum of squares
calculateResidualSumOfSquares()

Returns the sum of squared residuals.

public double

Returns:

R-square statistic
calculateRSquared()

Returns the R-Squared statistic, defined by the formula 

R2 = 1 - SSR / SSTO
public double

Returns:

SSTO - the total sum of squares
calculateTotalSumOfSquares()

Returns the sum of squared deviations of Y from its mean.

public void
newSampleData(double[]
the [n,1] array representing the y sample
y, double[][]
the [n,k] array representing the x sample
x)

Loads model x and y sample data, overriding any previous sample.

public void
newSampleData(double[]
input data array
data, int
number of observations (rows)
nobs, int
number of independent variables (columns, not counting y)
nvars)

Overrides org.apache.commons.math3.stat.regression.AbstractMultipleLinearRegression.newSampleData.

Loads model x and y sample data from a flat input array, overriding any previous sample.

protected void
newXSampleData(double[][]
the rectangular array representing the x sample
x)

Overrides org.apache.commons.math3.stat.regression.AbstractMultipleLinearRegression.newXSampleData.

Loads new x sample data, overriding any previous data.

Inherited from org.apache.commons.math3.stat.regression.AbstractMultipleLinearRegression:
calculateErrorVariancecalculateResidualscalculateYVarianceestimateErrorVarianceestimateRegressandVarianceestimateRegressionParametersestimateRegressionParametersStandardErrorsestimateRegressionParametersVarianceestimateRegressionStandardErrorestimateResidualsgetXgetYisNoInterceptnewYSampleDatasetNoInterceptvalidateCovarianceDatavalidateSampleData

Field Detail

qrback to summary
private QRDecomposition qr

Cached QR decomposition of X matrix

thresholdback to summary
private final double threshold

Singularity threshold for QR decomposition

Constructor Detail

OLSMultipleLinearRegressionback to summary
public OLSMultipleLinearRegression()

Create an empty OLSMultipleLinearRegression instance.

OLSMultipleLinearRegressionback to summary
public OLSMultipleLinearRegression(final double threshold)

Create an empty OLSMultipleLinearRegression instance, using the given singularity threshold for the QR decomposition.

Parameters
threshold:double

the singularity threshold

Since
3.3

Method Detail

calculateAdjustedRSquaredback to summary
public double calculateAdjustedRSquared()

Returns the adjusted R-squared statistic, defined by the formula 

R2adj = 1 - [SSR (n - 1)] / [SSTO (n - p)]
where SSR is the sum of squared residuals, SSTO is the total sum of squares, n is the number of observations and p is the number of parameters estimated (including the intercept).

If the regression is estimated without an intercept term, what is returned is 

 1 - (1 - calculateRSquared()) * (n / (n - p))

If there is no variance in y, i.e., SSTO = 0, NaN is returned.

Returns:double

adjusted R-Squared statistic

Exceptions
NullPointerException:
if the sample has not been set
SingularMatrixException:
if the design matrix is singular
Since
2.2
See Also
isNoIntercept()
calculateBetaback to summary
protected RealVector calculateBeta()

Implements abstract org.apache.commons.math3.stat.regression.AbstractMultipleLinearRegression.calculateBeta.

Calculates the regression coefficients using OLS.

Data for the model must have been successfully loaded using one of the newSampleData methods before invoking this method; otherwise a NullPointerException will be thrown.

Returns:RealVector

beta

Annotations
@Override
Exceptions
SingularMatrixException:
if the design matrix is singular
NullPointerException:
if the data for the model have not been loaded
calculateBetaVarianceback to summary
protected RealMatrix calculateBetaVariance()

Implements abstract org.apache.commons.math3.stat.regression.AbstractMultipleLinearRegression.calculateBetaVariance.

Calculates the variance-covariance matrix of the regression parameters.

Var(b) = (XTX)-1

Uses QR decomposition to reduce (XTX)-1 to (RTR)-1, with only the top p rows of R included, where p = the length of the beta vector.

Data for the model must have been successfully loaded using one of the newSampleData methods before invoking this method; otherwise a NullPointerException will be thrown.

Returns:RealMatrix

The beta variance-covariance matrix

Annotations
@Override
Exceptions
SingularMatrixException:
if the design matrix is singular
NullPointerException:
if the data for the model have not been loaded
calculateHatback to summary
public RealMatrix calculateHat()

Compute the "hat" matrix.

The hat matrix is defined in terms of the design matrix X by X(XTX)-1XT

The implementation here uses the QR decomposition to compute the hat matrix as Q IpQT where Ip is the p-dimensional identity matrix augmented by 0's. This computational formula is from "The Hat Matrix in Regression and ANOVA", David C. Hoaglin and Roy E. Welsch, The American Statistician, Vol. 32, No. 1 (Feb., 1978), pp. 17-22.

Data for the model must have been successfully loaded using one of the newSampleData methods before invoking this method; otherwise a NullPointerException will be thrown.

Returns:RealMatrix

the hat matrix

Exceptions
NullPointerException:
unless method newSampleData has been called beforehand.
calculateResidualSumOfSquaresback to summary
public double calculateResidualSumOfSquares()

Returns the sum of squared residuals.

Returns:double

residual sum of squares

Exceptions
SingularMatrixException:
if the design matrix is singular
NullPointerException:
if the data for the model have not been loaded
Since
2.2
calculateRSquaredback to summary
public double calculateRSquared()

Returns the R-Squared statistic, defined by the formula 

R2 = 1 - SSR / SSTO
where SSR is the sum of squared residuals and SSTO is the total sum of squares

If there is no variance in y, i.e., SSTO = 0, NaN is returned.

Returns:double

R-square statistic

Exceptions
NullPointerException:
if the sample has not been set
SingularMatrixException:
if the design matrix is singular
Since
2.2
calculateTotalSumOfSquaresback to summary
public double calculateTotalSumOfSquares()

Returns the sum of squared deviations of Y from its mean.

If the model has no intercept term, 0 is used for the mean of Y - i.e., what is returned is the sum of the squared Y values.

The value returned by this method is the SSTO value used in the R-squared computation.

Returns:double

SSTO - the total sum of squares

Exceptions
NullPointerException:
if the sample has not been set
Since
2.2
See Also
isNoIntercept()
newSampleDataback to summary
public void newSampleData(double[] y, double[][] x) throws MathIllegalArgumentException

Loads model x and y sample data, overriding any previous sample. Computes and caches QR decomposition of the X matrix.

Parameters
y:double[]

the [n,1] array representing the y sample

x:double[][]

the [n,k] array representing the x sample

Exceptions
MathIllegalArgumentException:
if the x and y array data are not compatible for the regression
newSampleDataback to summary
public void newSampleData(double[] data, int nobs, int nvars)

Overrides org.apache.commons.math3.stat.regression.AbstractMultipleLinearRegression.newSampleData.

Doc from org.apache.commons.math3.stat.regression.AbstractMultipleLinearRegression.newSampleData.

Loads model x and y sample data from a flat input array, overriding any previous sample.

Assumes that rows are concatenated with y values first in each row. For example, an input data array containing the sequence of values (1, 2, 3, 4, 5, 6, 7, 8, 9) with nobs = 3 and nvars = 2 creates a regression dataset with two independent variables, as below: 

  y   x[0]  x[1]
  --------------
  1     2     3
  4     5     6
  7     8     9

Note that there is no need to add an initial unitary column (column of 1's) when specifying a model including an intercept term. If isNoIntercept() is true, the X matrix will be created without an initial column of "1"s; otherwise this column will be added.

Throws IllegalArgumentException if any of the following preconditions fail:

  • data cannot be null
  • data.length = nobs * (nvars + 1)
  • nobs > nvars

This implementation computes and caches the QR decomposition of the X matrix.

Parameters
data:double[]

input data array

nobs:int

number of observations (rows)

nvars:int

number of independent variables (columns, not counting y)

Annotations
@Override
newXSampleDataback to summary
protected void newXSampleData(double[][] x)

Overrides org.apache.commons.math3.stat.regression.AbstractMultipleLinearRegression.newXSampleData.

Doc from org.apache.commons.math3.stat.regression.AbstractMultipleLinearRegression.newXSampleData.

Loads new x sample data, overriding any previous data.

The input x array should have one row for each sample observation, with columns corresponding to independent variables. For example, if 
 x = new double[][] {{1, 2}, {3, 4}, {5, 6}}
then setXSampleData(x) results in a model with two independent variables and 3 observations: 
  x[0]  x[1]
  ----------
    1    2
    3    4
    5    6

Note that there is no need to add an initial unitary column (column of 1's) when specifying a model including an intercept term.

This implementation computes and caches the QR decomposition of the X matrix once it is successfully loaded.

Parameters
x:double[][]

the rectangular array representing the x sample

Annotations
@Override