Top Description Fields Constructors Methods
org.apache.commons.math3.stat.regression

public Class MillerUpdatingRegression

extends Object
implements UpdatingMultipleLinearRegression
Class Inheritance
All Implemented Interfaces
org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression
Imports
java.util.Arrays, org.apache.commons.math3.exception.util.LocalizedFormats, org.apache.commons.math3.util.FastMath, .Precision, .MathArrays

This class is a concrete implementation of the UpdatingMultipleLinearRegression interface.

The algorithm is described in:

Algorithm AS 274: Least Squares Routines to Supplement Those of Gentleman
Author(s): Alan J. Miller
Source: Journal of the Royal Statistical Society.
Series C (Applied Statistics), Vol. 41, No. 2
(1992), pp. 458-478
Published by: Blackwell Publishing for the Royal Statistical Society
Stable URL: http://www.jstor.org/stable/2347583 

This method for multiple regression forms the solution to the OLS problem by updating the QR decomposition as described by Gentleman.

Since
3.0

Field Summary

Modifier and TypeField and Description
private final double[]
d

diagonals of cross products matrix

private final double
epsilon

zero tolerance

private boolean
hasIntercept

boolean flag whether a regression constant is added

private final boolean[]
lindep

flags for variables with linear dependency problems

private long
nobs

number of observations entered

private final int
nvars

number of variables in regression

private final double[]
r

the off diagonal portion of the R matrix

private final double[]
rhs

the elements of the R`Y

private final double[]
rss

residual sum of squares for all nested regressions

private boolean
rss_set

has rss been called?

private double
sserr

sum of squared errors of largest regression

private double
sumsqy

summation of squared Y values

private double
sumy

summation of Y variable

private final double[]
tol

the tolerance for each of the variables

private boolean
tol_set

has the tolerance setting method been called

private final int[]
vorder

order of the regressors

private final double[]
work_sing

workspace for singularity method

private final double[]
work_tolset

scratch space for tolerance calc

private final double[]
x_sing

singular x values

Constructor Summary

AccessConstructor and Description
private
MillerUpdatingRegression()

Set the default constructor to private access to prevent inadvertent instantiation

public
MillerUpdatingRegression(int
number of regressors to expect, not including constant
numberOfVariables
,
boolean
include a constant automatically
includeConstant
,
double
zero tolerance, how machine zero is determined
errorTolerance
)

This is the augmented constructor for the MillerUpdatingRegression class.

public
MillerUpdatingRegression(int
maximum number of potential regressors
numberOfVariables
,
boolean
include a constant automatically
includeConstant
)

Primary constructor for the MillerUpdatingRegression.

Method Summary

Modifier and TypeMethod and Description
public void
addObservation(final double[]
the array with regressor values
x
,
final double
the value of dependent variable given these regressors
y
)

Implements org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression.addObservation.

Adds an observation to the regression model.

public void
addObservations(double[][]
observations on the regressors
x
,
double[]
observations on the regressand
y
)

Implements org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression.addObservations.

Adds multiple observations to the model.

public void
clear()

Implements org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression.clear.

As the name suggests, clear wipes the internals and reorders everything in the canonical order.

private double[]

Returns:

an array with the variance covariance of the included regressors in lower triangular form
cov
(int
how many of the regressors to include (either in canonical order, or in the current reordered state)
nreq
)

Calculates the cov matrix assuming only the first nreq variables are included in the calculation.

public double

Returns:

the diagonal element of the hatmatrix
getDiagonalOfHatMatrix
(double[]
returns the diagonal of the hat matrix for this observation
row_data
)

Gets the diagonal of the Hat matrix also known as the leverage matrix.

public long

Returns:

number of observations
getN
()

Implements org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression.getN.

Gets the number of observations added to the regression model.

public int[]

Returns:

int[] with the current order of the regressors
getOrderOfRegressors
()

Gets the order of the regressors, useful if some type of reordering has been called.

public double[]

Returns:

an array with the partial correlations of the remainder of regressors with each other and the regressand, in lower triangular form
getPartialCorrelations
(int
how many of the regressors to include (either in canonical order, or in the current reordered state)
in
)

In the original algorithm only the partial correlations of the regressors is returned to the user.

public boolean

Returns:

true regression has an intercept, false no intercept
hasIntercept
()

Implements org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression.hasIntercept.

A getter method which determines whether a constant is included.

private void
include(final double[]
observations on the regressors
x
,
final double
weight of the this observation (-1,1)
wi
,
final double
observation on the regressand
yi
)

The include method is where the QR decomposition occurs.

private void
inverse(double[]
the storage for the inverse of r
rinv
,
int
how many of the regressors to include (either in canonical order, or in the current reordered state)
nreq
)

This internal method calculates the inverse of the upper-triangular portion of the R matrix.

private double[]

Returns:

an array with the estimated slope coefficients
regcf
(int
how many of the regressors to include (either in canonical order, or in the current reordered state)
nreq
)

The regcf method conducts the linear regression and extracts the parameter vector.

public RegressionResults

Returns:

RegressionResults the structure holding all regression results
regress
()

Implements org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression.regress.

Conducts a regression on the data in the model, using all regressors.

public RegressionResults

Returns:

RegressionResults the structure holding all regression results
regress
(int
many of the regressors to include (either in canonical order, or in the current reordered state)
numberOfRegressors
)

Conducts a regression on the data in the model, using a subset of regressors.

public RegressionResults

Returns:

RegressionResults the structure holding all regression results
regress
(int[]
array of variables to include in regression
variablesToInclude
)

Implements org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression.regress.

Conducts a regression on the data in the model, using regressors in array Calling this method will change the internal order of the regressors and care is required in interpreting the hatmatrix.

private int

Returns:

-1 error, 0 everything ok
reorderRegressors
(int[]
the regressors to move
list
,
int
where the list will be placed
pos1
)

ALGORITHM AS274 APPL.

private void
singcheck()

The method which checks for singularities and then eliminates the offending columns.

private double

Returns:

the sum of the a and b
smartAdd
(double
- an addend
a
,
double
- an addend
b
)

Adds to number a and b such that the contamination due to numerical smallness of one addend does not corrupt the sum.

private void
ss()

Calculates the sum of squared errors for the full regression and all subsets in the following manner:

rss[] ={
ResidualSumOfSquares_allNvars,
ResidualSumOfSquares_FirstNvars-1,
ResidualSumOfSquares_FirstNvars-2,
..., ResidualSumOfSquares_FirstVariable} 
private void
tolset()

This sets up tolerances for singularity testing.

private void
vmove(int
initial position
from
,
int
destination
to
)

ALGORITHM AS274 APPL.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

dback to summary
private final double[] d

diagonals of cross products matrix

epsilonback to summary
private final double epsilon

zero tolerance

hasInterceptback to summary
private boolean hasIntercept

boolean flag whether a regression constant is added

lindepback to summary
private final boolean[] lindep

flags for variables with linear dependency problems

nobsback to summary
private long nobs

number of observations entered

nvarsback to summary
private final int nvars

number of variables in regression

rback to summary
private final double[] r

the off diagonal portion of the R matrix

rhsback to summary
private final double[] rhs

the elements of the R`Y

rssback to summary
private final double[] rss

residual sum of squares for all nested regressions

rss_setback to summary
private boolean rss_set

has rss been called?

sserrback to summary
private double sserr

sum of squared errors of largest regression

sumsqyback to summary
private double sumsqy

summation of squared Y values

sumyback to summary
private double sumy

summation of Y variable

tolback to summary
private final double[] tol

the tolerance for each of the variables

tol_setback to summary
private boolean tol_set

has the tolerance setting method been called

vorderback to summary
private final int[] vorder

order of the regressors

work_singback to summary
private final double[] work_sing

workspace for singularity method

work_tolsetback to summary
private final double[] work_tolset

scratch space for tolerance calc

x_singback to summary
private final double[] x_sing

singular x values

Constructor Detail

MillerUpdatingRegressionback to summary
private MillerUpdatingRegression()

Set the default constructor to private access to prevent inadvertent instantiation

Annotations
@SuppressWarnings:unused
MillerUpdatingRegressionback to summary
public MillerUpdatingRegression(int numberOfVariables, boolean includeConstant, double errorTolerance) throws ModelSpecificationException

This is the augmented constructor for the MillerUpdatingRegression class.

Parameters
numberOfVariables:int

number of regressors to expect, not including constant

includeConstant:boolean

include a constant automatically

errorTolerance:double

zero tolerance, how machine zero is determined

Exceptions
ModelSpecificationException:
if numberOfVariables is less than 1
MillerUpdatingRegressionback to summary
public MillerUpdatingRegression(int numberOfVariables, boolean includeConstant) throws ModelSpecificationException

Primary constructor for the MillerUpdatingRegression.

Parameters
numberOfVariables:int

maximum number of potential regressors

includeConstant:boolean

include a constant automatically

Exceptions
ModelSpecificationException:
if numberOfVariables is less than 1

Method Detail

addObservationback to summary
public void addObservation(final double[] x, final double y) throws ModelSpecificationException

Implements org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression.addObservation.

Adds an observation to the regression model.

Parameters
x:double[]

the array with regressor values

y:double

the value of dependent variable given these regressors

Exceptions
ModelSpecificationException:
if the length of x does not equal the number of independent variables in the model
addObservationsback to summary
public void addObservations(double[][] x, double[] y) throws ModelSpecificationException

Implements org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression.addObservations.

Adds multiple observations to the model.

Parameters
x:double[][]

observations on the regressors

y:double[]

observations on the regressand

Exceptions
ModelSpecificationException:
if x is not rectangular, does not match the length of y or does not contain sufficient data to estimate the model
clearback to summary
public void clear()

Implements org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression.clear.

As the name suggests, clear wipes the internals and reorders everything in the canonical order.

covback to summary
private double[] cov(int nreq)

Calculates the cov matrix assuming only the first nreq variables are included in the calculation. The returned array contains a symmetric matrix stored in lower triangular form. The matrix will have ( nreq + 1 ) * nreq / 2 elements. For illustration

cov =
{
 cov_00,
 cov_10, cov_11,
 cov_20, cov_21, cov22,
 ...
} 
Parameters
nreq:int

how many of the regressors to include (either in canonical order, or in the current reordered state)

Returns:double[]

an array with the variance covariance of the included regressors in lower triangular form

getDiagonalOfHatMatrixback to summary
public double getDiagonalOfHatMatrix(double[] row_data)

Gets the diagonal of the Hat matrix also known as the leverage matrix.

Parameters
row_data:double[]

returns the diagonal of the hat matrix for this observation

Returns:double

the diagonal element of the hatmatrix

getNback to summary
public long getN()

Implements org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression.getN.

Gets the number of observations added to the regression model.

Returns:long

number of observations

getOrderOfRegressorsback to summary
public int[] getOrderOfRegressors()

Gets the order of the regressors, useful if some type of reordering has been called. Calling regress with int[]{} args will trigger a reordering.

Returns:int[]

int[] with the current order of the regressors

getPartialCorrelationsback to summary
public double[] getPartialCorrelations(int in)

In the original algorithm only the partial correlations of the regressors is returned to the user. In this implementation, we have

corr =
{
  corrxx - lower triangular
  corrxy - bottom row of the matrix
}
Replaces subroutines PCORR and COR of:
ALGORITHM AS274  APPL. STATIST. (1992) VOL.41, NO. 2 

Calculate partial correlations after the variables in rows 1, 2, ..., IN have been forced into the regression. If IN = 1, and the first row of R represents a constant in the model, then the usual simple correlations are returned.

If IN = 0, the value returned in array CORMAT for the correlation of variables Xi & Xj is:

sum ( Xi.Xj ) / Sqrt ( sum (Xi^2) . sum (Xj^2) )

On return, array CORMAT contains the upper triangle of the matrix of partial correlations stored by rows, excluding the 1's on the diagonal. e.g. if IN = 2, the consecutive elements returned are: (3,4) (3,5) ... (3,ncol), (4,5) (4,6) ... (4,ncol), etc. Array YCORR stores the partial correlations with the Y-variable starting with YCORR(IN+1) = partial correlation with the variable in position (IN+1).

Parameters
in:int

how many of the regressors to include (either in canonical order, or in the current reordered state)

Returns:double[]

an array with the partial correlations of the remainder of regressors with each other and the regressand, in lower triangular form

hasInterceptback to summary
public boolean hasIntercept()

Implements org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression.hasIntercept.

A getter method which determines whether a constant is included.

Returns:boolean

true regression has an intercept, false no intercept

includeback to summary
private void include(final double[] x, final double wi, final double yi)

The include method is where the QR decomposition occurs. This statement forms all intermediate data which will be used for all derivative measures. According to the miller paper, note that in the original implementation the x vector is overwritten. In this implementation, the include method is passed a copy of the original data vector so that there is no contamination of the data. Additionally, this method differs slightly from Gentleman's method, in that the assumption is of dense design matrices, there is some advantage in using the original gentleman algorithm on sparse matrices.

Parameters
x:double[]

observations on the regressors

wi:double

weight of the this observation (-1,1)

yi:double

observation on the regressand

inverseback to summary
private void inverse(double[] rinv, int nreq)

This internal method calculates the inverse of the upper-triangular portion of the R matrix.

Parameters
rinv:double[]

the storage for the inverse of r

nreq:int

how many of the regressors to include (either in canonical order, or in the current reordered state)

regcfback to summary
private double[] regcf(int nreq) throws ModelSpecificationException

The regcf method conducts the linear regression and extracts the parameter vector. Notice that the algorithm can do subset regression with no alteration.

Parameters
nreq:int

how many of the regressors to include (either in canonical order, or in the current reordered state)

Returns:double[]

an array with the estimated slope coefficients

Exceptions
ModelSpecificationException:
if nreq is less than 1 or greater than the number of independent variables
regressback to summary
public RegressionResults regress() throws ModelSpecificationException

Implements org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression.regress.

Conducts a regression on the data in the model, using all regressors.

Returns:RegressionResults

RegressionResults the structure holding all regression results

Exceptions
ModelSpecificationException:
- thrown if number of observations is less than the number of variables
regressback to summary
public RegressionResults regress(int numberOfRegressors) throws ModelSpecificationException

Conducts a regression on the data in the model, using a subset of regressors.

Parameters
numberOfRegressors:int

many of the regressors to include (either in canonical order, or in the current reordered state)

Returns:RegressionResults

RegressionResults the structure holding all regression results

Exceptions
ModelSpecificationException:
- thrown if number of observations is less than the number of variables or number of regressors requested is greater than the regressors in the model
regressback to summary
public RegressionResults regress(int[] variablesToInclude) throws ModelSpecificationException

Implements org.apache.commons.math3.stat.regression.UpdatingMultipleLinearRegression.regress.

Conducts a regression on the data in the model, using regressors in array Calling this method will change the internal order of the regressors and care is required in interpreting the hatmatrix.

Parameters
variablesToInclude:int[]

array of variables to include in regression

Returns:RegressionResults

RegressionResults the structure holding all regression results

Exceptions
ModelSpecificationException:
- thrown if number of observations is less than the number of variables, the number of regressors requested is greater than the regressors in the model or a regressor index in regressor array does not exist
reorderRegressorsback to summary
private int reorderRegressors(int[] list, int pos1)

ALGORITHM AS274 APPL. STATIST. (1992) VOL.41, NO. 2

Re-order the variables in an orthogonal reduction produced by AS75.1 so that the N variables in LIST start at position POS1, though will not necessarily be in the same order as in LIST. Any variables in VORDER before position POS1 are not moved. Auxiliary routine called: VMOVE.

This internal method reorders the regressors.

Parameters
list:int[]

the regressors to move

pos1:int

where the list will be placed

Returns:int

-1 error, 0 everything ok

singcheckback to summary
private void singcheck()

The method which checks for singularities and then eliminates the offending columns.

smartAddback to summary
private double smartAdd(double a, double b)

Adds to number a and b such that the contamination due to numerical smallness of one addend does not corrupt the sum.

Parameters
a:double

- an addend

b:double

- an addend

Returns:double

the sum of the a and b

ssback to summary
private void ss()

Calculates the sum of squared errors for the full regression and all subsets in the following manner:

rss[] ={
ResidualSumOfSquares_allNvars,
ResidualSumOfSquares_FirstNvars-1,
ResidualSumOfSquares_FirstNvars-2,
..., ResidualSumOfSquares_FirstVariable} 
tolsetback to summary
private void tolset()

This sets up tolerances for singularity testing.

vmoveback to summary
private void vmove(int from, int to)

ALGORITHM AS274 APPL. STATIST. (1992) VOL.41, NO. 2. Move variable from position FROM to position TO in an orthogonal reduction produced by AS75.1.

Parameters
from:int

initial position

to:int

destination