Represents an empirical probability distribution -- a probability distribution derived from observed data without making any assumptions about the functional form of the population distribution that the data come from.
An EmpiricalDistribution maintains data structures, called
distribution digests, that describe empirical distributions and
support the following operations:
EmpiricalDistribution to build grouped
frequency histograms representing the input data or to generate random values
"like" those in the input file -- i.e., the values generated will follow the
distribution of the values in the file.
The implementation uses what amounts to the Variable Kernel Method with Gaussian smoothing:
Digesting the input file
binCount "bins."EmpiricalDistribution implements the RealDistribution interface
as follows. Given x within the range of values in the dataset, let B
be the bin containing x and let K be the within-bin kernel for B. Let P(B-)
be the sum of the probabilities of the bins below B and let K(B) be the
mass of B under K (i.e., the integral of the kernel density over B). Then
set P(X < x) = P(B-) + P(B) * K(x) / K(B) where K(x) is the kernel distribution
evaluated at x. This results in a cdf that matches the grouped frequency
distribution at the bin endpoints and interpolates within bins using
within-bin kernels.
binCount is set by default to 1000. A good rule of thumb
is to set the bin count to approximately the length of the input file divided
by 10. | Modifier and Type | Class and Description |
|---|---|
| private class | EmpiricalDistribution.
|
| private abstract class | EmpiricalDistribution.
Provides methods for computing |
| private class | EmpiricalDistribution.
|
| Modifier and Type | Field and Description |
|---|---|
| private final int | binCount
number of bins |
| private final List | binStats
List of SummaryStatistics objects characterizing the bins |
| public static final int | DEFAULT_BIN_COUNT
Default bin count |
| private double | delta
Grid size |
| private static final String | FILE_CHARSET
Character set for file input |
| private boolean | loaded
is the distribution loaded? |
| private double | max
Max loaded value |
| private double | min
Min loaded value |
| protected final RandomDataGenerator | randomData
Hides org.
Deprecated
as a consequence of RandomDataGenerator instance to use in repeated calls to getNext()
RandomDataGenerator being deprecated.
|
| private SummaryStatistics | sampleStats
Sample statistics |
| private static final long | serialVersionUID
Hides org. |
| private double[] | upperBounds
upper bounds of subintervals in (0,1) "belonging" to the bins |
| Access | Constructor and Description |
|---|---|
| public | |
| public | EmpiricalDistribution(int
number of bins. Must be strictly positive. binCount)Creates a new EmpiricalDistribution with the specified bin count. |
| public | EmpiricalDistribution(int
number of bins. Must be strictly positive. binCount, RandomGenerator random data generator (may be null, resulting in default JDK generator) generator)Creates a new EmpiricalDistribution with the specified bin count using the
provided |
| public | EmpiricalDistribution(RandomGenerator
random data generator (may be null, resulting in default JDK generator) generator)Creates a new EmpiricalDistribution with default bin count using the
provided |
| public | EmpiricalDistribution(int
number of bins binCount, RandomDataImpl random data generator (may be null, resulting in default JDK generator) randomData)
Deprecated
As of 3.1.
Creates a new EmpiricalDistribution with the specified bin count using the
provided |
| public | EmpiricalDistribution(RandomDataImpl
random data generator (may be null, resulting in default JDK generator) randomData)
Deprecated
As of 3.1.
Creates a new EmpiricalDistribution with default bin count using the
provided |
| private | EmpiricalDistribution(int
number of bins. Must be strictly positive. binCount, RandomDataGenerator Random data generator. randomData)
Deprecated
as a consequence of
RandomDataGenerator being deprecated.
Private constructor to allow lazy initialisation of the RNG contained
in the |
| Modifier and Type | Method and Description |
|---|---|
| private double | Returns: sum of the probabilities of bins through binIndexmaximum bin index binIndex)The combined probability of the bins up to and including binIndex. |
| public double | cumulativeProbability(double
the point at which the CDF is evaluated x)Implements org. For a random variable |
| public double | density(double
the point at which the PDF is evaluated x)Implements org. Returns the probability density function (PDF) of this distribution
evaluated at the specified point |
| private void | fillBinStats(final EmpiricalDistribution.
object providing access to the data daFills binStats array (second pass through data file). |
| private int | Returns: the index of the bin containing the valuethe value whose bin we are trying to find value)Returns the index of the bin to which the given value belongs |
| public int | |
| public List | Returns: List of bin statistics.Returns a List of |
| public double[] | Returns: array of upper bounds of subintervals used in data generationReturns a fresh copy of the array of upper bounds of the subintervals of [0,1] used in generating data from the empirical distribution. |
| protected RealDistribution | Returns: within-bin kernel parameterized by bStatssummary statistics for the bin bStats)The within-bin smoothing kernel. |
| public double | |
| public double | getNumericalMean()
Implements org. Use this method to get the numerical value of the mean of this distribution. |
| public double | getNumericalVariance()
Implements org. Use this method to get the numerical value of the variance of this distribution. |
| public StatisticalSummary | Returns: the sample statisticsReturns a |
| public double | getSupportLowerBound()
Implements org. Access the lower bound of the support. |
| public double | getSupportUpperBound()
Implements org. Access the upper bound of the support. |
| public double[] | Returns: array of bin upper boundsReturns a fresh copy of the array of upper bounds for the bins. |
| public double | inverseCumulativeProbability(final double
the cumulative probability p)Overrides org. Implements org. Computes the quantile function of this distribution. |
| public boolean | Returns: true if the distribution has been loadedProperty indicating whether or not the distribution has been loaded. |
| public boolean | isSupportConnected()
Implements org. Use this method to get information about whether the support is connected, i.e. whether all values between the lower and upper bound of the support are included in the support. |
| public boolean | isSupportLowerBoundInclusive()
Implements org. Whether or not the lower bound of support is in the domain of the density function. |
| public boolean | isSupportUpperBoundInclusive()
Implements org. Whether or not the upper bound of support is in the domain of the density function. |
| private RealDistribution | Returns: the within-bin kernel of the bin containing xthe value to locate within a bin x)The within-bin kernel of the bin that x belongs to. |
| private double | Returns: the difference in the within-bin kernel cdf between the upper and lower endpoints of bin iindex of the bin i)Mass of bin i under the within-bin kernel of the bin. |
| public void | load(double[]
the input data array in)Computes the empirical distribution from the provided array of numbers. |
| public void | |
| public void | |
| private double | Returns: the probability that selection begins in bin ithe index of the bin i)The probability of bin i. |
| private double | Returns: the probability that selection begins in a bin below bin i.the index of the bin i)The combined probability of the bins up to but not including bin i. |
| public double | probability(double
the point at which the PMF is evaluated x)Overrides org. Implements org. For a random variable |
| public void | |
| public void | reseedRandomGenerator(long
the new seed seed)Overrides org. Implements org. Reseed the random generator used to generate samples. |
| binCount | back to summary |
|---|---|
| private final int binCount number of bins | |
| binStats | back to summary |
|---|---|
| private final List<SummaryStatistics> binStats List of SummaryStatistics objects characterizing the bins | |
| DEFAULT_BIN_COUNT | back to summary |
|---|---|
| public static final int DEFAULT_BIN_COUNT Default bin count | |
| delta | back to summary |
|---|---|
| private double delta Grid size | |
| FILE_CHARSET | back to summary |
|---|---|
| private static final String FILE_CHARSET Character set for file input | |
| loaded | back to summary |
|---|---|
| private boolean loaded is the distribution loaded? | |
| max | back to summary |
|---|---|
| private double max Max loaded value | |
| min | back to summary |
|---|---|
| private double min Min loaded value | |
| randomData | back to summary |
|---|---|
| protected final RandomDataGenerator randomData Hides org. Deprecated as a consequence of RandomDataGenerator instance to use in repeated calls to getNext() | |
| sampleStats | back to summary |
|---|---|
| private SummaryStatistics sampleStats Sample statistics | |
| serialVersionUID | back to summary |
|---|---|
| private static final long serialVersionUID Hides org. Serializable version identifier | |
| upperBounds | back to summary |
|---|---|
| private double[] upperBounds upper bounds of subintervals in (0,1) "belonging" to the bins | |
| EmpiricalDistribution | back to summary |
|---|---|
| public EmpiricalDistribution() Creates a new EmpiricalDistribution with the default bin count. | |
| EmpiricalDistribution | back to summary |
|---|---|
| public EmpiricalDistribution(int binCount) Creates a new EmpiricalDistribution with the specified bin count.
| |
| EmpiricalDistribution | back to summary |
|---|---|
| public EmpiricalDistribution(int binCount, RandomGenerator generator) Creates a new EmpiricalDistribution with the specified bin count using the
provided
| |
| EmpiricalDistribution | back to summary |
|---|---|
| public EmpiricalDistribution(RandomGenerator generator) Creates a new EmpiricalDistribution with default bin count using the
provided
| |
| EmpiricalDistribution | back to summary |
|---|---|
| public EmpiricalDistribution(int binCount, RandomDataImpl randomData)
Deprecated As of 3.1. Please use Creates a new EmpiricalDistribution with the specified bin count using the
provided
| |
| EmpiricalDistribution | back to summary |
|---|---|
| public EmpiricalDistribution(RandomDataImpl randomData)
Deprecated As of 3.1. Please use Creates a new EmpiricalDistribution with default bin count using the
provided
| |
| EmpiricalDistribution | back to summary |
|---|---|
| private EmpiricalDistribution(int binCount, RandomDataGenerator randomData)
Deprecated as a consequence of Private constructor to allow lazy initialisation of the RNG contained
in the
| |
| cumBinP | back to summary |
|---|---|
| private double cumBinP(int binIndex) The combined probability of the bins up to and including binIndex.
| |
| cumulativeProbability | back to summary |
|---|---|
| public double cumulativeProbability(double x) Implements org. Doc from org. For a random variable Algorithm description:
| |
| density | back to summary |
|---|---|
| public double density(double x) Implements org. Doc from org. Returns the probability density function (PDF) of this distribution
evaluated at the specified point Returns the kernel density normalized so that its integral over each bin equals the bin mass. Algorithm description:
| |
| fillBinStats | back to summary |
|---|---|
| private void fillBinStats(final EmpiricalDistribution. Fills binStats array (second pass through data file).
| |
| findBin | back to summary |
|---|---|
| private int findBin(double value) Returns the index of the bin to which the given value belongs
| |
| getBinCount | back to summary |
|---|---|
| public int getBinCount() Returns the number of bins.
| |
| getBinStats | back to summary |
|---|---|
| public List Returns a List of
| |
| getGeneratorUpperBounds | back to summary |
|---|---|
| public double[] getGeneratorUpperBounds() Returns a fresh copy of the array of upper bounds of the subintervals of [0,1] used in generating data from the empirical distribution. Subintervals correspond to bins with lengths proportional to bin counts. Preconditions:
In versions 1.0-2.0 of commons-math, this array was (incorrectly) returned
by
| |
| getKernel | back to summary |
|---|---|
| protected RealDistribution getKernel(SummaryStatistics bStats) The within-bin smoothing kernel. Returns a Gaussian distribution
parameterized by
| |
| getNextValue | back to summary |
|---|---|
| public double getNextValue() throws MathIllegalStateException Generates a random value from this distribution. Preconditions:
| |
| getNumericalMean | back to summary |
|---|---|
| public double getNumericalMean() Implements org. Doc from org. Use this method to get the numerical value of the mean of this distribution.
| |
| getNumericalVariance | back to summary |
|---|---|
| public double getNumericalVariance() Implements org. Doc from org. Use this method to get the numerical value of the variance of this distribution.
| |
| getSampleStats | back to summary |
|---|---|
| public StatisticalSummary getSampleStats() Returns a
| |
| getSupportLowerBound | back to summary |
|---|---|
| public double getSupportLowerBound() Implements org. Doc from org. Access the lower bound of the support. This method must return the same
value as
| |
| getSupportUpperBound | back to summary |
|---|---|
| public double getSupportUpperBound() Implements org. Doc from org. Access the upper bound of the support. This method must return the same
value as
| |
| getUpperBounds | back to summary |
|---|---|
| public double[] getUpperBounds() Returns a fresh copy of the array of upper bounds for the bins.
Bins are: Note In versions 1.0-2.0 of commons-math, this method
incorrectly returned the array of probability generator upper
bounds now returned by
| |
| inverseCumulativeProbability | back to summary |
|---|---|
| public double inverseCumulativeProbability(final double p) throws OutOfRangeException Overrides org. Implements org. Doc from org. Computes the quantile function of this distribution. For a random
variable
Algorithm description:
| |
| isLoaded | back to summary |
|---|---|
| public boolean isLoaded() Property indicating whether or not the distribution has been loaded.
| |
| isSupportConnected | back to summary |
|---|---|
| public boolean isSupportConnected() Implements org. Doc from org. Use this method to get information about whether the support is connected, i.e. whether all values between the lower and upper bound of the support are included in the support.
| |
| isSupportLowerBoundInclusive | back to summary |
|---|---|
| public boolean isSupportLowerBoundInclusive() Implements org. Doc from org. Whether or not the lower bound of support is in the domain of the density
function. Returns true iff
| |
| isSupportUpperBoundInclusive | back to summary |
|---|---|
| public boolean isSupportUpperBoundInclusive() Implements org. Doc from org. Whether or not the upper bound of support is in the domain of the density
function. Returns true iff
| |
| k | back to summary |
|---|---|
| private RealDistribution k(double x) The within-bin kernel of the bin that x belongs to.
| |
| kB | back to summary |
|---|---|
| private double kB(int i) Mass of bin i under the within-bin kernel of the bin.
| |
| load | back to summary |
|---|---|
| public void load(double[] in) throws NullArgumentException Computes the empirical distribution from the provided array of numbers.
| |
| load | back to summary |
|---|---|
| public void load(URL url) throws IOException, NullArgumentException, ZeroException Computes the empirical distribution using data read from a URL. The input file must be an ASCII text file containing one valid numeric entry per line.
| |
| load | back to summary |
|---|---|
| public void load(File file) throws IOException, NullArgumentException Computes the empirical distribution from the input file. The input file must be an ASCII text file containing one valid numeric entry per line.
| |
| pB | back to summary |
|---|---|
| private double pB(int i) The probability of bin i.
| |
| pBminus | back to summary |
|---|---|
| private double pBminus(int i) The combined probability of the bins up to but not including bin i.
| |
| probability | back to summary |
|---|---|
| public double probability(double x) Overrides org. Implements org. Doc from org. For a random variable
| |
| reSeed | back to summary |
|---|---|
| public void reSeed(long seed) Reseeds the random number generator used by
| |
| reseedRandomGenerator | back to summary |
|---|---|
| public void reseedRandomGenerator(long seed) Overrides org. Implements org. Doc from org. Reseed the random generator used to generate samples.
| |
DataAdapter for data provided as array of doubles.
| Modifier and Type | Field and Description |
|---|---|
| private double[] | inputArray
Array of input data values |
| Access | Constructor and Description |
|---|---|
| pack-priv | ArrayDataAdapter(double[]
double[] array holding the data in)Construct an ArrayDataAdapter from a double[] array |
| Modifier and Type | Method and Description |
|---|---|
| public void | computeBinStats()
Implements abstract org. Compute bin stats. |
| public void | computeStats()
Implements abstract org. Compute sample statistics. |
| inputArray | back to summary |
|---|---|
| private double[] inputArray Array of input data values | |
| ArrayDataAdapter | back to summary |
|---|---|
| pack-priv ArrayDataAdapter(double[] in) throws NullArgumentException Construct an ArrayDataAdapter from a double[] array
| |
| computeBinStats | back to summary |
|---|---|
| public void computeBinStats() throws IOException Implements abstract org. Doc from org. Compute bin stats.
| |
| computeStats | back to summary |
|---|---|
| public void computeStats() throws IOException Implements abstract org. Doc from org. Compute sample statistics.
| |
sampleStats and
beanStats abstracting the source of data.
| Access | Constructor and Description |
|---|---|
| private |
| Modifier and Type | Method and Description |
|---|---|
| public abstract void | |
| public abstract void |
| DataAdapter | back to summary |
|---|---|
| private DataAdapter() | |
| computeBinStats | back to summary |
|---|---|
| public abstract void computeBinStats() throws IOException Compute bin stats.
| |
| computeStats | back to summary |
|---|---|
| public abstract void computeStats() throws IOException Compute sample statistics.
| |
DataAdapter for data provided through some input stream
| Modifier and Type | Field and Description |
|---|---|
| private BufferedReader | inputStream
Input stream providing access to the data |
| Access | Constructor and Description |
|---|---|
| pack-priv | StreamDataAdapter(BufferedReader
BufferedReader input stream in)Create a StreamDataAdapter from a BufferedReader |
| Modifier and Type | Method and Description |
|---|---|
| public void | computeBinStats()
Implements abstract org. Compute bin stats. |
| public void | computeStats()
Implements abstract org. Compute sample statistics. |
| inputStream | back to summary |
|---|---|
| private BufferedReader inputStream Input stream providing access to the data | |
| StreamDataAdapter | back to summary |
|---|---|
| pack-priv StreamDataAdapter(BufferedReader in) Create a StreamDataAdapter from a BufferedReader
| |
| computeBinStats | back to summary |
|---|---|
| public void computeBinStats() throws IOException Implements abstract org. Doc from org. Compute bin stats.
| |
| computeStats | back to summary |
|---|---|
| public void computeStats() throws IOException Implements abstract org. Doc from org. Compute sample statistics.
| |