Statistics Dictionary
Short, general definitions of key statistical terms, organized around major concepts. These are standard knowledge and are not from any particular book or resource.
Terms are grouped by conceptual categories. Within each category, terms are listed alphabetically.
Probability Fundamentals
axiomatic approach (to probability)
A formal definition of probability based on axioms, such as Kolmogorov's axioms.
Bayes theorem
A formula that describes how to update probabilities based on new evidence, using prior and conditional probabilities.
choices, multiplication of
The fundamental counting principle for independent choices.
combinations
The number of ways to choose items without regard to order.
complement (of an event)
The event that the original event does not occur.
compound events
Events composed of multiple simple events.
conditional probability
The probability of an event given that another event has occurred.
De Morgan laws
Rules relating the complement of unions and intersections of sets.
element (of sample space)
A possible outcome in the sample space.
elimination, rule of
A probability rule for conditional events.
events
Subsets of the sample space.
equally likely
Each with same probability.
independent
Occurrence of one doesn't affect another.
intersection
Both events occur.
mutually exclusive
Cannot occur together.
pairwise independent
Every pair independent.
union
At least one occurs.
equally-likely events
Events with equal probabilities.
frequency interpretation (probability)
Probability as the long-run relative frequency of an event.
general addition rule
P(A or B) = P(A) + P(B) - P(A and B).
inclusion-exclusion principle
A counting rule for unions of sets.
intersection (of events)
The event that all occur.
multiplication rule (of probabilities)
P(A and B) = P(A) P(B|A).
odds
Ratio of probability of event to its complement.
permutations
Arrangements where order matters.
circular
Circular arrangements.
probability
Measure of likelihood of events.
of acceptance
Probability of accepting a lot.
conditional
Probability given evidence.
convergence in
Convergence type.
density function
PDF.
distribution
Probability assignment.
equally-likely events
Equal probabilities.
general addition rule
Union probability.
multiplication rule
Intersection probability.
rule of elimination
Conditional probability rule.
special addition rule
For disjoint events.
subjective method
Personal belief probability.
sample space
Set of all possible outcomes.
countable
Countably infinite sample space.
discrete
Discrete outcomes.
finite
Finite outcomes.
special addition rule
For disjoint events: sum probabilities.
subjective method (probability)
Personal assessment of probability.
tree diagram
Visual for multistage experiments.
union (of events)
At least one occurs.
Venn diagram
Visual for set relationships.
Random Variables and Expectations
conditional density
The density function of one variable given the value of another.
conditional distribution
The distribution of one variable given the value of another.
joint
The joint distribution given a condition.
conditional mean
The expected value of a variable given another variable's value.
conditional variance
The variance of a variable given another variable's value.
continuous random variable
A random variable that can take any value in a continuous interval.
covariance
A measure of how two variables change together, the expected value of their product deviation from means.
cumulant
A set of parameters describing the properties of a probability distribution, related to moments.
cumulative distribution
The function giving the probability that a random variable is less than or equal to a value.
density function
A function describing the relative likelihood for a continuous random variable.
joint probability
The density for two or more continuous variables.
discrete random variable
A random variable that takes countable values.
expected value
The long-run average value of repetitions of the experiment.
product of independent variables
Product of their expected values.
independence
When events or variables do not influence each other.
of discrete random variables
Joint PMF equals product of marginals.
and covariance
Independent variables have zero covariance.
and variance
Variance of sum equals sum of variances for independents.
joint conditional distribution
Joint distribution given a condition.
joint marginal distribution
The marginal of a joint distribution.
joint moment-generating function
Generating function for joint moments.
joint probability distribution
The distribution for multiple variables.
joint probability density
The PDF for joint continuous variables.
marginal distribution
The distribution of one variable from a joint.
mathematical expectation
The mean of a random variable.
conditional
Expected value given another variable.
mean
The average value of a distribution or sample.
conditional
Mean given another variable.
moments
Expected powers of deviations from mean or origin.
about the mean
Central moments.
about the origin
Raw moments.
factorial
Moments using falling factorials.
of linear combinations
Moments of sums or linear transforms.
moment generating function
A function whose derivatives give moments.
factorial
For factorial moments.
joint
For multiple variables.
product moments
Expected products of variables.
about the mean
Central product moments.
about the origin
Raw product moments.
random variable
A function assigning numbers to outcomes.
continuous
Takes continuous values.
discrete
Takes countable values.
expected value
Mean of the variable.
variance
Expected squared deviation from mean.
conditional
Variance given another variable.
of linear combination
Weighted sum of variances + covariances.
Discrete Distributions
Bernoulli distribution
A discrete distribution for a single trial with two outcomes: success (probability p) or failure (1-p).
binomial distribution
A discrete distribution for the number of successes in n independent Bernoulli trials with success probability p.
mean and variance
Mean np, variance np(1-p).
moment generating function
The function used to derive moments of the distribution.
discrete uniform distribution
A distribution where each outcome in a finite set is equally likely.
geometric distribution
The distribution of trials until the first success in Bernoulli trials.
hypergeometric distribution
Distribution for sampling without replacement from a finite population.
multivariate
Extension to multiple categories.
multinomial distribution
Generalization of binomial to multiple categories.
negative binomial distribution
Number of trials until a fixed number of successes.
Poisson distribution
Distribution for count of rare events.
mean and variance
Both λ.
moment-generating function
exp(λ(e^t - 1)).
Continuous Distributions
beta distribution
A continuous distribution on [0,1] used to model proportions or probabilities, parameterized by shape parameters α and β.
beta function
A special function that serves as the normalizing constant for the beta distribution.
Cauchy distribution
A continuous distribution with heavy tails and no defined mean or variance, used in robust statistics.
chi-square distribution
A distribution of the sum of squared standard normal variables, used in hypothesis testing and confidence intervals.
exponential distribution
A continuous distribution for waiting times in Poisson processes.
two-parameter
With location and scale parameters.
F distribution
The distribution of the ratio of two chi-square variables divided by degrees of freedom.
gamma distribution
A continuous distribution generalizing the exponential, used for waiting times.
log-normal distribution
Distribution whose log is normal.
normal distribution
Symmetric bell-shaped continuous distribution.
bivariate
Joint normal for two variables.
circular
Isotropic bivariate normal.
moment-generating function
exp(μt + σ²t²/2).
standard
Normal with mean 0, variance 1.
Pareto distribution
Power-law distribution for modeling inequalities.
Rayleigh distribution
Distribution for radial errors or wind speed.
Student t distribution
Distribution used for small sample inference.
uniform distribution
Constant probability over interval.
Weibull distribution
For modeling lifetimes and failures.
Sampling and Limit Theorems
central limit theorem
The theorem stating that the distribution of sample means approaches normality as sample size increases, for most distributions.
Chebyshev's theorem
A bound on the probability that a random variable deviates from its mean by more than k standard deviations.
law of large numbers
Sample mean converges to population mean as n increases.
random sample
A sample where each element has equal chance.
finite population
Sampling from finite population.
sampling
Process of selecting a sample.
distribution
Distribution of a statistic.
distribution of the mean
Sampling distribution of sample mean.
without replacement
Sampling where items are not returned.
sampling bias
Bias from non-representative sampling.
Estimation
asymptotically unbiased estimator
An estimator whose bias approaches zero as the sample size increases to infinity.
asymptotically efficient estimator
An estimator that achieves the Cramer-Rao lower bound for variance as sample size grows large.
Bayesian estimation
An estimation method that incorporates prior information with observed data using Bayes' theorem.
bias
The systematic error in an estimator, measured as the difference between its expected value and the true parameter.
consistent estimator
An estimator that converges in probability to the true parameter value as sample size grows.
Cramer-Rao inequality
A lower bound on the variance of any unbiased estimator.
efficient estimator
An unbiased estimator with the smallest possible variance.
estimator
A statistic used to estimate a parameter.
interval estimator
A range estimating the parameter.
maximum likelihood estimator
The value of the parameter that maximizes the likelihood function, i.e., makes the observed data most probable.
method of least squares
Method to minimize sum of squared residuals.
method of moments
Estimating parameters by equating sample and population moments.
minimum variance unbiased estimator
The unbiased estimator with smallest variance.
point estimator
A single value estimating a parameter.
pooled estimator
Combined estimate from multiple samples.
posterior distribution
Updated prior after data in Bayesian inference.
prior distribution
Initial belief before data in Bayesian inference.
sufficient estimator
Estimator containing all information about parameter.
unbiased estimator
Expected value equals parameter.
Hypothesis Testing and Inference
alternative hypothesis
The hypothesis that there is an effect or difference, opposing the null hypothesis.
composite hypothesis
A hypothesis that specifies a range of values for the parameter, not a single value.
critical region
The set of test statistic values leading to rejection of the null hypothesis.
most powerful
The critical region that maximizes power for a given significance level.
critical values
Threshold values for test statistics that determine rejection regions.
distribution-free tests
Nonparametric tests that do not assume a specific distribution.
hypothesis
A testable statement about a parameter.
simple
Specifies a single value.
large-sample test
Tests relying on asymptotic normality for large n.
level of significance
The probability of Type I error (α).
likelihood ratio test
A test based on the ratio of likelihoods under null and alternative.
likelihood ratio statistic
The test statistic from the likelihood ratio.
Neyman-Pearson lemma
Provides the most powerful test for simple hypotheses.
nonparametric statistics
Methods that do not assume a specific distribution.
P-value
Probability of more extreme data under null hypothesis.
power
The probability of correctly rejecting the null when false.
power function of a test
Power as a function of the true parameter.
rejection region
Critical region for rejecting null.
robustness
Insensitivity to assumption violations.
significance tests
Hypothesis tests at a significance level.
simple hypothesis
Hypothesis specifying a single value.
test of hypothesis
Procedure to assess evidence against a hypothesis using data.
acceptance region for
Non-rejection region.
goodness of fit
Test if data fits a distribution.
large-sample
Asymptotic test for large n.
likelihood ratio test
Based on likelihood ratio.
operating characteristic curve
Power curve.
power
1 - beta.
power function
Power vs parameter.
rejection region for
Critical region.
sequential
Test that accumulates data until decision.
type I error
Rejecting true null.
type II error
Failing to reject false null.
uniformly most powerful
Most powerful across alternatives.
type I and type II errors
False positive and false negative.
Confidence Intervals
confidence interval
A range of values, derived from sample data, that is likely to contain the true population parameter with a specified level of confidence (e.g., 95%).
difference of two proportions
Confidence interval for the difference between two population proportions.
one-sided
A confidence interval with only an upper or lower bound.
proportion
Confidence interval for a population proportion.
regression coefficient
Confidence interval for the slope in a regression model.
limits of prediction
Interval for predicting future observations.
one-sided confidence interval
Interval with one infinite bound.
prediction, limits of
Interval for future observations.
tolerance limits
Bounds containing a proportion of the population with confidence.
Regression and Correlation
correlation coefficient
A measure of the linear relationship between two variables, ranging from -1 to 1.
sample
The correlation coefficient calculated from sample data.
correlation analysis
The study of relationships between variables using correlation measures.
multiple regression
Regression with multiple predictors.
coefficients
Slopes for each predictor.
residuals
Differences between observed and predicted.
normal regression analysis
Regression assuming normal errors.
regression
Modeling the relationship between variables.
bivariate
Regression with two variables.
coefficients
Parameters in regression model.
confidence interval for beta
Interval for slope.
normal regression analysis
Regression with normal errors.
rank-correlation coefficient
Nonparametric measure of association based on ranks.
Spearman's rank-correlation coefficient
Nonparametric correlation based on ranks.
Analysis of Variance and Experimental Design
analysis of variance
A statistical method to compare means of three or more groups by analyzing variance components.
one-way
ANOVA for comparing means across one factor's levels.
two-way
ANOVA for comparing means across two factors and their interaction.
block
A grouping factor in experimental design to control for variability.
effects
The impact of the blocking factor on the response.
sum of squares
The portion of total variance attributed to blocks in ANOVA.
completely randomized design
An experimental design where treatments are assigned randomly to units without blocking.
confounding
When the effect of a variable is mixed with another, making it hard to isolate causes.
controlled experiment
An experiment where variables are manipulated and controlled to test causal relationships.
Duncan multiple-range test
A post-hoc test for comparing means in ANOVA.
effects
The impact of factors in an experiment.
interaction
When the effect of one factor depends on another.
main
The direct effect of a factor, averaging over others.
treatment
The effect of experimental treatments.
error mean square
The variance estimate in ANOVA from residuals.
experimental design
The plan for conducting experiments to minimize error and bias.
Latin square
Design to control two sources of variation.
incomplete block
Block design where not all treatments are in each block.
factorial experiments
Experiments testing multiple factors and their interactions.
2^n factorials
Full factorial with n factors at 2 levels each.
factors
Independent variables in an experiment.
fractional replicate
A subset of a full factorial design to reduce experiment size.
incomplete block designs
Block designs where blocks do not contain all treatments.
interaction
When the effect of one factor depends on the level of another.
Latin square design
Design to control two blocking factors.
least significant range
The smallest difference considered significant in multiple comparisons.
main effects
The average effect of a factor across levels of others.
mean square
Sum of squares divided by degrees of freedom.
treatments
MST in ANOVA.
multiple comparisons
Post-hoc tests after ANOVA to compare pairs.
multiple-range test
A method for pairwise comparisons in ANOVA.
one-way analysis of variance
ANOVA for one factor.
replication
Repeating treatments in experiments.
sum of squares
Sum of squared deviations.
treatment effects
Differences due to treatments.
treatment mean square
MST in ANOVA.
treatments
Levels of the factor under test.
two-way analysis of variance
ANOVA for two factors.
Nonparametric Methods
H test
The Kruskal-Wallis test for comparing medians across groups.
Kruskal-Wallis test
Nonparametric test for comparing medians across groups.
Mann-Whitney test
Nonparametric test for comparing two independent samples.
nonparametric statistics
Methods that do not assume a specific distribution.
rank-sum tests
Nonparametric tests using ranks.
runs, test based on
Nonparametric test for randomness.
sign test
Nonparametric test using signs of differences.
one-sample
Sign test for median.
paired-sample
Sign test for paired data.
signed-rank test
Wilcoxon test using signed ranks.
U test
Mann-Whitney U test.
Wilcoxon signed-rank test
Nonparametric test for paired data using ranks.
Wilcoxon test
Rank-sum test for two samples.
Data Description and Visualization
bar chart
A graphical representation of categorical data using bars of different heights.
boundary, class
The endpoints of intervals in a frequency distribution or histogram.
class
An interval or category in a frequency distribution.
boundary
The lower and upper limits of a class interval.
frequencies
The number of observations in each class.
interval
The range of values for a class in a histogram.
limits
The endpoints defining a class interval.
mark
The midpoint of a class interval.
double stem display
A variation of stem-and-leaf plot for finer detail.
exploratory data analysis
Techniques to summarize and visualize data patterns.
frequency, class
The number of data points in a class interval.
frequency distribution
A table or graph showing how data is distributed across classes.
cumulative
Shows running total frequencies.
frequency polygon
A line graph connecting midpoints of histogram bars.
histogram
A bar graph of frequency distribution for continuous data.
skewed
A histogram with asymmetry in distribution.
kurtosis
A measure of tailedness or peakedness in a distribution.
leaf
Digits in a stem-and-leaf plot representing data values.
mean
The average value of a distribution or sample.
median
The middle value in an ordered data set.
mode
The most frequent value in data.
ogive
Graph of cumulative frequencies.
outlier
An extreme value far from other data points.
range
Difference between max and min values.
sample
Range of sample data.
skewness
Measure of asymmetry in a distribution.
standard deviation
Square root of variance, measure of dispersion.
sample
Square root of sample variance.
stem
Leading digits in stem-and-leaf plot.
variance
Expected squared deviation from mean.
sample
Average squared deviation from sample mean.
Game Theory and Decision Making
Bayes criterion
A decision rule that minimizes the expected loss or risk under a prior distribution.
Bayes risk
The expected risk under a prior distribution in Bayesian decision theory.
decision function
A rule that maps observations to decisions in statistical decision theory.
equitable game
A game where expected payoffs are zero or fair.
games, theory of
The study of strategic decision-making.
loss function
A function quantifying the cost of errors in decisions.
minimax criterion
Minimizing the maximum possible loss.
minimax strategy
The strategy achieving the minimax criterion.
payoff matrix
Table of outcomes in a game.
pure strategy
A fixed choice in game theory.
randomized strategy
A mixed strategy in games.
regret
Difference from optimal payoff in decision theory.
risk function
Expected loss as a function of parameter.
saddle point
Equilibrium in zero-sum games.
statistical games
Games analyzed with statistical methods.
strategies
Choices in game theory.
zero-sum two-person game
Game where one player's gain is the other's loss.
Other Concepts
acceptable quality level
The maximum proportion of defective items considered acceptable in a lot during quality inspection.
acceptance number
The maximum number of defects allowed in a sample before rejecting the lot in acceptance sampling.
acceptance region
The range of test statistic values for which the null hypothesis is not rejected.
acceptance sampling
A quality control method where a sample is tested to decide whether to accept or reject a lot of products.
AQL
Abbreviation for acceptable quality level.
binomial coefficient
The number of ways to choose k items from n, denoted C(n,k) or n choose k.
computer simulation
Using computational methods to mimic random processes or systems.
consumer's risk
The probability of accepting a bad lot in acceptance sampling (Type II error).
continuity correction
An adjustment to approximate a discrete distribution with a continuous one, like normal for binomial.
count data
Data consisting of non-negative integers representing counts.
counting
The process of determining the number of elements in a set or arrangements.
extrapolation
Predicting outside the range of observed data.
failure rate
The instantaneous rate of failure for a system or component.
finite population correction factor
An adjustment to variance when sampling without replacement from a finite population.
Jacobian
The determinant of the matrix of partial derivatives in transformations.
likelihood function
The probability of data as a function of parameters.
lot tolerance percentage defective
The defect rate a consumer is willing to tolerate with low probability of acceptance.
Markov's inequality
A bound on the probability of deviation from the mean.
multiplication of choices
Counting principle for sequential choices.
OC curve
Operating characteristic curve showing acceptance probability vs quality.
operating characteristic curve
Plot of power vs parameter values under alternative.
outcome (of experiment)
A single result from an experiment.
partition
Dividing a set into disjoint subsets.
Pascal's triangle
Triangular array for binomial coefficients.
Petersburg paradox
A paradox where expected value is infinite but fair price is finite.
pivotal method
Using pivotal quantities for inference.
Poisson process
A process counting events at constant rate.
population
The entire set of interest.
median
Population middle value.
probability integral transformation
Transforming to uniform using CDF.
producer's risk
Probability of rejecting a good lot (Type I error in sampling).
random error
Unsystematic variability in measurements.
randomized response technique
A method to elicit truthful answers to sensitive questions.
repeated trials
Independent replications of an experiment.
standard form (standardizing)
Transforming to z-score: (x - μ)/σ.
standard error of the mean
Standard deviation of sample mean distribution.
statistic
A function of sample data.
triangular probability density
A continuous distribution with triangular PDF.
variance-ratio distribution
F distribution.
waiting time
Time until an event in a process.