Study note for Continuous Probability Distributions

Basics of Probability

  • Probability density function (pdf). Let X be a continuous random variable. Then a probability distribution or probability density function (pdf) of X is a function f(x) such that any two numbers a and b with Study note for Continuous Probability Distributions
    Study note for Continuous Probability Distributions

    That is, the probability that X takes on a value in the interval [a, b] is the area above the interval and under the graph of the density function. The graph of f(x) is often referred to as the density curve.

    • The pdf is a function that describes the relative likelihood for the random variable to take on a given value. Intuitively, one can think of f(x) as being the probability of a random variable X falling within the infinitesimal interval [x, x+dx]. My understanding: a probability is regarded as an absolute likelihood?
    • Study note for Continuous Probability Distributions for all x;
    • Study note for Continuous Probability Distributions
    • For continuous random variable X, the probability for any single possible value is 0: Study note for Continuous Probability Distributions
    • Intuitively, since continuous variable may have infinity possible values, and hence for each single value, the  probability will be extremely small (the chance of a specific event occurring is rare) and approximating 0 by the limitation. On the other hand, for a continuous random variable, it is more meaningful to look at the probability in a certain interval than the probability at a specific point.
    • A continuous random variable usually represents events related to measurements.
  • In mathematics, a moment is, loosely speaking, a quantitative measure of the shape of a set of points
    • The first moment, or the raw moment refers to the meanof a point distribution.
    • The second moment, or the central moment is the variance. The normalized n-th central moment or standardized moment is the n-th central moment divided by Study note for Continuous Probability Distributions; the normalized n-th central moment of Study note for Continuous Probability Distributions
    • The third central moment is the skewness.
    • The fourth central moment is called "kurtosis", a measure of whether the distribution is tall and skinny or short and squat, comparing to the normal distribution of the same variance.
    • High-order moments are moments beyond 4th-order moments.
  • Likelihood is a function of how likely an event is, which is weaker than probability. In statistics, probability is the function of data given the parameters while likelihood is the function of parameters given the observed data.

Uniform Distribution

  • The uniform distribution is summarized as follows:
    • notation: U(a, b), where a, b are the minimum and maximum values of a uniform distribution, a<b.
    • p.d.f: 
      Study note for Continuous Probability Distributions
    • mean: 1/2 * (a+b)
    • variance 1/12 * (b-a)2

Normal Distribution

  • The normal (Gaussian) distribution is summarized as follows:
    • notation: Study note for Continuous Probability Distributions, where Study note for Continuous Probability Distributions is the mean of the distribution, andStudy note for Continuous Probability Distributions is the standard deviation. if Study note for Continuous Probability Distributions, the distribution is called the standard normal distribution.
    • p.d.f: 
      Study note for Continuous Probability Distributions
    • mean: Study note for Continuous Probability Distributions
    • variance: Study note for Continuous Probability Distributions
    • P(a<x<b): the integral for arbitrary a and b cannot be evaluated analytically. Hence, it is usually converted to a standard normal distribution (a.k.a standardization) from which the c.d.f can be directly read from a table.
  • Normal distribution are often used in the natural and social sciences for real-valued random variables whose distributions are not known.
  • Standardization: if X is a normal random variable with mean Study note for Continuous Probability Distributions and standard deviationStudy note for Continuous Probability Distributions, thenStudy note for Continuous Probability Distributions is a standard normal random variable.
  • Central Limit Theorem
    • Gaussian distribution is important because of the central limit theorem
    • A crude statement of the central limit theorem: things that are the result of the addition of lots ofsmall effects tend to become Gaussian. That is, no one term in sum should dominate the sum.
    • A more exact statement:
      • Let Y1, Y2, ..., Yn be an infinite sequence of independent random variables (that may be from different pdf), each with the same probability distribution
      • Suppose that the mean and variance of this distribution are bothfinite.
      • For any numbers a and b: 
        Study note for Continuous Probability Distributions
    • It tells us that under a wild range of circumstances the probability distribution that describes the sum of random variables tends to a Gaussian distribution as the number of terms in the sum Study note for Continuous Probability Distributions

Multivariate Distributions

  • We can generalize the definition of random variables to vectors. A vector Study note for Continuous Probability Distributions is a vector whose componentsStudy note for Continuous Probability Distributions are univariate random variables. IfStudy note for Continuous Probability Distributions are all discrete, thenStudy note for Continuous Probability Distributions is a discrete random vector. IfStudy note for Continuous Probability Distributions are all continuous, Study note for Continuous Probability Distributions is called a continuous random vector.
  • The distribution of a random vector is characterized by the joint c.d.f that is defined as: 
    Study note for Continuous Probability Distributions

References

  1. Paola Sebastiani, A Tutorial on Probability Theory
上一篇:< 独立项目 - 文本挖掘 > - 2016/11/13 第二更 -


下一篇:Dive into Python