Probability Distributions

Probability can be used for more than calculating the likelihood of one event; it can summarize the likelihood of all possible outcomes. A thing of interest in probability is called a random variable, and the relationship between each possible outcome for a random variable and their probabilities is called a probability distribution.

This tutorial is divided into four parts; they are:

• Random Variables
• Probability Distribution
• Discrete Probability Distribution
• Continuous Probability Distribution

1. Random Variables

A random variable is a quantity that is produced by a random process.

In probability, a random variable can take on one of many possible values, e.g. events from the state space. 

A specific value or set of values for a random variable can be assigned a probability.

A random variable is often denoted as a capital letter, e.g. X, and values of the random variable are denoted as a lowercase letter and an index, e.g. x1, x2, x3.

The values that a random variable can take is called its domain, and the domain of a random variable may be discrete or continuous.

• Discrete Random Variable: Values are drawn from a finite set of states.  For example, colors of a car.
  
• Boolean Random Variable: Values are drawn from the set of {true, false}.
• Continuous Random Variable: Values are drawn from a range of real-valued numerical values.

A value of a random variable can be specified via an equals operator: for example, X = True.

The probability of a random variable is denoted as a function using the upper case P; for example, P(X) is the probability of all values for the random variable X.

The probability of a value of a random variable can be denoted P(X = True), in this case indicating the probability of the X random variable having the value True.

2. Probability Distribution

A probability distribution is a summary of probabilities for the possible values of a random variable. 

As a distribution, the mapping of the values of a random variable to a probability has a shape when all values of the random variable are lined up. 

The distribution also has general properties that can be measured. 

Two important properties of a probability distribution are the expected value and the variance. 

Mathematically, these are referred to as the first and second moments of the distribution. 

Other moments include the skewness (3rd moment) and the kurtosis (4th moment).

• Expected Value: The average value of a random variable.
• Variance: The average spread of values around the expected value.

3. Discrete Probability Distributions

A discrete probability distribution summarizes the probabilities for a discrete random variable.

The probability mass function, or PMF, defines the probability distribution for a discrete random variable. It is a function that assigns a probability for specific discrete values. 

A discrete probability distribution has a cumulative distribution function, or CDF. This is a function that assigns a probability that a discrete random variable will have a value of less than or equal to a specific discrete value. F(x) = P(X<=x)

Some examples of well known discrete probability distributions include:

• Bernoulli and binomial distributions.
• Multinoulli and multinomial distributions.
• Poisson distribution.

Some examples of common domains with well-known discrete probability distributions include:

• The probabilities of dice rolls form a discrete uniform distribution.
• The probabilities of coin flips form a Bernoulli distribution.
• The probabilities car colors form a multinomial distribution.

4. Continuous Probability Distributions

A continuous probability distribution summarizes the probability for a continuous random variable. 

The probability distribution function, or PDF, defines the probability distribution for a continuous random variable.

Like a discrete probability distribution, the continuous probability distribution also has a cumulative distribution function, or CDF, that defines the probability of a value less than or equal to a specific numerical value from the domain. F(x) = P(X<=x)

As a continuous function, the structure forms a smooth curve. Some examples of well-known

continuous probability distributions include:

• Normal or Gaussian distribution.
• Exponential distribution.
• Pareto distribution

Some examples of domains with well-known continuous probability distributions include:

• The probabilities of the heights of humans form a Normal distribution.
• The probabilities of movies being a hit form a Power-law distribution.
• The probabilities of income levels form a Pareto distribution.