Parameters for the binomial probability mass function

Shiny app by
Gail Potter

Base R code by
Gail Potter

Shiny source files:
GitHub Gist

You have learned about the
*probability mass function*
(PMF) for the binomial random variable.
This is a function which has two parameters, n (number of trials) and p (probability of success), which determine its shape.
The function takes as input the number of successes, and gives as output the probability of that many successes in n trials.
The equation for the probability mass function is $$P(X=x) ={n \choose x} p^x (1-p)^{n-x}$$

In an experiment, you usually don't know which of these possible PMFs is the truth, and
you observe a single value of x, the number of successes.
The goal is to estimate p based on your observation, x.
This motivates the
*likelihood function.*
In the likelihood function, the functional form
is the same, but we treat p as variable and x, as fixed. The number of trials, n, is also fixed (by the experimental design).

Specifications for the likelihood function:

Shiny app by
Gail Potter

Base R code by
Gail Potter

Shiny source files:
GitHub Gist

Since the likelihood function treats p as a variable and x as fixed, it is written L(p|x, n). The likelihood function for this example is $$L(p|x, n) ={n \choose x} p^x (1-p)^{n-x} $$

For different values of x and n, determine the value of p where the likelihood function achieves its max. This value of p is called the Maximum Likelihood Estimate (MLE) for p. Can you derive the formula for this estimator of p, in terms of x and n?

Specifications for the log likelihood function:

Shiny app by
Gail Potter

Base R code by
Gail Potter

Shiny source files:
GitHub Gist

In practice, it is usually easier to find the MLE by maximizing the
*log likelihood function*
instead of the likelihood function. Since the log transformation is monotone,
the two functions achieve their maximum in the same place. The log likelihood function for this example is
$$ \log (L(p|x, n)) = \log \Big( {n \choose x} p^x (1-p)^{n-x} \Big) $$

We have introduced the concept of maximum likelihood in the context of estimating a binomial proportion, but the concept of maximum likelihood is very general. Maximum likelihood is used to estimate parameters for a wide variety of distributions.