Parameters for the binomial probability mass function

Shiny app by Gail Potter
Base R code by Gail Potter
Shiny source files: GitHub Gist

### Probability Mass Function

You have learned about the probability mass function (PMF) for the binomial random variable. This is a function which has two parameters, n (number of trials) and p (probability of success), which determine its shape. The function takes as input the number of successes, and gives as output the probability of that many successes in n trials. The equation for the probability mass function is $$P(X=x) ={n \choose x} p^x (1-p)^{n-x}$$

In an experiment, you usually don't know which of these possible PMFs is the truth, and you observe a single value of x, the number of successes. The goal is to estimate p based on your observation, x. This motivates the likelihood function. In the likelihood function, the functional form is the same, but we treat p as variable and x, as fixed. The number of trials, n, is also fixed (by the experimental design).

Specifications for the likelihood function:

Shiny app by Gail Potter
Base R code by Gail Potter
Shiny source files: GitHub Gist

### Likelihood function

Since the likelihood function treats p as a variable and x as fixed, it is written L(p|x, n). The likelihood function for this example is $$L(p|x, n) ={n \choose x} p^x (1-p)^{n-x}$$

For different values of x and n, determine the value of p where the likelihood function achieves its max. This value of p is called the Maximum Likelihood Estimate (MLE) for p. Can you derive the formula for this estimator of p, in terms of x and n?

Specifications for the log likelihood function:

Shiny app by Gail Potter
Base R code by Gail Potter
Shiny source files: GitHub Gist

### Log Likelihood function

In practice, it is usually easier to find the MLE by maximizing the log likelihood function instead of the likelihood function. Since the log transformation is monotone, the two functions achieve their maximum in the same place. The log likelihood function for this example is $$\log (L(p|x, n)) = \log \Big( {n \choose x} p^x (1-p)^{n-x} \Big)$$

We have introduced the concept of maximum likelihood in the context of estimating a binomial proportion, but the concept of maximum likelihood is very general. Maximum likelihood is used to estimate parameters for a wide variety of distributions.