The ANOVA F-test is used to test for difference in means between groups, and requires the conditions of normality (or large sample size), independence, and constant variance in order to be valid. This app evaluates robustness of the ANOVA F-test to violation of the constant variance condition. At left, specify the sample sizes and standard deviations for each group. Below left are simulated data from normal distributions with the specified standard deviations and mean zero. In the right plot, the F-statistic for the simulated data is added to the sampling distribution. The critical value for a 0.05 significance test is shown in red.
Type I error
power
1. If conditions for ANOVA are satisfied, the Type I Error rate should be equal to 0.05. Simulate data that satisfy conditions and verify that this is true. Perform several hundred simulations to get a good estimate for the error rate.
2. Simulate samples of size 20 from populations with equal means and standard deviations 6, 6, and 6. What is your Type I error rate?
3. Simulate samples of size 20 from populations with equal means and standard deviations 4, 6, and 8. Now what is your Type I error rate?
4. Simulate samples of size 20 from populations with equal means and standard deviations 1, 6, and 11. Now what is your Type I error rate?
5. Do the error rates you found in 2, 3, or 4 vary by sample size, when sample sizes are equal?
6. Next repeat your simulation study with sample sizes 10, 20, and 30. How do results differ?
7. Finally, repeat the above simulation studies, but specify population means to be -3, 0, and 3, so that you study the power of the test under different conditions.