LINK TO SUMMARY SLIDE FOR VIDEO:

StatsExamples-f-test-examples.pdf

TRANSCRIPT OF VIDEO:

Slide 1.

Sometimes we need to know whether a set of more than two variances are equal. For example, if we want to do an ANOVA, that test requires that the variances of all the compared groups be equal. The most simple way to test for this is a technique called the F-max test.

Slide 2.

Let's think about the following scenario, we want to know if the variances of more than two populations differ.
For this example we're looking at 4 populations.
► If it was just 2 populations, we do could a variance ratio F-test. If you're unfamiliar with this, I suggest taking a look at our F-test video on this channel.
That test calculates a test statistic by taking the larger sample variance and dividing it by the smaller sample variance. Larger test statistic values tend to convince us that the population variances may be different.
► So in theory we could do a bunch of F-tests of all the pairs of sample variances taken from our populations.
► However, each test we do has a risk of type I error. For this scenario that means that if we wanted to compare these 4 populations we would need to do 6 F-tests and each of them has a 0.05 probability of a type I error.
By the time we do them all we're looking at a thirty percent chance of making at least one type I error.
This illustrates a major problem with comparing multiple populations. When we do many comparisons, the overall risk of type I error is too high by the time we're done.
► We need a better statistical test, a test where the overall risk of type I error is 5% which we typically find to be an acceptable risk.

Slide 3.

We can visualize what this test would look like.
First, our default or null hypothesis will be that the population variances are all equal as shown in the equation.
If we took a sample from each population and got values like shown on the bottom left, where the sample variances are all fairly similar, this is the likely result if the variances of the populations are equal.
We should get a p-value larger than 0.05 for our statistical test.
On the other hand, if we took a sample from each population and got sample variances values like shown on the bottom right, where some values seem much smaller or larger than one another, this is an unlikely result if the variances are equal.
We should get a p-value less than 0.05 for our statistical test.
Now we just need a way to calculate a test statistic and a way to get p-values.

Slide 4.

We'll develop this new test from our F-test for two populations.
When we choose two samples from populations with the same variances, and divide them by each other, the value follows the F distribution.
This is the basis of the F-test which uses the test statistic shown on the top to the right. We compare this test statistic to the F distribution to get the p-value.
It turns out that when we choose many samples from populations with the same variances, and divide the largest by the smallest, the value follows the F-max distribution.
This will be the basis of our F-max test which uses the second test statistic shown to the right. We compare this test statistic to the F-max distribution to get the p-value.
 
Slide 5.

Let's review the F-test.
There was a different distribution for each pair of degrees of freedom values.
Since there were two tails for our test, because either variance could be larger, the two-tails meant using the alpha equals 0.025 area at top of the F distribution.
We calculated an F value and compared it to the F values on the X-axis of the probability distribution to see if our calculated value was in the top 2.5% of the distribution.
If that happened, we knew our overall p-value was less than 5% and we could reject the null hypothesis. We would conclude that one variance was larger than the other.
If the F calculated value was smaller and that didn't happen, we knew our overall p-value was larger than 5% and we would fail to reject the null hypothesis. We would not be able to conclude that either variance was larger than the other.

Slide 6.

For the F-max test things are only slightly different.
There is a different F-max distribution for each number of groups and degrees of freedom for each group. Usually we try to ensure that our samples from each population are identical in size.
Unlike the F-distribution which has both one-tailed and two-tailed applications, the F-max distribution is only used for this test. The distribution incorporates the many tails and we would always use the alpha equals 0.05 area at the top. We would calculate an F-max value and compared it to the F-max values on the X-axis of the probability distribution to see if our calculated value is in the top 5% of the distribution.
If this happens, we know our overall p-value is less than 5% and we can reject the null hypothesis. We would therefore conclude that at least one variance is larger than at least one other variance.
If the F-max calculated value is smaller and it doesn't fall in the top 5% of the distribution, we knew our overall p-value is larger than 5% and we would fail to reject the null hypothesis. We would not be able to conclude that any of the variances are larger than any of the others.

Slide 7.

Tables of critical values for the F-max test are usually presented with a separate table for each critical value because there's a different distribution for each number of groups and degree of freedom for each group.
► Here are two of the F-max critical value tables from the StatsExamples website, for alpha values of 0.05 and 0.01. These tables are arranged with the columns corresponding to the number of groups of populations being compared and the rows corresponding to the degrees of freedom in each group.

Slide 8.

OK, let's take a look at the formal procedure for doing an F-max test.
First, same as with every statistical test, we set up null and alternative hypotheses.
The null hypothesis is that all the population variances are equal, the variance of population one equals the variance of population 2 equals the variance of population 3, etc.
If we're doing this test as a pre-test before an ANOVA, this is the result we would want. The alternative hypothesis is that the population variances are not equal, the variance of at least one population is different from at least one of the others.
► The next step is to create our test statistic, the F-max value.
For this we calculate all of the sample variances and divide the largest variance by the smallest variance.
► Then we compare our F-max calculated value to various F-max critical values from the F-max distribution.
As mentioned, there is a different F-max distribution for each combination of the number of groups and the degrees of freedom within each of these groups.
In the F-max table shown, available on the StatsExamples website, the columns indicate the number of groups and each row corresponds to the degrees of freedom for each group.
Our p-value is the area to the right of our calculated value.
The values within this table are the critical values that correspond to an Alpha value of 0.05. They show how far out on each X axis we need to go so that 5% of the area under the F-max distribution curve is to the right.
If the calculated F-max value is larger than the critical value then the p-value is less than 0.05
► By looking at a variety of tables, each corresponding to a different alpha value, we can determine the smallest alpha value that we would be able to use and still reject the null hypothesis when our F-max calculated value is larger than the F-max critical value.
That minimum alpha value is our probability, the P-value, of seeing an F calculated value as large as we do.
Effectively, this is the exact probability of getting an F-max as large we do if the null hypothesis of equal population variances is true.
If we have access to a computer, the exact P-value can be calculated directly.
► We then decide whether to "reject the null hypothesis" or "fail to reject the null hypothesis" based on the P-value.
The null hypothesis, the variances of the populations all being equal corresponds to most P-values except for very small ones.
The alternative hypothesis, that at least one variance is bigger than at least one other, would give us small P-values.
Typically, the threshold we use is 5% so when our P-value is less than 0.05 we reject the null hypothesis and conclude that the population variances are not all equal.
When our P-value is not less than 0.05 we fail to reject the null hypothesis. This decision indicates that we lack the evidence to conclude that any of the variances are different from each other, so we would generally assume that they are the same.
Keep in mind, as with all statistical tests, we are not proving either hypothesis, we are making a decision about which one is likely based on the probability of seeing the test statistic we do. We should always keep in mind that we may be making a type one or type two error.

Slide 9.

As mentioned, the F-max test most often arises as a precursor to doing an ANOVA. Let's think about the flow of decisions and tests we may take when looking at this situation.
Consider a situation in which we want to do an ANOVA on our data sets.
► First thing to do is check for the equality of variances using the F-max test or some other technique.
► If the results of that test lead us to conclude that the variances are equal, we can go ahead and do the ANOVA. This is what we want.
► On the other hand, if the results of that test lead us to conclude that the variances are not equal, we can't do the ANOVA. We then ask ourselves, has the data been transformed?
► If not, then we transform the data and then check the equality of variances of these transformed values.
The companion video to this one, walking step by step through a couple of examples, shows how this is done. ► If that doesn't work, we can try other transformations and see if they work. Often we'll try a bunch of these because what we really are hoping for is to get a transformation that equalizes the variances so we can do the ANOVA.
► If none of our transformations work, then we have to use some other statistical test. We generally prefer not to do this because other tests are more complicated and have less statistical power than the ANOVA.

Slide 10.

We'll finish with a few notes about the F-max test.
► First, the F-max test is also called Hartley's F-max test for the homogeneity, or equality, of variance.
The last part is just descriptive and the first part comes from Herman Hartley, a German statistician who earned degrees from the universities of Berlin and Oxford and worked in the United States.
► Next, similar to the regular F-test, the F-max test fails if the distribution of values is not normal.
When the data isn't normally distributed this test is prone to making type I or type II errors.
► The F-max test also required equal sample sizes for our groups.
If the sample sizes from the population aren't all the same, we should use the Barlett's, Levene's, or Brown-Forsythe test instead.
These tests also don't have the same extreme sensitivity to when the data isn't normally distributed.
► Lastly, when would we do this test or one like it.
Tests for equality of multiple variances are typically done before using an ANOVA technique because these methods require equal variances for all the groups they compare.

Zoom out.

I hope you found this introduction to the F-max test useful.
This is a fairly straightforward test which, even though it shouldn't be used if the population values are not normally distributed, is used by people all the time.

End screen.

Go ahead and share, like, or subscribe if you found this video useful.