The objective of a two-sample equivalence test is to determine whether the means of two populations are equivalent based on two independent samples from these populations; here “equivalent” means that the two means differ by a small pre-defined amount. This margin of equivalence is determined by knowledge of the domain under study and represents the tolerance that is acceptable.

A two one-sided t-test (TOST) is used to make this determination. Essentially, TOST reverses the roles of the null and alternative hypotheses in a two-sided t-test.

If θ represents the margin of equivalence, then we test the hypotheses:

H₀: μ₂ – μ₁ ≤ –θ or μ₂ – μ₁ ≥ θ

H₁: –θ < μ₂ – μ₁ < θ

This is done by conducting two one-sided t-tests, each of which is based on a null-hypothesis that is one of the parts of the above null hypothesis. If the null hypothesis of both tests are rejected then the difference falls within the equivalence interval and you can claim that the two population means are equivalent. The larger p-value of the two t-tests is used as the p-value of the TOST.

Another way of looking at this method is to conduct a two-sided t-test and determine the 1–2α confidence interval I. If the confidence interval I lies completely within the interval (-θ, θ) then we accept that the two means are equivalent.

We can also use two different limits for the margin of equivalence, an upper value θ_U and a lower value θ_L. In this case, θ_U replaces θ in the TOST method described above and θ_L replaces –θ.

The TOST approach can also be used for a one-sample test (and similarly for a two dependent sample test). In this case, we test the equivalence between the mean of a single population and some hypothetical mean μ₀.

H₀: μ – μ₀ ≤ θ_L or μ – μ₀ ≥ θ_U

H₁: θ_L < μ – μ₀ < θ_U

This time we conduct two one-sample t-tests. Alternatively, if the 1–2α confidence interval for the two-tailed one-sample t-test lies completely within the interval (θ_L, θ_U) then we accept that the population mean and hypothetical mean are equivalent.

Example 1: A company that markets a premium brand of fresh salmon has found a second supplier, but wants to make sure that the level of omega-3 is within 25 mg of their existing suppliers’ for a 100 gram serving. Using the random sample of 12 servings from each supplier as shown on the left side of Figure 1 determine whether the sources are equivalent based on alpha = .05.

The right side of Figure 1 shows the analysis for a two independent sample t-test. Since the 90% confidence interval (-9.17, 23.33) is completely contained in the interval (-25, 25), we conclude that the two sources are equivalent.

Note that if we test the two one-sided null hypotheses directly, we would obtain p-values of .036 and .0013. Since both are less than alpha = .05, we again conclude that the two sources are equivalent (with p-value = .036).

Note too that a two-sided t-test would yield a p-value = .46 and so we can also conclude that there is no significant difference between the two suppliers.

Figure 1 – TOST