Bonferroni adjustment

  • What is it?

    The Bonferroni method is a simple technique for controlling the overall probability of a false significant result when multiple comparisons are to be carried out.

    In a single hypothesis test, the risk of getting a statistically significant result, when no effect is present is set at alpha symbol = 0.05 or 5%. When multiple comparisons are made – for example when comparisons are made between a treatment group and a control group, or between men and women, using a whole list of variables or outcome measures, then the risk that one or more of the whole set of comparisons will generate a significant result when no effect is present will be higher than 5%. The Bonferroni method is used to reduce this risk.

    The reduction in risk is achieved by dividing the significance level alpha symbol = 0.05 by the number of comparisons made and using the result as the cut-off point for a significant result. This means that for the whole set of tests, the overall significance level does not increase to a level above 5%.

    Example

    A survey records the level of spending of men and women on 10 categories of consumer goods. The mean level of spending on each category will be compared, ie 10 tests will be carried out. To achieve an overall significance level of alpha symbol = 0.05, the Bonferroni method requires a significance level of alpha symbol/10 = 0.005. This means that to produce a significant result, a test must produce a P-value of P<0.005.

    The result of using a stricter cut-off point for the individual tests is that the overall significance level for the collection of 10 tests is maintained at 5%.

    Example PDF »

    Further information

    The following paper can be accessed online:

    J.M.Bland and D.G.Altman (1995) Multiple significance tests: the Bonferroni method, BMJ, 310:710.