Statistical tests

• This page contains general information for choosing commonly used statistical tests. The examples linked provide general guidance which should be used alongside the conventions of your subject area. Where possible, a brief explanation of the test is given with links to performing this test using Excel, SPSS and R. It is worth noting that the examples often contain information about interpreting the output and results so can act as a guide to interpreting statistical results too.

Categorical data

To navigate this table, consider the following questions:

• Is your outcome variable categorical?
• How many samples (or groups)  do you have?
• Are the outcomes paired (or dependent)?

Chi-square goodness-of-fit

A nonparametric test designed to see if the sample follows a known distribution.

Binomial Logistic Regression

A statistical model used when analysing dichotomous data that may depend on a number of factors. The logistic regression model allows the effects of all factors to be studied simultaneously. Results are usually expressed in terms of odds ratio.

2 Samples

Paired

(or dependent)

McNemar's test

Comparing differences between paired (or related) dichotomous data.

Agreement

(2 raters)

Cohen's Kappa

A measure of agreement between two raters recording a categorical outcome variable.

2 (+) Samples

Unpaired

(or independent)

Chi-squared test of association

A nonparametric test designed to explore if there is an association or relationship between two categorical variables.

2 (+) Samples

Unpaired

(or independent)

Fisher's exact

A nonparametric test designed Expected values (E ≤ 5) in for more than 25% of cases. The example describes a Chi-squared test and details when to use the Fisher's exact test and details which output variables to report.

Cochran's Q-test

Cochran's Q-test determines if there are differences between three or more related groups.

Continuous data

To navigate this table, consider the following questions:

• Is your outcome variable continuous?
• Does the data meet the requirements for a parametric test (e.g. normality)? Notice, that for each parametric test, where possible, a corresponding nonparametric test is presented.
• How many samples (or groups)  do you have?
• Are the outcomes paired (or dependent)?

Parametric

One sample t test

If sigma is unknown, use a one sample t test to determine if the sample is likely to have come from a given population with a defined mean.

2 Samples

Paired

(or dependent, repeated measures)

Paired t test

The paired-samples t test is used when the data is from related, paired or longitudinal samples.

2 Samples

Unpaired

(or independent)

Unpaired t test.

An unpaired t test is used to assess if the mean values of two independent samples are equal. Firstly, you need to assess equality of variances using an F-test, details of which are given within the examples below.

3+ Samples

Related (or dependent)

One-way repeated measures (within groups) ANOVA

One-way repeated measures analysis of variance (ANOVA) is a method for detecting differences between related mean values, it is an extension of the paired t test. A post-hoc test is needed to investigate where these differences might occur.

3+ Samples

Unrelated (or independent)

One-way (between groups) ANOVA

One-way analysis of variance (ANOVA) is a method for testing whether three or more populations have the same mean value and is an extension of the unpaired t test. A post-hoc test is needed to investigate where these differences might occur.

3+ Samples

Unrelated (or independent)

Two-way (between-groups) ANOVA

Two-way (or three-way analysis of variance) is used to explore if two or more factors can influence the dependent variable.

Pearson's Correlation

Pearson’s Product-Moment Correlation (r) is used to measure the strength and direction of the association between two variables. The value of Pearson’s r is between +1 and –1: where r = +1 is a perfect positive correlation, r = -1 a perfect negative correlation and r=0 indicates no correlation between the variables.

Nonparametric

One sample sign (or median) test

A one sample sign test is used to explore if the median of the sample data is equal to a given value.

2 Samples

Paired

(or dependent)

Wilcoxon signed rank test

The Wilcoxon signed ranks test is used to compare the medians of two related samples.

2 Samples

Unpaired

(or independent)

Mann-Whitney U test

The Mann-Whitney U test is used to compare the medians of two independent samples.

3+ Samples

Related (or dependent)

Friedman test

The Friedman test is designed to test whether three or more populations have the same median values, using data collected from related samples. It is the nonparametric equivalent of a simple repeated measures analysis of variance (ANOVA).

3+ Samples

Unrelated (or independent)

Kruskal-Wallis test (independent observations)

The Kruskal Wallis test tests whether three or more populations have the same median values. It is the nonparametric equivalent of a one-way analysis of variance (ANOVA).

Spearman's Rank test

Spearman’s correlation coefficient rho (ρ) is calculated from the ranked data and is used to measure the correlation between two variables The value of Spearman’s rho is between +1 and –1: and the sign and value of ρ are interpreted in the same way as the more conventional correlation coefficient, r.