Inferential statistics

  • Statistical tests are used as a way to quantify if there is an association, a difference or relationship between variables. These tests are performed on sample data, with the aim of inferring beyond the sample data to the wider population. Take a look at the following Minitab pages for an example of inferential statistics compared to descriptive statistics.

    The sections below provide a range of resources to help you navigate the steps involved in performing inferential statistics from defining your hypothesis, to performing the statistical test and finally interpreting your or published results. 


    Hypothesis testing 

    Starting with your research question, consider how you might write this in terms of a testable statement or statements (see the null and alternative hypotheses). This will help you think about what type of data to collect and what type of statistical test is appropriate to use to answer your research question.

    • Start with your research question.
    • Define the Null and Alternative Hypotheses.
    • Collect sample data.
    • Set the Significance Level (alpha).
    • Choose and perform an appropriate statistical test.
    • Interpret the results.
    • Reporting the results  - linking back to your research question.
    The sections below detail the key stages of hypothesis testing.


  • Define your null and alternative hypothesis. 

    Different terminology is used to refer to the Null (Ho or Hn) and Alternative hypothesis (H1 or Ha) . Be sure to use what is expected of you in your subject area.

         H0: A null hypothesis usually assumes no association, no difference or no relationship between variables.

         H1: An alternative hypothesis opposes the null hypothesis, often assuming an association, a difference or a relationship between variables. 


    Choose either a one-tailed or two-tailed test.

         A two-tailed test looks for differences in either direction.

         A one-tailed test looks for differences in one direction only.

    For additional information and examples take a look at the following resources:


    Need to know more?

    • Take a look at this short guide from Laerd Statistics that outlines the process of hypothesis testing. Starting with your research question, examples are given on how to define a null and alternative hypothesis and the implications these have on the statistical test.
    • An example of performing a basic hypothesis test by Minitab support (2019).

    Set your Significance Level (α)

    Once you have defined your statistical hypotheses, the next step is to select your chosen Significance Level (α, alpha). Many fields of study use α = 0.05 (or 5%) but this isn't always the case, be sure to check which level you should use. 

    The Significance Level is a value chosen by a researcher chooses as a cut-off to decide if a statistical test result is considered to be significant or not significant. This cut-off value is referred to as the critical value. For more details on this, please refer to What is a critical value? by Minitab support.

    Finding a significant result does not guarantee that one really exists, and this 5% indicates the level of risk (or statistical error) that the researcher has chosen to be acceptable. In other words, there is a 5% chance of finding a significant result when one doesn't really exist - you've just found one by chance. 


    Statistical Errors

    Finding a statistically significant result does not guarantee that there really is an association, a difference or a relationship between your variables. 

    When performing a statistical test, there are four possible outcomes. Two possibilities result in the correct decision and two the incorrect decision.

    Error Decision

    These incorrect decisions form the basis of Type I and Type II Errors. Each error has a probability of occurring.  

    • Type I Error has a probability of α, the level of significance.
    • Type II Error has a probability of β, the statistical power.

    For additional information on the implications of these errors take a look at  What are type I and type II errors? a short blog with examples by Minitab support. 

    Type I and II Error

     


    Statistical power (β)

    What is statistical power (β, beta)? Statistical power is the probability of finding a true significant difference when one exists. For a more detailed explanation, take a look at What is power? by Minitab support.

    Which statistical test should I use?

    The type of statistical test to choose depends on the type of data that you have, and your research question. One way to choose an appropriate statistical test is to make use of a statistical flowchart. This flow chart for selecting commonly used statistical tests by Gerwien (2014) will help you visualize the steps involved.


    Choose your statistical test

    Click on the links below for examples of commonly used statistical tests. For each test there is a brief description outlining what the test can be used for along with additional links on how to perform this test using different statistical software packages.

    Test statistic value or P-value


    There are two ways to determine if the output from a statistical test can be considered as significant or not significant. 

    • The first approach compares the test statistic value to the critical value.

    • The second approach compares the p-value to the significance level.

    The critical value and the p-value approach to hypothesis testing is an excellent resource describing the two approaches (Hartmann, Krois and Waske, 2018).


    Need to know more?


    Effect size

    A significant result does not necessarily translate to a meaningful effect. Take a look at What does effect size tell you? for details on how to calculate and interpret effect sizes and why it is important to report them (McLeod, 2019).

    Using Effect Size - or Why the P Value is Not Enough (Sullivan and Feinn, 2012).



    The overall aim of gathering sample data and performing statistical analysis is to substantiate your research findings. Begin by reporting key descriptive statistics - these will help summarize key characteristics of your dataset for the reader. 


    Reporting and interpreting descriptive statistics 

    Key measure to report might include:

    • Sample size, number of participants - provide full details.
    • Demographics.
    • Frequencies, percentages and/ or proportions.
    • Average values and measures of spread.
    • Confidence intervals.

    Information can be succinctly summarized using tables and/or graphs. Take a look at your lecture notes and/ or further reading for ideas on how to present these findings.


    Reporting inferential statistics

    • Inferential statistics allow you to make inferences (or predictions) about a population based on the results of the sample data. For guidance on how to report your inferential statistics, take a look at your lecture notes. Different subject areas use different conventions, if you are in doubt, ask your module lead for guidance.

    • If you are looking for inspiration, worked examples for commonly used statistical tests can be found here. These examples often take you through how to perform the test in addition to how to interpret the output. 

    Need to know more?


  • Additional statistical resources

    Statistics uses lots of different terminology and types of tests. Here are some additional resources to help navigate statistical analysis.

  • Know your variables

    In order to choose an appropriate statistical test, it is important to be able to identify your Independent and Dependent Variables.


    Changing the independent variable causes a change in the dependent variable.

    • Independent Variable (IV) is the variable that is being changed by the researcher.
    • Dependent Variable (DV) the output or effect that you are expecting to change as a result of changing the independent variable.

    One confusion in identifying these is that they can be referred to by different names. Here are some words that can help us identify these variables:

    Independent Variable

    cause, explanatory, predictor.

    Dependent Variable

    effect, observed, measured.

    When plotting these variables, the independent variable is normally plotted along the x-axis while the dependent variable is normally plotted along the y-axis.


    Examples

    Take a look at the following pages for examples of independent and dependent variables.

    • A detailed guide on the different Types of Variable with worked examples by Laerd Statistics.

    What is the difference between paired (dependent or related) and unpaired (independent or unrelated) data?

    • Paired or related data are two or more data points taken from the same sample. When data is paired (or repeated) there exist the same number of data points in each sample.

    • Unpaired or unrelated data are two or more data points taken from different samples. When data is unpaired (or unrelated) the number of data points in each sample might differ.

    A short guide on how are dependent and independent samples different? by Minitab Express Support (2019).

    What is the difference between parametric and nonparametric?

    • Parametric tests are used to test sample means (average values).
      These tests are often used on continuous scale data and often require normally distributed data. Details on how to test for normality can be found here. Researchers like using parametric tests as they are more likely to detect a significant result if one really exists.

    • Nonparametric tests are used to test sample medians (midpoints of ordered data).
      These tests can be used on scale data that does not require the assumption of normally distributed data. These tests are advisable if you have a small sample size.

    Take a look at this short blog by Minitab Blog Editor (2015) for additional details on Choosing between a nonparametric test and a parametric test.

    If you are tasked with analysing  numerical (scale) data, then you should first check to see if your data is normally distributed. Depending on the task at hand, it might be that either a histogram or box-and-whisker plot is sufficient.


    What is a normal distribution?

    Take a look at this page by Maths is Fun for a brief introduction to the  normal distribution.


    Why do I need to test for normality?

    If your data is normally distributed then you can use parametric statistical tests. This is often sought after as a parametric test is more likely to detect a difference when one exists.


    How do I test for normality?

    Here are some short guides on how to formally test data to determine if it can be considered as normally distributed (or not).


    What should I do if my data is not normally distributed?

    Confidence Intervals (CI)

    A confidence interval estimates the range of values that’s likely to include the true population value, with a specified level of certainty e.g. 95%. Take a look at this introduction to confidence intervals by Maths is Fun for details on how this value is calculated.

     

    Odds Ratio (OR)

    The odds ratio is the probability (or odds) of an event happening expressed as a proportion of the probability (or odds) of the event not happening.

    • When OR < 1: A decrease in the event happening.

    • When OR = 1: No change.

    • When OR > 1: An increase in the event happening.

    Odds ratios OR are often reported alongside a confidence interval (CI).



    Meta-analysis is a quantitative approach used to systematically combine and analyze published data for similar research areas in order to draw conclusions about that research area.

     


  • Statstutor

    A group of UK universities have developed materials for  Statstutor a website with resources for anyone at any level in a range of formats, from 3G mobile to video. Use their coloured menus (or our links below) to take you to materials grouped in different ways – everything is organised A-Z so it's easy to find what you want.


    Software training


    Recommended books