Set your Significance Level (α)
Once you have defined your statistical hypotheses, the next step is to select your chosen Significance Level (α, alpha). Many fields of study use α = 0.05 (or 5%) but this isn't always the case, be sure to check which level you should use.
The Significance Level is a value chosen by a researcher chooses as a cut-off to decide if a statistical test result is considered to be significant or not significant. This cut-off value is referred to as the critical value. For more details on this, please refer to What is a critical value? by Minitab support.
Finding a significant result does not guarantee that one really exists, and this 5% indicates the level of risk (or statistical error) that the researcher has chosen to be acceptable. In other words, there is a 5% chance of finding a significant result when one doesn't really exist - you've just found one by chance.
Statistical Errors
Finding a statistically significant result does not guarantee that there really is an association, a difference or a relationship between your variables.
When performing a statistical test, there are four possible outcomes. Two possibilities result in the correct decision and two the incorrect decision.
These incorrect decisions form the basis of Type I and Type II Errors. Each error has a probability of occurring.
- Type I Error has a probability of α, the level of significance.
- Type II Error has a probability of β, the statistical power.
For additional information on the implications of these errors take a look at What are type I and type II errors? a short blog with examples by Minitab support.
Statistical power (β)
What is statistical power (β, beta)? Statistical power is the probability of finding a true significant difference when one exists. For a more detailed explanation, take a look at What is power? by Minitab support.