The Correct Interpretation of Confidence Intervals

RelaTIonshIp beTween ConfIdenCe InTeRvals and p-values CIs and p-values are closely related although they provide different information. While p-values are the outcome of hypothesis tests and indicate whether or not the sample data provide sufficient evidence to reject the null hypothesis (e.g. whether an observed difference between 2 treatments is because the 2 treatments really have different levels of efficacy, or whether it is due to random chance), CIs describe how uncertain our estimated difference is. Simply, CI indicates whether the estimate is a precise one or only a very “rough” estimate. For example, if the objective of a study is to test whether there is a difference in systolic blood pressure between 2 groups of people, the p-value only tells if there is a statistically “significant” or statistically “non-significant” difference in the systolic blood pressure between the 2 groups. It does not provide an estimate of what this difference is. To get such an estimate, we need to compute the mean difference in the systolic blood pressure, along with the associated CI, which tells us how precise (i.e. how “good”) this estimate is in predicting the population level. A narrower CI will indicate a more precise estimate, while a wider CI indicates a less precise estimate. If the 95% CI for the DIFFERENCE between the 2 groups contains the value 0, this means that the p-value will be greater than 0.05. Conversely, if the 95% CI does not contain the value 0, then the p-value will be strictly less than 0.05. The same applies when comparing groups using a ratio, such as an odds ratio or risk ratio. When using a RATIO instead of a DIFFERENCE, the situation of no difference between the 2 groups will be indicated by a value of 1 instead of 0. If the ratio equals to 1, the 2 groups are equal. Hence, if the 95% CI of the ratio contains the value 1, the p-value will be greater than 0.05. Alternatively, if the 95% CI does not contain the value 1, the p-value is strictly less than 0.05.