Case Study
Null Hypothesis
In the Physicians' Reactions study, the researchers hypothesized that physicians would expect to spend less time with obese patients. The null hypothes is that the two types of patients are treated identically is put forward with the hope that it can be discredited and therefore rejected. So the null hypotheis is
H0: μobese = μaverage
Probability Value
In the physician reaction study, we compute the probability of getting a difference as large or larger than the observed difference (31.4 - 24.7 = 6.7 minutes) if the difference were, in fact, due solely to chance. This probability can be computed to be 0.0057. Since this is such a low probability, we have confidence that the difference in times is due to the patient's weight and is not due to chance.
Significance Testing
The probability value below which the null hypothesis is rejected is called the α level or simply α. It is also called the significance level. When the null hypothesis is rejected, the effect is said to be statistically significant. It is very important to keep in mind that statistical significance means only that the null hypothesis of exactly no effect is rejected; it does not mean that the effect is important. Do not confuse statistical significance with practical significance.
Two ways of significance tests
- A significance test is conducted and the probability value reflects the strength of the evidence against the null hypothesis. Higher probabilities provide less evidence that the null hypothesis is false. (For scientific research)
Probability | Meaning |
p<0.01 | The data provide strong evidence that the null hypothesis is false. |
0.01<p<0.05 | The null hypothesis is typically rejected, but not with as much confidence as it would be if the probability value were below 0.01. |
0.05<p<0.1 | The data provide weak evidence against the null hypothesis and are not considered low enough to justify rejecting it. |
- Specify an α level before analyzing the data. If the data analysis results in a probability value below the α level, then the null hypothesis is rejected; if it is not, then the null hypothesis is not rejected. If a result is significant, then it does not matter how significant it is.
If it is not significant, then it does not matter how close to being significant it is.
Type I and II Errors
Type I error (弃真错误) occurs when a significance test results in the rejection of a true null hypothesis. α is the probability of a Type I error given that the null hypothesis is true.
Type II error (弃伪错误) is failing to reject a false null hypothesis. If the null hypothesis is false, then the probability of a Type II error is called β (beta). The probability of correctly rejecting a false null hypothesis equals 1- β and is called power. Actually, a Type II error is not really an error. When a statistical test is not significant, it means that the data do not provide strong evidence that the null hypothesis is false. Lack of significance does not support the conclusion that the null hypothesis is true. One way to decrease the value of β is to increase the volume of samples. With the constance volume of samples, β will increase with smaller value of α. In practice, we should perform a trade of between α and β.