Hypothesis Testing

Reference: http://www.ltcconline.net/greenl/courses/201/hyptest/index.htm
http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html

A statistical hypothesis test, or hypothesis test, is an algorithm to state the alternative (for or against the hypothesis) which minimizes certain risks.

Procedures in Hypothesis Testing

When we test a hypothesis we proceed as follows:

Formulate the null and alternative hypothesis.
Choose a level of significance.
Determine the sample size. (Same as confidence intervals)
Collect data.
Calculate z (or t) score.
Utilize the table to determine if the z score falls within the acceptance region.
Decide to
- Reject the null hypothesis and therefore accept the alternative hypothesis or
- Fail to reject the null hypothesis and therefore state that there is not enough evidence to suggest the truth of the alternative hypothesis.

If the test statistic is outside the critical region, the only conclusion is that there is not enough evidence to reject the hypothesis. This is not the same as evidence in favor of the hypothesis. That we cannot obtain using these arguments, since lack of evidence against a hypothesis is not evidence for it. On this basis, statistical research progresses by eliminating error, not by finding the truth.

Setting up and testing hypotheses is an essential part of statistical inference. In order to formulate such a test, usually some theory has been put forward, either because it is believed to be true or because it is to be used as a basis for argument, but has not been proved, for example, claiming that a new drug is better than the current drug for treatment of the same symptoms.

In each problem considered, the question of interest is simplified into two competing claims / hypotheses between which we have a choice; the null hypothesis, denoted H0, against the alternative hypothesis, denoted H1. These two competing claims / hypotheses are not however treated on an equal basis, special consideration is given to the null hypothesis. We have two common situations:

1. The experiment has been carried out in an attempt to disprove or reject a particular hypothesis, the null hypothesis, thus we give that one priority so it cannot be rejected unless the evidence against it is sufficiently strong. For example, H0: there is no difference in taste between coke and diet coke against H1: there is a difference.

2. If one of the two hypotheses is 'simpler' we give it priority so that a more 'complicated' theory is not adopted unless there is sufficient evidence against the simpler one. For example, it is 'simpler' to claim that there is no difference in flavour between coke and diet coke than it is to say that there is a difference.

The hypotheses are often statements about population parameters like expected value and variance, for example H0 might be that the expected value of the height of ten year old boys in the Scottish population is not different from that of ten year old girls? A hypothesis might also be a statement about the distributional form of a characteristic of interest, for example that the height of ten year old boys is normally distributed within the Scottish population?

The outcome of a hypothesis test test is 'reject H0' or 'do not reject H0'.

Null Hypothesis

The null hypothesis or "Status Quo", H0 represents a theory that has been put forward, either because it is believed to be true or because it is to be used as a basis for argument, but has not been proved. For example, in a clinical trial of a new drug, the null hypothesis might be that the new drug is no better, on average, than the current drug. We would write H0: there is no difference between the two drugs on average.

*********************

We give special consideration to the null hypothesis. This is due to the fact that the null hypothesis relates to the statement being tested, whereas the alternative hypothesis relates to the statement to be accepted if / when the null is rejected.

The final conclusion once the test has been carried out is always given in terms of the null hypothesis. We either 'reject H0 in favour of H1' or 'do not reject H0'; we never conclude 'reject H1', or even 'accept H1'.

If we conclude 'do not reject H0', this does not necessarily mean that the null hypothesis is true, it only suggests that there is not sufficient evidence against H0 in favour of H1; rejecting the null hypothesis then, suggests that the alternative hypothesis may be true.

Alternative Hypothesis

The alternative (researched) hypothesis, H1, is a statement of what a statistical hypothesis test is set up to establish. For example, in a clinical trial of a new drug, the alternative hypothesis might be that the new drug has a different effect, on average, compared to that of the current drug. We would write H1: the two drugs have different effects, on average. The alternative hypothesis might also be that the new drug is better, on average, than the current drug. In this case we would write H1: the new drug is better than the current drug, on average.

*********************

Errors in Hypothesis Tests

We define a type I error as the event of rejecting the null hypothesis when the null hypothesis was true. The probability of a type I error (a) is called the significance level.

We define a type II error (with probability b) as the event of failing to reject the null hypothesis when the null hypothesis was false.

Example

Suppose that you are a lawyer that is trying to establish that a company has been unfair to minorities with regard to salary increases. Suppose the mean salary increase per year is 8%.

You set the null hypothesis to be

H0: m = .08

H1: m < .08

Q. What is a type I error?

A. We put sanctions on the company, when they were not being discriminatory.

Q. What is a type II error?

A. We allow the company to go about its discriminatory ways.

Note: Larger a results in a smaller b, and smaller a results in a larger b.

Definition of Hypothesis

Status quo