Hypothesis Testing
-
Foundations for Hypothesis Testing
-
Steps for Data-Driven Decision Making
-
What is hypothesis?
- A hypothesis is an idea that can be tested
- It is a supposition of proposed explanation made on the basis of limited evidence as a starting point for further investigation
-
Examples of hypothesis
- Apples are expensive and they cost more than 50 rs
- Example of not an hypothesis
- Apples are expensive and they cost more than 50 rs
-
Null and Alternative Hypothesis
- Null Hypothesis:
- This is hypothesis to be tested
- It is status-quo. Everything which was believed until now that we are contesting with our test
- Alternative Hypothesis:
- This is change or innovation that is contesting the status quo
- If null is the status quo then the act of performing a test, shows we have doubts about the truthfulness of null. More often than not the researchers opinion is contained in alternative Hypothesis
- Null Hypothesis:
Examples
- Glassdoor publishes an article trying to state average data scientist salary is 113K USD
- Your friend tries to tell you that " I think data scientists make more than $125,000 "
- You want to check if your height is above average compared to your class mates. Try to find the null and alternative hypothesis for this test from the below options
- H0: My height is higher or equal to the average height in class H1: My height is lower than the average height of the class
- H0: My height is higher to the average height in class H1: My height is lower or equal than the average height of the class
- H0: My height is lower or equal to the average height in class H1: My height is higher than the average height of the class
- H0: My height is lower to the average height in class H1: My height is higher or equal than the average height of the class
- You want to test if the Obama administration issue fewer executive orders than bush administration. State the null and alternative hypothesis
- H0: The obama adminstration issued at least as many executive orders as the bush administration. H1: The obama adminsitration issued fewer executive orders than the bush administration
Decisions you can take
- When testing there are two decisions that can be made
- to accept the null hypothesis
- to reject the null hypothesis
- To accept the null means that there isn’t enough data to support the change or innovation brought by alternative
- To reject the null means that there is enough statistical evidence that status quo is not representative of truth
Rejection Region and a Significance Level
-
Significance level:
- This is represented by α
- The probability of rejecting a null hypothesis, if it is true
- Typical values for alpha are 0.01, 0.05, 0.1
- The most common alpha value is 0.05
-
You have called your placement officer and he has said that the average percentage marks of Engineering students is 70%
- H0: Population mean percentage is 70%
- H1: Population mean percentage is not 70%
-
You have called your placement officer and he has said that the average percentage marks of Engineering students is lower than 70%
- H0: Population mean percentage is <= 70%
- H1: Population mean percentage is > 70%