• Definition: 
  • A goal of a researcher conducting a study is to get a significant result (rejecting Ho, and accepting H1) - but only if the   
    research hypothesis is really is true.
     
  • The statistical power (1-b ) of a study is the probability that the study will have a significant result if the research hypothesis is true.  
     
  • In other words, it is the probability of rejecting false null hypothesis.

If the research hypothesis is true, won't the study automatically give a significant result?  No, because of sampling error, small sample, or violation of internal validity

• What determines the power of a study?

 

Feature of the study	Increases power	Decreases power
Effect size	Large	Small
Effect size combines the following two features 1. Hypothesized difference between pop. Means 2. Population standard deviation	Large difference   Small SD	Small difference   Large SD
Sample size (n)	Large n	Small n
Significance level	High alpha	Low alpha
One-tail vs. two-tail	One-tail	Two-tail

• Role of power when planning a study
  
  If you do a study in which the power is low, even if the research hypothesis is true, this study will probably not give statistically significant results.  When the power of a planned study is found to be low, researchers look for practical ways to increase the power to an acceptable level.  
  
  Rule of thumb of acceptable power -- .8 (80%) Cohen (1988)
  
   
• Summary of Practical Ways of Increasing the Power of a Planned Experiment.
   

Feature of the study	Practical Way of Raising Power	Disadvantage
Predicted difference between population means	Increase the intensity of experimental procedure	May not be practical or distort study's meaning
Standard deviation	Use a less diverse population	May not be available; decreases generalizability
Sample size	Use a larger sample size	Easy way to increase power, but not always practical; can be costly
Significance level	Use a more lenient level of significance	Raise Type I error
One-tail vs. two-tail	Use a one-tail test	May not be appropriate to the logic of the study

Source: Aron & Aron (1999)

Power calculation for mean differences:  one sample and two sample tests (see pp. 239-240)

   Calculating Power The following factors should be known.

            a) Significance criterion (α level)

            b) Effect size (d)  or means and pooled SDs
                        small: d = .20
                        medium: d = .50
                        large: d = .80

            c) Sample size
            d) Direction: one-tailed or two-tailed. 

   Determining appropriate sample size:  Following factors should be known.

         a) Significance criterion ( level)

            b) Effect size (d)  or means and pooled SDs
                 small: d = .20
                        medium: d = .50
                        large: d = .80

            c) Power (maximum power you would like to achieve for your study; usually power of .80 is used)
            d) Direction: one-tailed or two-tailed. 
 
pp. 239-240

8.1: a)                   c)

8.3

8.7

8.15

Power calculation for Pearson r

    Calculating Power The following factors should be known.

            a) Significance criterion (α level)

            b) Effect size (ES): r
                        small: r = .10
                        medium: r = .30
                        large: r = .50

            c) Sample size
            d) Direction: one-tailed or two-tailed. 
 

   Determining appropriate sample size:  Following factors should be known.

         a) Significance criterion ( level)

            b) Effect size (ES): r
                        small: r = .10
                        medium: r = .30
                        large: r = .50

            c) Power (maximum power you would like to achieve for your study; usually power of .80 is used)
            d) Direction: one-tailed or two-tailed. 
 

Cohen (1988), p. 96

A personality psychologist has performed an experiment in which he obtained paired measures (X = extraversion test score; Y = neurophysiological measure score) on a sample of 50 subjects.  Although his theory dictates a strong relationship, unreliability and lack of high construct validity of his measures (social desirability) lead him to expect only a medium ES (r = .30).  What is the power of the test of the significance of r he performs?  He use α = .05, with one-tailed test.   In other words, what are your chances of finding a significant sample correlation? 

How many sample size would he need for power = .80?