LECTURE 8: power analysis

class: center, middle, inverse, title-slide

.title[
# LECTURE 8: power analysis
]
.subtitle[
## FANR 6750 (Experimental design)
]
.author[
### Fall 2022
]

---

# outline

1) Motivation

2) Type I and Type II error

3) Two-sample *t*-test

4) ANOVA

---
class: inverse

# motivation

> A statistical test will not be able to detect a true difference if the sample size is too small compared with the magnitude of the difference.

> Since data are sampled at random, there is always a risk of reaching a wrong conclusion, and things can go wrong in two ways - Dalgaard (2008)

---
# type i & type ii errors

#### Type I error (i.e., false positive)

> The null hypothesis is correct, but the test rejects it.

`$$\large \alpha = Pr(Type\;I\;error)$$`

#### Type II error (i.e., false negative)

> The null hypothesis is wrong, but the test fails to reject it.

`$$\large \beta = Pr(Type\;II\;error)$$`

#### Power

> The test's ability to reject a false null hypothesis.

`$$\large Power = 1 - \beta$$`
---
# type i & type ii errors

#### The type I error rate is set by the scientist

#### The type II error rate, and hence the power of the test, depends on many factors

#### In the context of a two-sample *t*-test, these are:

1) Magnitude of the difference ( `$\delta$` )

2) Standard deviation (or variance) of population ( `$\sigma$` )

3) The sample size ( `$n$` )

4) The Type I error rate ( `$\alpha$` )

---
# magnitude of the difference

---
# magnitude of the difference

---
# standard deviation

---
# standard deviation

---
# sample size

---
# sample size

---
# type i error rate

---
# type i error rate

---
# factors affecting power

#### In two-sample *t*-test, power increases when:

1) The difference in means increases

2) The standard deviation of the population decreases

3) The sample size increases

4) The Type I error rate increases

---
# example in `R`

```r
power.t.test(n = 5, 
             delta = 10, 
             sd = 5, 
             sig.level = 0.05, 
             power = NULL)
```

```
## 
##      Two-sample t test power calculation 
## 
##               n = 5
##           delta = 10
##              sd = 5
##       sig.level = 0.05
##           power = 0.7905
##     alternative = two.sided
## 
## NOTE: n is number in *each* group
```

---
## When should I do a power analysis?

#### Prospective is always better than retrospective!

**Retrospective** (conducted after experiment)

- If you failed to reject the null, then your power was low

- But you can't use this as an excuse!

- Only useful as a way of planning a subsequent experiment

**Prospective** (Done before the experiment)

- Used to determine sample size or power, given `$\delta$` and `$\sigma$`

- How can `$\delta$`  and/or `$\sigma$` be known ahead of time?

+ Requires prior knowledge, perhaps from a pilot study

+ Requires clear-headed thinking about what consitutes a biologically signiffcant difference

---
## What level of power should I aim for?

#### We want power to be as close to 1 as possible

#### Sometimes it may be prohibitively expensive to obtain a sample size large enough to achieve power close to 1

#### In practice, we are usually satisfied with power > 0.8

---
# one-way anova

#### To conduct a power analysis in the context of a one-way ANOVA, we need:

- The among group variance (MSa) instead of the difference in means, and

- The within group variance (MSe) instead of the standard deviation of the population

#### Power goes up when:

- MSa increases

- MSe decreases

- Same rules about `$\large n$` and `$\large \alpha$` from before also apply

---
# example in `R`

```r
power.anova.test(groups = 4, 
                 n = 5, 
                 between.var = 101, 
                 within.var = 20, 
                 sig.level = 0.05, 
                 power = NULL)
```

```
## 
##      Balanced one-way analysis of variance power calculation 
## 
##          groups = 4
##               n = 5
##     between.var = 101
##      within.var = 20
##       sig.level = 0.05
##           power = 1
## 
## NOTE: n is number in each group
```

---
# summary

- Power analysis let's you determine the necessary sample size (or power) for testing an effect size of interest

- Power is influenced by the magnitude of the effect, the standard deviation of the population, the Type I error rate, and the sample size

- Retrospective power analysis isn't useful unless you are planning a subsequent experiment
 
--

- `R` has several functions for conducting power analysis, but only for simple tests

- More complicated power analysis can be performed using simulation (not covered in this course)