class: center, middle, inverse, title-slide .title[ # LECTURE 3: principles of stastical inference ] .subtitle[ ## FANR 6750 (Experimental design) ] .author[ ### <br/><br/><br/>Fall 2023 ] --- class: inverse # outline <br/> #### 1) Uncertainty in statistical models <br/> -- #### 2) Sampling distributions <br/> -- #### 3) Standard error <br/> -- #### 4) Confidence intervals <br/> --- class: inverse # statistics = information `\(\small +\)` uncertainty #### In the last lecture, we learned that models allow us to quantify relationships between variables based on **samples** -- #### We also learned that sampling = uncertainty - Parameter estimates from samples will never exactly equal population parameters -- #### Inference about populations requires quantifying the magnitude of this uncertainty -- #### But how can we measure how far our estimates are from the population parameters if we don't know the population parameters? --- # parameters vs statistics ### Parameters - Attributes of the population + Mean ( `\(\mu\)` ) + Variance ( `\(\sigma^2\)` ) + Standard deviation ( `\(\sigma\)` ) -- - Usually unknown -- - Parameters are the quantities of interest -- ### Statistics - Attributes of the sample + Mean ( `\(\bar{y}\)` or `\(\hat{\mu}\)` ) + Variance ( `\(s^2\)` or `\(\hat{\sigma}^2\)` ) + Standard deviation ( `\(s\)` or `\(\hat{\sigma}\)` ) -- - Often treated as estimates of parameters --- # sampling error #### Question: What is the probablity that `\(\large \bar{y} = \mu\)`? -- - Answer: 0 (why?) -- #### Fact: The sample mean will never equal the population mean -- - The difference between `\(\large \bar{y}\)` and `\(\large \mu\)` is **sampling error** -- - Sampling error can be reduced but it cannot be eliminated -- #### Problem: If we don't know `\(\large \mu\)`, how do we know how far our estimate is from the true value? -- - Answer: We don't (for any specific sample) -- - BUT...we do know how far, on average, a sample of size `\(n\)` will be from the true value --- class:inverse, center, middle # the sampling distribution --- # a single sample (n = 25) <br/> <img src="03_inference_files/figure-html/sampling-1.png" width="648" style="display: block; margin: auto;" /> --- # standard deviation <br/> <img src="03_inference_files/figure-html/sampling_sd-1.png" width="648" style="display: block; margin: auto;" /> -- **Remember** - this error bar is the standard deviation **of our sample**! --- # standard deviation But remember, what we really want to know is, how far is the sample mean from the true parameter value? -- <img src="03_inference_files/figure-html/sample_se-1.png" width="648" style="display: block; margin: auto;" /> --- # the sampling distribution Imagine we could repeat our experiment many, many times <img src="samples.gif" style="display: block; margin: auto;" /> --- # the sampling distribution The collection of sample means is referred to as the **sampling distribution** <img src="03_inference_files/figure-html/sampling_dist, -1.png" width="648" style="display: block; margin: auto;" /> --- # the sampling distribution The collection of sample means is referred to as the **sampling distribution** <img src="03_inference_files/figure-html/sampling_dist2, -1.png" width="648" style="display: block; margin: auto;" /> --- # the sampling distribution The standard deviation *of the sampling distribution* measures, on average, how far each sample mean from the true population value <img src="03_inference_files/figure-html/sampling_se, -1.png" width="648" style="display: block; margin: auto;" /> --- # the sampling distribution #### We rarely repeat experiments <br/> -- #### But we can estimate the standard deviation of the sampling distribution from a single sample! <br/> -- #### How? The **central limit theorem**! --- class:inverse # central limit theorem > For a population with mean `\(\mu\)` and standard deviation `\(\sigma\)`, the sampling distribution will be (approximately) normally distributed with mean `\(\mu_X = \mu\)` and standard deviation `\(\sigma_X = \sigma/\sqrt{n}\)`. <img src="03_inference_files/figure-html/crt-1.png" width="648" style="display: block; margin: auto;" /> --- class:inverse # central limit theorem > For a population with mean `\(\mu\)` and standard deviation `\(\sigma\)`, the sampling distribution will be (approximately) normally distributed with mean `\(\mu_X = \mu\)` and standard deviation `\(\sigma_X = \sigma/\sqrt{n}\)`. This might seem academic but it is hugely important - given sufficient sample size ( `\(\sim n > 30\)` ), we *know* the sampling distribution will be normally distributed without needing to repeat the experiment - although we never know the true population mean `\(\mu\)`, we can estimate how far (on average) our sample mean is likely to be away from it (that's what the standard deviation is!) - the CRT will hold true regardless of whether the source population is normal! - [Demonstration using simulations](http://www.ltcconline.net/greenl/java/Statistics/clt/cltsimulation.html) --- # standard error #### The standard deviation of the sampling distribution is called the **standard error of the mean** (or just the standard error) `$$\Large SE =\frac{s}{\sqrt{n}}$$` - Standard error tells us how far (on average) our sample mean is likely to be from the population mean - It is the key to estimating uncertainty in our estimates - Smaller is better --- class:inverse # descriptive vs inferential statistics > The sample standard deviation ( `\(s\)` ) is a descriptive statistic `$$\Large s = \sqrt{s^2}$$` - `\(\large s\)` tells us how far, on average, each observation `\(\large y\)` is from the sample mean `\(\large \bar{y}\)` <br/> -- > The standard error (SE) is an inferential statistic `$$\Large SE = \frac{s}{\sqrt{n}}$$` - `\(\large SE\)` tells us how far, on average, each sample mean `\(\large \bar{y}\)` is from the population mean `\(\large \mu\)` --- # quick review 1) sampling is a stochastic process - sample statistics (mean, standard deviation) will vary from sample to sample -- 2) the sampling distribution is an (imaginary) collection of statistics from repeated samples of the same population - assumes that the procedure for generating samples (e.g., sample size) is identical -- 3) Standard error is the standard deviation of the sampling distribution - `\(\large SE = s/\sqrt{n}\)` - measures how far, on average, sample statistics are from the true population value (smaller is better!) -- #### We will explore these concepts in lab, using `R` to generate and visualize repeated samples, calculate properties of the sampling distribution --- class:inverse,center,middle # confidence intervals --- # confidence intervals #### Uncertainty is commonly reported using *confidence intervals* - this is a concept that seems intuitive but in reality it is commonly misunderstood -- > If we calculated a `\(x\)`% confidence interval from repeated samples of the population, about `\(x\)`% of those confidence intervals would contain the true population mean -- <img src="03_inference_files/figure-html/ci-1.png" width="576" style="display: block; margin: auto;" /> --- # confidence intervals #### What a confidence interval is **NOT**: - "there is an `\(x\)`% probability that the true population parameter is inside this interval" -- #### The true population value is considered a *fixed* parameter -- #### Before we collect our sample, the ends of the confidence intervals are considered *random variables* (i.e., their value will change each time we collect a sample) - Like all random variables, we can describe the long-term (i.e., asymptotic) expectations of confidence intervals if we repeated the experiment many times -- #### After we collect our sample, the confidence interval either *does* or *does not* contain the true population value --- # confidence intervals .pull-left[ Think of confidence intervals like trying to determine where the stake is in a game of horseshoes by looking at where the horseshoes landed - The true parameter (i.e., the stake) does not move - Each horseshoe is the CI based on a single sample (i.e., one throw) - Before a horseshoe is thrown, there is some probability it will land around the stake - After it is thrown, it is either around the stake or not or... ] -- .pull-right[ <blockquote class="tiktok-embed" cite="https://www.tiktok.com/@dunk/video/7218281145279220997" data-video-id="7218281145279220997" data-embed-from="oembed" style="max-width: 605px;min-width: 325px;" > <section> <a target="_blank" title="@dunk" href="https://www.tiktok.com/@dunk?refer=embed">@dunk</a> <p>Tag someone you could beat in reverse basketball 😭 (@creationsross)</p> <a target="_blank" title="♬ original sound - Chizi" href="https://www.tiktok.com/music/original-sound-7212359167532075782?refer=embed">♬ original sound - Chizi</a> </section> </blockquote> <!--SHINY.SINGLETON[e123f97525dee37cc53ba5b9f17fd3caf1b72842]--> <script async data-external="1" src="https://www.tiktok.com/embed.js"></script> <style>blockquote.tiktok-embed {border:unset;padding:unset;}</style> <!--/SHINY.SINGLETON[e123f97525dee37cc53ba5b9f17fd3caf1b72842]--> ] --- # confidence intervals I like to think of confidence intervals providing a range of values that, based on our sample, are consistent with the population mean (*plausible interval*?) - If want to be more confident that the CI contains the parameter (e.g., 50% CI vs 95% CI), what happens to the size of the interval? -- It's worth remembering that 1 - `\(x\)`% of the time, the confidence interval we calculate from our sample **will not** include the true population mean. <img src="03_inference_files/figure-html/ci2-1.png" width="504" style="display: block; margin: auto;" /> Of course, with our real data, we have no way of knowing if our sample is one of the black points on this graph 😀 or one of the red dots 😢 --- # for thought #### If our goal is generally to decrease uncertainty in parameter estimates: -- - What factors determine the magnitude of our uncertainty estimates (SE or confidence intervals)? <br/> -- - What can we, as researchers, control when we design experiments to minimize uncertainty? What can we not control? --- # looking ahead <br/> ### **Next time**: Linear models, part 1: Categorical predictor with 2 levels <br/> ### **Reading**: [Fieberg chp. 3.6](https://statistics4ecologists-v1.netlify.app/matrixreg#categorical-predictors)