Announcements

• Check presentation schedule and let me know if there are any concerns.
• We'll start presentations at 12pm (not 12:10pm)!

• Statistical inference is the process of using sample data to make conclusions about the underlying population the sample came from.
• Similar to tasting a spoonful of soup while cooking to make an inference about the entire pot.

Estimation

So far we have done lots of estimation (mean, median, slope, etc.), i.e.

• used data from samples to calculate sample statistics
• which can then be used as estimates for population parameters

• If we report a point estimate, we probably won’t hit the exact population parameter.
• If we report a range of plausible values we have a good shot at capturing the parameter.

Confidence intervals

A plausible range of values for the population parameter is a confidence interval.

• In order to construct a confidence interval we need to quantify the variability of our sample statistic.
• For example, if we want to construct a confidence interval for a population mean, we need to come up with a plausible range of values around our observsed sample mean.
• This range will depend on how precise and how accurate our sample mean is as an estimate of the population mean.
• Quantifying this requires a measurement of how much we would expect the sample mean to vary from sample to sample.

Quantifying the variability of a sample statistic

We can quantify the variability of sample statistics using

• simulation: via bootstrapping (today)

or

• theory: via Central Limit Theorem (in Stat 2!)

Bootstrapping

• The term bootstrapping comes from the phrase "pulling oneself up by one’s bootstraps", which is a metaphor for accomplishing an impossible task without any outside help.
• In this case the impossible task is estimating a population parameter, and we’ll accomplish it using data from only the given sample.
• Note that this notion of saying something about a population parameter using only information from an observed sample is the crux of statistical inference, it is not limited to bootstrapping.

Rent in Edinburgh

Sample

Fifteen 3 BR flats in Edinburgh were randomly selected on rightmove.co.uk.

library(tidyverse)
edi_3br <- read_csv2("data/edi-3br.csv") # ; separated

## # A tibble: 15 x 4
##    flat_id  rent title                      address                        
##    <chr>   <dbl> <chr>                      <chr>                          
##  1 flat_01   825 3 bedroom apartment to re… Burnhead Grove, Edinburgh, Mid…
##  2 flat_02  2400 3 bedroom flat to rent     Simpson Loan, Quartermile, Edi…
##  3 flat_03  1900 3 bedroom flat to rent     FETTES ROW, NEW TOWN, EH3 6SE  
##  4 flat_04  1500 3 bedroom apartment to re… Eyre Crescent, Edinburgh, Midl…
##  5 flat_05  3250 3 bedroom flat to rent     Walker Street, Edinburgh       
##  6 flat_06  2145 3 bedroom flat to rent     George Street, City Centre, Ed…
##  7 flat_07  1500 3 bedroom flat to rent     Waverley Place , Edinburgh EH7…
##  8 flat_08  1950 3 bedroom flat to rent     Drumsheugh Place, Edinburgh    
##  9 flat_09  1725 3 bedroom flat to rent     Crighton Place, Leith, Edinbur…
## 10 flat_10  2995 3 bedroom flat to rent     Simpson Loan, Meadows, Edinbur…
## 11 flat_11  1400 3 bedroom flat to rent     42, Learmonth Court, Edinburgh…
## 12 flat_12  1995 3 bedroom apartment to re… Chester Street, Edinburgh, Mid…
## 13 flat_13  1250 3 bedroom duplex to rent   Elmwood Terrace, Lochend, Edin…
## 14 flat_14  1995 3 bedroom apartment to re… Great King Street, Edinburgh, …
## 15 flat_15  1600 3 bedroom ground floor fl… Roseneath Terrace,Edinburgh,EH9

Observed sample

Observed sample

Sample mean ≈ £1895 😱

Bootstrap population

Generated assuming there are more flats like the ones in the observed sample... Population mean = ❓

Bootstrapping scheme

1. Take a bootstrap sample - a random sample taken with replacement from the original sample, of the same size as the original sample.
2. Calculate the bootstrap statistic - a statistic such as mean, median, proportion, slope, etc. computed on the bootstrap samples.
3. Repeat steps (1) and (2) many times to create a bootstrap distribution - a distribution of bootstrap statistics.
4. Calculate the bounds of the XX% confidence interval as the middle XX% of the bootstrap distribution.

Let's bootstrap

Two ways

1. Using for loops
2. Using the infer package

Bootstrapping with for loops

set.seed(9014)
boot_df <- tibble(
  replicate = 1:100,
  stat = rep(NA, 100)
  )
for (i in 1:100){
   boot_df$stat[i] <- edi_3br %>% 
     sample_n(15, replace = TRUE) %>% 
     summarise(stat = mean(rent)) %>% 
     pull()
}

Bootstrap results

ggplot(boot_df, aes(x = stat)) +
  geom_histogram(binwidth = 100)

boot_df %>%
  summarise(
    lower = quantile(stat, 0.025),
    upper = quantile(stat, 0.975),
    )

## # A tibble: 1 x 2
##   lower upper
##   <dbl> <dbl>
## 1 1594. 2292.

infer $\in$ tidymodels

library(infer)

Generate bootstrap means

edi_3br %>%
  # specify the variable of interest
  specify(response = rent)

Generate bootstrap means

edi_3br %>%
  # specify the variable of interest
  specify(response = rent)
  # generate 15000 bootstrap samples
  generate(reps = 15000, type = "bootstrap")

Generate bootstrap means

edi_3br %>%
  # specify the variable of interest
  specify(response = rent)
  # generate 15000 bootstrap samples
  generate(reps = 15000, type = "bootstrap")
  # calculate the mean of each bootstrap sample
  calculate(stat = "mean")

Generate bootstrap means

# save resulting bootstrap distribution
boot_df <- edi_3br %>%
  # specify the variable of interest
  specify(response = rent) %>% 
  # generate 15000 bootstrap samples
  generate(reps = 15000, type = "bootstrap") %>% 
  # calculate the mean of each bootstrap sample
  calculate(stat = "mean")

The bootstrap sample

boot_df

## # A tibble: 15,000 x 2
##    replicate  stat
##        <int> <dbl>
##  1         1 1793.
##  2         2 1938.
##  3         3 2175 
##  4         4 2159.
##  5         5 2084 
##  6         6 1761 
##  7         7 1787.
##  8         8 1817.
##  9         9 1963.
## 10        10 1800.
## # … with 14,990 more rows

Visualize the bootstrap distribution

ggplot(data = boot_df, mapping = aes(x = stat)) +
  geom_histogram(binwidth = 100) +
  labs(title = "Bootstrap distribution of means")

Calculate the confidence interval

A 95% confidence interval is bounded by the middle 95% of the bootstrap distribution.

boot_df %>%
  summarize(lower = quantile(stat, 0.025),
            upper = quantile(stat, 0.975))

## # A tibble: 1 x 2
##   lower upper
##   <dbl> <dbl>
## 1 1603. 2213.

Visualize the confidence interval

Interpret the confidence interval

Confidence level

We are 95% confident that ...

• Suppose we took many samples from the original population and built a 95% confidence interval based on each sample.
• Then about 95% of those intervals would contain the true population parameter.

Commonly used confidence levels

Precision vs. accuracy

Changing confidence level

edi_3br %>%
  specify(response = rent) %>% 
  generate(reps = 15000, type = "bootstrap") %>% 
  calculate(stat = "mean") %>%
  summarize(lower = quantile(stat, 0.025),
            upper = quantile(stat, 0.975))

Recap

• Sample statistic $\ne$ population parameter, but if the sample is good, it can be a good estimate.
• We report the estimate with a confidence interval, and the width of this interval depends on the variability of sample statistics from different samples from the population.
• Since we can't continue sampling from the population, we bootstrap from the one sample we have to estimate sampling variability.
• We can do this for any sample statistic:
  • For a mean: calculate(stat = "mean")
  • For a median: calculate(stat = "median")
  • Learn about calculating bootstrap intervals for other statistics in your homework.