---
title: 'Formative Exercise 05: MSc Data Skills Course'
author: "Psychology, University of Glasgow"
output: html_document
---

```{r setup, include=FALSE}
# please do not alter this code chunk
knitr::opts_chunk$set(echo = TRUE, 
                      message = FALSE, 
                      error = TRUE)
library(tidyverse)
library(ukbabynames)

# install the class package dataskills to access built-in data
# devtools::install_github("psyteachr/msc-data-skills)
# or download data from the website
# https://psyteachr.github.io/msc-data-skills/data/data.zip
```

Edit the code chunks below and knit the document. You can pipe your objects to `glimpse()` or `print()` to display them.

## UK Baby Names

Here we will convert the data table from the ukbabynames package to a tibble and assign it the variable name `ukb`. Use this data tibble for questions 1-13.

```{r ukb}
# do not alter this code chunk
ukb <- as_tibble(ukbabynames) # convert to a tibble
```


### Question 1

How many records are in the dataset?

```{r Q1}
nrecords <- nrow(ukb)

## or: 

nrecords <- count(ukb) %>% pull(n) %>% print()
```

### Question 2

Remove the column `rank` from the dataset.

```{r Q2}
norank <- ukb %>%
  select(-rank) %>%
  glimpse()
```

### Question 3

What is the range of birth years contained in the dataset? Use `summarise` to make a table with two columns: `minyear` and `maxyear`.

```{r Q3}
birth_range <- ukb %>%
  summarise(minyear = min(year),
            maxyear = max(year)) %>%
  print()
```

### Question 4

Make a table of only the data from babies named Hermione.

```{r Q4}
hermiones <- ukb %>%
  filter(name == "Hermione") %>%
  print()
```

### Question 5

Sort the dataset by sex and then by year (descending) and then by rank (descending).

```{r Q5}
sorted_babies <- ukb %>%
  arrange(sex, desc(year), desc(rank)) %>%
  glimpse()
```

### Question 6

Create a new column, `decade`, that contains the decade of birth (1990, 2000, 2010).  Hint: see `?floor`

```{r Q6}
ukb_decade <- ukb %>%
  mutate(decade = floor(year / 10) * 10) %>%
  glimpse()
```

### Question 7

Make a table of only the data from male babies named Courtney that were born between 1998 and 2001 (inclusive).

```{r Q7}
courtney <- ukb %>%
  filter(name == "Courtney", sex == "M",
         year >= 1998, year <= 2001) %>%
  print()
```


### Question 8

How many distinct names are represented in the dataset? Make sure `distinct_names` is an integer, not a data table.

```{r Q8}
distinct_names <- n_distinct(ukb$name)

# or

distinct_names <- ukb %>%
  distinct(name) %>%
  count() %>%
  pull() %>%
  print()
```

### Question 9

Make a table of only the data from the female babies named Frankie that were born before 1999 or after 2010.

```{r Q9}
frankie <- ukb %>%
  filter(name == "Frankie", 
         sex == "F",
         (year < 1999) | (year > 2010)) %>%
  print()
```

### Question 10

How many total babies in the dataset were named 'Emily'? Make sure `emily` is an integer, not a data table.

```{r Q10}
emily <- ukb %>%
  filter(name == "Emily") %>%
  summarise(tot = sum(n)) %>%
  pull(tot) %>%
  print()
```

### Question 11

How many distinct names are there for each sex?

```{r Q11}
names_per_sex <- ukb %>% 
  group_by(sex) %>%
  distinct(name) %>%
  count %>%
  print()
```

### Question 12

What is the most popular name in the dataset?

```{r Q12}
most_popular <- ukb %>%
  group_by(name) %>%
  summarise(tot = sum(n), .groups = "drop") %>%
  arrange(desc(tot)) %>%
  slice(1) %>%
  pull(name)

## alternatively, this will give you all the top names if there are ties
most_popular <- ukb %>%
  group_by(name) %>%
  summarise(tot = sum(n), .groups = "drop") %>%
  filter(rank(tot) == max(rank(tot))) %>%
  pull(name) %>%
  print()

```

### Question 13

How many babies were born each year for each sex?  Make a plot.

```{r Q13}
babies_per_year <- ukb %>%
  group_by(year, sex) %>%
  summarise(tot = sum(n), .groups = "drop")

ggplot(babies_per_year, aes(year, tot, color = sex)) +
  geom_line()
```

## Select helpers

Load the dataset [dataskills::personality](https://psyteachr.github.io/msc-data-skills/data/personality.csv).

Select only the personality question columns (not the user_id or date).

```{r SH1}
q_only <- dataskills::personality %>%
  select(-user_id, -date) %>%
  glimpse()
```

Select the `user_id` column and all of the columns with questions about openness.

```{r SH2}
openness <- dataskills::personality %>%
  select(user_id, starts_with("Op")) %>%
  glimpse()
```

Select the `user_id` column and all of the columns with the first question for each personality trait.

```{r SH3}
q1 <- dataskills::personality %>%
  select(user_id, ends_with("1")) %>%
  glimpse()
```


## Window fuctions

The code below sets up a fake dataset where 10 subjects respond to 20 trials with a `dv` on a 5-point Likert scale. 

```{r window-setup}
set.seed(10)

fake_data <- tibble(
  subj_id = rep(1:10, each = 20),
  trial = rep(1:20, times = 10),
  dv = sample.int(5, 10*20, TRUE)
)
```

### Question 14

You want to know how many times each subject responded with the same dv as their last trial. For example, if someone responded 2,3,3,3,4 for five trials they would have repeated their last response on the third and fourth trials. Use an offset function to determine how many times each subject repeated a response.

```{r window}
repeated_data <- fake_data %>%
  group_by(subj_id) %>%
  mutate(repeated = dv == lag(dv)) %>%
  summarise(repeats = sum(repeated, na.rm = TRUE),
            .groups = "drop") %>%
  print()
```

### Question 15

Create a table `too_many_repeats` with the top two repeaters from `repeated_data` (and anyone who might be tied with them) using ranking functions. 

```{r}
too_many_repeats <- repeated_data %>%
  mutate(rank = dense_rank(repeats)) %>%
  filter(rank == max(rank) | rank == max(rank)-1) %>%
  print()
```


## Advanced Questions

There are several ways to complete the following two tasks. Different people will solve them different ways, but you should be able to tell if your answers make sense.

### Question 16

Load the dataset [dataskills::family_composition](https://psyteachr.github.io/msc-data-skills/data/family_composition.csv) from last week's exercise.

Calculate how many siblings of each sex each person has, narrow the dataset down to people with fewer than 6 siblings, and generate at least two different ways to graph this.

```{r Q16a}

family <- dataskills::family_composition %>%
  gather("sibtype", "n", oldbro:twinsis) %>%
  separate(sibtype, c("sibage", "sibsex"), sep = -3)

sib6 <- family %>%
  group_by(user_id, sex, sibsex) %>%
  summarise(n = sum(n), .groups = "drop") %>%
  group_by(user_id) %>%
  filter(sex %in% c("male", "female"), sum(n) < 6)

ggplot(sib6, aes(n, fill = sibsex)) +
  geom_histogram(binwidth = 1, colour = "black", position = "dodge")

```
    
```{r Q16b}
sib6 <- family %>%
  group_by(user_id, sex, sibsex) %>%
  summarise(n = sum(n), .groups = "drop") %>%
  filter(sex %in% c("male", "female")) %>%
  spread(sibsex, n) %>%
  filter(bro + sis < 6)

ggplot(sib6, aes(bro, sis)) +
  geom_count()
```

```{r Q16c}
# or
ggplot(sib6, aes(bro, sis)) +
  geom_bin2d(binwidth = c(1,1)) +
  stat_bin2d(geom = "text", aes(label = ..count..), 
             binwidth = c(1, 1), color = "white")
```   


### Question 17

Use the dataset [dataskills::eye_descriptions](https://psyteachr.github.io/msc-data-skills/data/eye_descriptions.csv) from last week's exercise.

Create a list of the 10 most common descriptions from the eyes dataset. Remove useless descriptions and merge redundant descriptions.
    
```{r Q17}
eyes <- dataskills::eye_descriptions %>%
  gather("face_id", "description", t1:t50) %>%
  separate(description, c("d1", "d2", "d3", "d4"), sep = "(,|;|\\/)+", extra = "merge", fill = "right") %>%
  gather("desc_n", "description", d1:d4) %>%
  filter(!is.na(description)) %>%          # gets rid of rows with no description
  mutate(
    description = trimws(description),     # get rid of white space around string
    description = tolower(description)     # make all characters lowercase
  ) %>%
  group_by(description) %>%
  summarise(n = n(), .groups = "drop") %>% # count occurrences of each description
  arrange(desc(n)) %>%                     # sort by count (descending)
  filter(nchar(description) > 1) %>%       # get rid of 1-character descriptions
  filter(row_number() < 11)
```

```{r Q17-print, results='asis'}
# displays the table in a nice format
knitr::kable(eyes) 
```

