Descriptive Statistics
Theory
These are the solutions to the exercises contained within the handout to Descriptive Statistics which walks you through the basics of descriptive statistics and its parameters. The analyses presented here are using data from the StarWars data set supplied through the dplyr package that have been saved as a .csv file. Keep in mind that there is probably a myriad of other ways to reach the same conclusions as presented in these solutions.
I have prepared some Lecture Slides for this session.
Data
Find the data for this exercise here.
Packages
As you will remember from our lecture slides, the calculation of the mode in R can either be achieved through some intense coding or simply by using the mlv(..., method="mfv") function contained within the modeest package (unfortunately, this package is out of date and can sometimes be challenging to install).
Conclusively, it is now time for you to get familiar with how packages work in R. Packages are the way by which R is supplied with user-created and moderator-mediated functionality that exceeds the base applicability of R. Many things you will want to accomplish in more advanced statistics is impossible without such packages and even basic tasks such as data visualisation (dealt with in our next seminar) are reliant on R packages.
If you want to get a package and its functions into R there are two ways we will discuss in the following. In general, it pays to load all packages at the beginning of a coding document before any actual analyses happen (in the preamble) so you get a good overview of what the program is calling upon.
Basic Preamble
This is the most basic version of getting packages into R and is widely practised and taught. Unsurprisingly, I am not a big fan of it.
First, you use function install.packages() to download the desired package off dedicated servers (usually CRAN-mirrors) to your machine where it is then unpacked into a library (a folder that’s located in your documents section by default). Secondly, you need to invoke the library() function to load the R package you need into your active R session. In our case of the package modeest it would look something like this:
install.packages("modeest")
library(modeest)
The reason I am not overly fond of this procedure is that it is clunky, can break easily through spelling mistakes and starts cluttering your preamble super fast if the analyses you are wanting to perform require excessive amounts of packages. Additionally, when you are some place with a bad internet connection you might not want to re-download packages that are already contained on your hard drive.
Advanced Preamble
There is a myriad of different preamble styles (just as there are tons of different, personalised coding styles). I am left with presenting my preamble of choice here but I do not claim that this is the most sophisticated one out there.
The way this preamble works is that it is structured around a user-defined function (something we will touch on later in our seminar series) which first checks whether a package is already downloaded and then installs (if necessary) and/or loads it into R. This function is called install.load.package() and you can see its specification down below (don’t worry if it doesn’t make sense to you yet - it is not supposed to at this point). Unfortunately, it can only ever be applied to one package at a time and so we need a workaround to make it work on multiple packages at once. This can be achieved by establishing a vector of all desired package names (package_vec) and then applying (sapply()) the install.load.package() function to every item of the package name vector iteratively as follows:
# function to load packages and install them if they haven't been installed yet
install.load.package <- function(x) {
if (!require(x, character.only = TRUE))
install.packages(x)
require(x, character.only = TRUE)
}
# packages to load/install if necessary
package_vec <- c("modeest")
# applying function install.load.package to all packages specified in package_vec
sapply(package_vec, install.load.package)
## Loading required package: modeest
## modeest
## TRUE
Why do I prefer this? Firstly, it is way shorter than the basic method when dealing with many packages (which you will get into fast, I promise), reduces the chance for typos by 50% and does not override already installed packages hence speeding up your processing time.
Loading the Excel data into R
Our data is located in the Data folder and is called DescriptiveData.csv. Since it is a .csv file, we can simply use the R in-built function read.csv() to load the data by combining the former two identifiers into one long string with a backslash separating the two (the backslash indicates a step down in the folder hierarchy). Given this argument, read.csv() will produce an object of type data.frame in R which we want to keep in our environment and hence need to assign a name to. In our case, let that name be Data_df (I recommend using endings to your data object names that indicate their type for easier coding without constant type checking):
Data_df <- read.csv("Data/DescriptiveData.csv") # load data file from Data folder
What’s contained within our data?
Now that our data set is finally loaded into R, we can finally get to trying to make sense of it. Usually, this shouldn’t ever be something one has to do in R but should be manageable through a project-/data-specific README file (we will cover this in our seminar on hypotheses testing and project planning) but for now we are stuck with pure exploration of our data set. Get your goggles on and let’s dive in!
Firstly, it always pays to asses the basic attributes of any data object (remember the Introduction to R seminar):
- Name - we know the name (it is
Data_df) since we named it that - Type - we already know that it is a
data.framebecause we created it using theread.csvfunction - Mode - this is an interesting one as it means having to subset our data frame
- Dimensions - a crucial information about how many observations and variables are contained within our data set
Dimensions
Let’s start with the dimensions because these will tell us how many modes (these are object attribute modes and not descriptive parameter modes) to asses:
dim(Data_df)
## [1] 87 8
Using the dim() function, we arrive at the conclusion that our Data_df contains 87 rows and 8 columns. Since data frames are usually ordered as observations $\times$ variables, we can conclude that we have 87 observations and 8 variables at our hands.
You can arrive at the same point by using the function View() in R. I’m not showing this here because it does not translate well to paper and would make whoever chooses to print this waste paper.
Modes
Now it’s time to get a hang of the modes of the variable records within our data set. To do so, we have two choices, we can either subset the data frame by columns and apply the class() function to each column subset or simply apply the str() function to the data frame object. The reason str() may be favourable in this case is due to the fact that str() automatically breaks down the structure of R-internal objects and hence saves us the sub-setting:
str(Data_df)
## 'data.frame': 87 obs. of 8 variables:
## $ name : chr "Luke Skywalker" "C-3PO" "R2-D2" "Darth Vader" ...
## $ height : int 172 167 96 202 150 178 165 97 183 182 ...
## $ mass : num 77 75 32 136 49 120 75 32 84 77 ...
## $ hair_color: chr "blond" "" "" "none" ...
## $ skin_color: chr "fair" "gold" "white, blue" "white" ...
## $ eye_color : chr "blue" "yellow" "red" "yellow" ...
## $ birth_year: num 19 112 33 41.9 19 52 47 NA 24 57 ...
## $ gender : chr "male" "" "" "male" ...
As it turns out, our data frame knows the 8 variables of name, height, mass, hair_color, skin_color, eye_color, birth_year, gender which range from integer to numeric and factor modes.
Data Content
So what does our data actually tell us? Answering this question usually comes down to some analyses but for now we are only really interested in what kind of information our data frame is storing.
Again, this would be easiest to asses using a README file or the View() function in R. However, for the sake of brevity we can make due with the following to commands which present the user with the first and last five rows of any respective data frame:
head(Data_df)
## name height mass hair_color skin_color eye_color birth_year
## 1 Luke Skywalker 172 77 blond fair blue 19.0
## 2 C-3PO 167 75 gold yellow 112.0
## 3 R2-D2 96 32 white, blue red 33.0
## 4 Darth Vader 202 136 none white yellow 41.9
## 5 Leia Organa 150 49 brown light brown 19.0
## 6 Owen Lars 178 120 brown, grey light blue 52.0
## gender
## 1 male
## 2
## 3
## 4 male
## 5 female
## 6 male
tail(Data_df)
## name height mass hair_color skin_color eye_color birth_year gender
## 82 Finn NA NA black dark dark NA male
## 83 Rey NA NA brown light hazel NA female
## 84 Poe Dameron NA NA brown light brown NA male
## 85 BB8 NA NA none none black NA none
## 86 Captain Phasma NA NA unknown unknown unknown NA female
## 87 Padmé Amidala 165 45 brown light brown 46 female
The avid reader will surely have picked up on the fact that all the records in the name column of Data_df belong to characters from the Star Wars universe. In fact, this data set is a modified version of the StarWars data set supplied by the dplyr package and contains information of many of the important cast members of the Star Wars movie universe.
Parameters of descriptive statistics
Names
As it turns out (and should’ve been obvious from the onset if we’re honest), every major character in the cinematic Star Wars Universe has a unique name to themselves. Conclusively, the calculation of any parameters of descriptive statistics makes no sense with the names of our characters for the two following reasons:
- The name variable is of mode character/factor and only allows for the calculation of the mode
- Since every name only appears once, there is no mode
As long as the calculation of descriptive parameters of the name variable of our data set is concerned, Admiral Ackbar said it best: “It’s a trap”.
Height
Let’s get started on figuring out some parameters of descriptive statistics for the height variable of our Star Wars characters.
Subsetting
First, we need to extract the data in question from our big data frame object. This can be achieved by indexing through the column name as follows:
Height <- Data_df$height
Location Parameters
Now, with our Height vector being the numeric height records of the Star Wars characters in our data set, we are primed to calculate location parameters as follows:
mean <- mean(Height, na.rm = TRUE)
median <- median(Height, na.rm = TRUE)
mode <- mlv(na.omit(Height), method = "mfv")
min <- min(Height, na.rm = TRUE)
max <- max(Height, na.rm = TRUE)
range <- max - min
# Combining all location parameters into one vector for easier viewing
LocPars_vec <- c(mean, median, mode, min, max, range)
names(LocPars_vec) <- c("mean", "median", "mode", "minimum", "maximum", "range")
LocPars_vec
## mean median mode minimum maximum range
## 174.358 180.000 183.000 66.000 264.000 198.000
As you can clearly see, there is a big range in the heights of our respective Star Wars characters with mean and median being fairly disjunct due to the outliers in height on especially either end.
Distribution Parameters
Now that we are aware of the location parameters of the Star Wars height records, we can move on to the distribution parameters/parameters of spread. Those can be calculated in R as follows:
var <- var(Height, na.rm = TRUE)
sd <- sd(Height, na.rm = TRUE)
quantiles <- quantile(Height, na.rm = TRUE)
# Combining all location parameters into one vector for easier viewing
DisPars_vec <- c(var, sd, quantiles)
names(DisPars_vec) <- c("var", "sd", "0%", "25%", "50%", "75%", "100%")
DisPars_vec
## var sd 0% 25% 50% 75% 100%
## 1208.98272 34.77043 66.00000 167.00000 180.00000 191.00000 264.00000
Notice how some of the quantiles (actually quartiles in this case) link up with some of the parameters of central tendency.
Summary
Just to round this off, have a look at what the summary() function in R supplies you with:
summary <- summary(na.omit(Height))
summary
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 66.0 167.0 180.0 174.4 191.0 264.0
This is a nice assortment of location and dispersion parameters.
Mass
Now let’s focus on the weight of our Star Wars characters.
Subsetting
Again, we need to extract our data from the data frame. For the sake of brevity, I will refrain from showing you the rest of the analysis and only present its results to save some space.
Mass <- Data_df$mass
Location Parameters
## mean median mode minimum maximum range
## 97.31186 79.00000 80.00000 15.00000 1358.00000 1343.00000
As you can see, there is a huge range in weight records of Star Wars characters and especially the outlier on the upper end (1358kg) push the mean towards the upper end of the weight range and away from the median. We’ve got Jabba Desilijic Tiure to thank for that.
Distribution Parameters
## var sd 0% 25% 50% 75% 100%
## 28715.7300 169.4572 15.0000 55.6000 79.0000 84.5000 1358.0000
Quite obviously, the wide range of weight records also prompts a large variance and standard deviation.
Hair Color
Hair colour in our data set is saved in column 4 of our data set and so when sub-setting the data frame to obtain information about a characters hair colour, instead of calling on Data_df$hair_color we can also do so as follows:
HCs <- Data_df[, 4]
Of course, hair colour is not a numeric variable and much better represent by being of mode factor. Therefore, we are unable to obtain most parameters of descriptive statistics but we can show a frequency count as follows which allows for the calculation of the mode:
table(HCs)
## HCs
## auburn auburn, grey auburn, white black
## 5 1 1 1 13
## blond blonde brown brown, grey grey
## 3 1 18 1 1
## none unknown white
## 37 1 4
Eye Colour
Eye colour is another factor mode variable:
ECs <- Data_df$eye_color
We can only calculate the mode by looking for the maximum in our table() output:
table(ECs)
## ECs
## black blue blue-gray brown dark
## 10 19 1 21 1
## gold green, yellow hazel orange pink
## 1 1 3 8 1
## red red, blue unknown white yellow
## 5 1 3 1 11
Birth Year
Subsetting
As another numeric variable, birth year allows for the calculation of the full range of parameters of descriptive statistics:
BY <- Data_df$birth_year
Keep in mind that StarWars operates on a different time reference scale than we do.
Location Parameters
## mean median mode minimum maximum range
## 87.56512 52.00000 19.00000 8.00000 896.00000 888.00000
Again, there is a big disparity here between mean and median which stems from extreme outliers on both ends of the age spectrum (Yoda and Wicket Systri Warrick, respectively).
Distribution Parameters
## var sd 0% 25% 50% 75% 100%
## 23929.4414 154.6914 8.0000 35.0000 52.0000 72.0000 896.0000
Unsurprisingly, there is a big variance and standard deviation for the observed birth year/age records.
Gender
Gender is another factor mode variable (obviously):
Gender <- Data_df$gender
We can, again, only judge the mode of our data from the output of the table() function:
table(Gender)
## Gender
## female hermaphrodite male none
## 3 19 1 62 2