2  Missing Values, Subsetting Vectors, Logic, and One- and Two-sample t-Tests

2.1 Missing Values, Subsetting Vectors and Logic

2.1.1 Missing Values

In R missing values are represented by the symbol NA (a.k.a. Not Assigned or Not Available). Undefined mathematical operations (e.g., divisions by zero) are represented by the symbol NaN (a.k.a. Not a Number). R uses the same symbol for data of class character and data of class numeric.

2.1.1.1 Testing for missing values

The is.na() function is used to test for NA values in an object and returns a logical vector with the same length as the tested object, where TRUE indicates NA and FALSE indicates non-NA values. To test for NaN values in an object, the is.nan() function can be used in the same way as described for is.na().

NA values also have a class, so there are integer NA , character NA, etc. A NaN value is also NA, but the opposite is not true (not every NA is a NaN).

2.1.1.2 Excluding missing values from analyses

Many functions in R have an argument, which allows for exclusion of missing values. An example is mean(). Try the following commands yourself:

x <- c(1, 2, NA, 10, 3)
mean(x)

#> [1] NA

mean(x, na.rm = TRUE)

#> [1] 4

There are many other functions to exclude missing values from your analyses, some of them you will come across in this course, for example by using is.na() and is.nan() introduced in Section 2.1.1.1.

2.1.2 Subsetting Vectors

Subsetting is important and often used in statistics and data analysis. Not always analyses are run over the full set. Sometimes the data sets are split into training and validation sets, where the training set is used to build your statistical model. Subsetting of a vector is achieved by placing an index vector between square brackets after the object name of the vector to be subsetted, e.g. x[1:2] for the first two positions of x. Within R four types of index vectors can be used:

• logical vectors
• vectors of positive integers
• vectors of negative integers
• vectors of character strings

In the following example the first four elements of a character vector are taken:

x <- c("a", "b", "c", "c", "d", "a")
class(x)

#> [1] "character"

x[1:4]

#> [1] "a" "b" "c" "c"

This example uses a vector of positive integers to subset the given character vector.

In SWIRL Lesson 6: Subsetting Vectors, which is one of the exercises, you will learn about all possible index vectors you can use for subsetting a vector.

2.1.3 Logic

Logical vectors can be used for subsetting vectors. The two logical values in R, also called Boolean values, are TRUE and FALSE (write them with all capitalized letters in R!). Apart from subsetting with logical vectors, relational and logical operators can be used in R to construct logical expressions, which will evaluate to either TRUE or FALSE. These are useful when building functions later on in this course to evaluate certain conditions.

The first relational operator is the equality operator, represented by two equals signs ==. For example:

TRUE == TRUE

should evaluate to TRUE.

Just like arithmetic, parentheses can be used to set the priority of evaluation, e.g.

(FALSE == TRUE) == FALSE

What do you think the outcome of this logical expression will be? Execute it to check your answer.

The equality operator can be used to check two objects of the same class for equality, where it does not matter which class as long as they are the same. The following example should evaluate to FALSE:

6 == 7

Thankfully relational operators exist, that allow testing for a value less than or greater than another value as well. The less than operator < tests, whether the number on the left-hand side of the operator (called the left operand) is less than the number on the right-hand side of the operator (the right operand). There is also a less-than-or-equal-to operator <=. Equivalently there is a greater than > and a greater-than-or-equal-to >= operator.

The inequality operator is represented by !=, which can be read as ‘not’ displayed by ! and ‘equal’ represented by =. The exclamation point in R is, what is called the negation operator, basically meaning NOT. In order to negate logical expressions you can use the NOT operator. An exclamation point ! will cause !TRUE (say: not true) to evaluate to FALSE and !FALSE (say: not false) to evaluate to TRUE.

At some point the need will arise to examine relationships between multiple expressions. This is where the AND operator, and the OR operator come into play. First let us scrutinize the operation of AND. There are two AND operators in R, & and &&. Both operators work similarly, if the right and left operands of AND are both TRUE the entire expression evaluates to TRUE, otherwise it is FALSE. The & operator can be used to evaluate AND across a vector. The && version of AND only evaluates single logical outcomes on the left, and right hand side of the operator with each other. A few examples:

FALSE & FALSE
TRUE & c(TRUE, FALSE, FALSE)
# and
TRUE && c(TRUE, FALSE, FALSE) # Try also TRUE && TRUE

In order to truly get what is happening, try to figure out the answer yourself before executing these commands in the console.

A similar set of rules is followed by the OR operator. The | version of OR evaluates across an entire vector, while the || version of OR only evaluates the first member of a vector. An expression using the OR operator will evaluate to TRUE if the left operand or the right operand is TRUE. If both are TRUE, the expression will result TRUE, however if neither are TRUE (or equivalently both are FALSE), then the expression will be FALSE. Two examples:

TRUE | c(TRUE, FALSE, FALSE)
# and
TRUE || c(TRUE, FALSE, FALSE)

Again try to figure out the outcome, before trying it in the console.

The power of logical operators lies in the fact that they can be chained together just like arithmetic operators. The expressions:

6 != 10 && FALSE && 1 >= 2
# or
TRUE || 5 < 9.3 || FALSE

are perfectly normal in R. Just keep in mind that logical operators have an order of operation, just as arithmetic operators.

ImportantImportant: Evaluation order of AND and OR operators

All AND operators are evaluated before OR operators.

Therefore, use parentheses to set precedence in the evaluation order.

Try to figure out the following example:

5 > 8 || 6 != 8 && 4 > 3.9

This expression will evaluate to TRUE. First the left and right operands of the AND operator are evaluated. 6 is not equal 8, 4 is greater than 3.9, therefore both operands are TRUE so the resulting expression TRUE && TRUE evaluates to TRUE. Then the left operand of the OR operator is evaluated: 5 is not greater than 8 so the entire expression is reduced to FALSE || TRUE. Since the right operand of this expression is TRUE the entire expression evaluates to TRUE.

2.1.3.1 R functions for dealing with logical expressions

Having familiarized with R’s logical operators a few functions will be introduced for dealing with logical expressions.

The function isTRUE() takes one argument. If that argument is TRUE, the function will return TRUE, otherwise it returns FALSE.

The function identical() will return TRUE if the two R objects passed to its arguments are identical. For example:

identical("twins", "twins")

Another quite useful function is the xor(), which takes two arguments. The xor() stands for exclusive OR. This function only evaluates to TRUE, if and only if just one of the two arguments is TRUE. When both arguments are TRUE the function evaluates to FALSE, hence the exclusivity.

The which() function takes a logical vector as an argument and returns the indices of the vector that are TRUE. For example:

which(c(TRUE, FALSE, TRUE))

would return the vector c(1, 3). Like the which() function, the functions any() and all() take logical vectors as their argument. The any() function will return TRUE, when one or more of the elements in the logical vector is or are TRUE. The all() function will return TRUE, when every element in the logical vector is TRUE.

2.1.4 Exercise SWIRL: Missing Values

Start swirl() from the package swirl and work through lesson 5 about Missing Values in R. If you do not remember how to start up SWIRL, have a glimpse at Section 1.6.3.

2.1.5 Exercise SWIRL: Subsetting Vectors

Start up SWIRL and work through lesson 6, where you will learn more about Subsetting Vectors as announced in Section 2.1.2.

2.2 One- and Two-sample t-Tests in R

For theory see O&L Sections 6.2, 6.4 (in both \(6^{\mbox{th}}\) and \(7^{\mbox{th}}\) Ed. [1,2]).

2.2.1 Exercise Descriptive Statistics and t-Test: Tasting Coffee

In a test it was investigated whether consumers are more positive or negative about a certain brand of coffee in a test situation at home compared to tasting the coffee in a laboratory situation. The consumers were asked to score their judgment about the coffee by moving an arrow on a vertical axis with numbers between 0 and 100. You may assume that the differences between the two scores per consumer are normally distributed. The results of 10 randomly selected consumers are given in Table 2.1.

Table 2.1: Coffee tasting data.

Consumer	1	2	3	4	5	6	7	8	9	10
Judgement at home	50	76	81	60	30	70	74	64	76	70
Judgement in the lab	55	74	79	49	34	65	68	66	73	64

These data are also available in the file named tasting_coffee.csv within the course data file you downloaded from Brightspace and extracted into your folder ~/R/Data.

1. Assuming that your default working directory has been set to ~/R, load the data for this exercise by executing in the console:

judgmentData <- read.csv(file = "./data/tasting_coffee.csv")

NoteNote: Eye-ball (inspect) the data

You can inspect the data by printing the object (autoprinting or explicit print command) or by executing View(judgmentData) in the console of RStudio.

ImportantImportant: Transform into vectors

The result might surprise you. Instead of three vector objects, named Consumer, home and lab, we obtain judgmentData with ‘10 observations of 3 variables’, a so called data frame (which will be discussed in more detail in lesson 3).

Since the use of a data.frame object has not been discussed yet, let’s for now transform this data frame into vector objects by executing, sequentially, the following three commands in the console:

consumer <- judgmentData$Consumer
home <- judgmentData$home
lab  <- judgmentData$lab

2. Make a histogram of the judgments at home and ask for summary statistics.

3. Also make a histogram of the judgments in the lab, ask for summary statistics and compare with the result in b.

4. Since the data are paired (a consumer gives a judgment both at home and in a lab) calculate the difference between the two judgments (and call the new vector d).

5. Make a boxplot of d. Is the assumption that the differences between the two scores per consumer are normally distributed reasonable?

6. What is the best estimate of the population mean difference in judgment?

7. How precise is this estimate, i.e. what is the 95% confidence interval for the mean difference?

8. What is the meaning of this 95% confidence interval? So, what can you conclude about the difference in judgment at home as compared to judgment in the lab?

TipTip: Check your answer

A fast way to check your answers is to use t.test(). Look at the help of t.test() and try to figure out which function arguments to use!

2.2.2 Exercise: Comparing samples from Normal Distributions

1. Draw a sample of size 100 from a normal distribution, with mean of 50 and a standard deviation of 5, and assign it to an object named x1.

Note

Looking at the help page for rnorm() (executing ?rnorm` in the console) you can read, which arguments you have to specify to obtain the desired response that creates x1. It should be looking like this:

x1 <-  rnorm(n = 100, mean = 50, sd = 5)

2. Make a histogram of the resulting object x1.

3. Calculate the mean and standard deviation of the sample. Are they what you expected (close to the given mean and standard deviation)?

4. Test in a one-sample t-test if the mean is different from (the expected) 50.

Note

Reading the help page (?t.test) shows that a One-Sample t-test can be performed with:

t.test(x = x1, mu = 50) # alternative = "two.sided" [by default]

5. Draw another sample (assign to x2) from a normal distribution but now with mean = 53 and sd = 5. Again, make a histogram. What is the difference? What changes in the shape of the histogram?

6. Compare the two means using a t-test, in other words test \(\mbox{H}_0:\ \mu_1 = \mu_2\) against the alternative that the means differ. Here you can assume equal variances. What is the conclusion?

NoteNote: Independent Samples T-test and Levene’s Test for Homogeneity of variances

You can do an Independent Samples t-test, with the assumption of equal variances, in R with the following command:

t.test(x1, x2, var.equal = TRUE)

By default the Independent Samples t-test is performed in R with the assumption of unequal variances, and R uses the Welch-Satterthwaite approximation to determine the degrees of freedom.

Levene’s Test for Homogeneity of variances can be performed in R, but this test is not by default available as R function. You can extend the functionality of R with the car package [3] and perform Levene’s Test by executing:

# Load the car package
library(car)
# Group object x1 and x2 in object y
y <- c(x1, x2)
# Create a grouping variable named group
group <- as.factor(c(rep(1, length(x1)), rep(2, length(x2))))
# Perform Levene's Test for Homogeneity of Variance
leveneTest(y = y, group = group, center = mean)
# Unload the car package
detach("package:car", unload = TRUE)

From the output of Levene’s Test you can conclude, whether the Independent Samples t-test should be performed assuming equal variances or not.

7. Now we are going to investigate what would happen if the sample size is much smaller. So, we only want to use the first 12 values of x1 and x2. Make a subset of x1 and x2 of the first 12 values and call the subsets x3 and, respectively, x4.

8. Perform a test to compare the mean of x3 with x4, in other words test the hypothesis \(\mbox{H}_0\mbox{: }\mu_3 = \mu_4\) against the alternative that the means differ (again assume equal variances). What is the conclusion?

9. Compare the result of the test in f. with result in h. and explain the difference.

2.2.3 Exercise: Two-sample t-Test

A full description of this problem can be found in O&L \(6^{\mbox{th}}\) Edition pp. 292-293, and 325-330 [1] or O&L \(7^{\mbox{th}}\) Edition pp. 302-303, and 336-341 [2].

On January 7, 1992, an underground oil pipeline broke and caused the contamination of a marsh along the Chiltipin Creek in Texas, USA. The cleanup process consisted of burning the contaminated regions in the marsh. To evaluate the influence of the oil spill on the mean flora density (\(\mu\)), researchers designed a study of plant growth 1 year after the burning. The data collected consists of 40 measurements of flora density in the uncontaminated (1 = control) sites and 40 density measurements in the contaminated (2 = burned) regions.

The data are available in the file oil_spill.csv.

1. Read in the data with the read.csv() function and assign it to an R object named oilspill. If you don’t remember how to do this, check Section 2.2.1. Transform the data frame (which will be handled in more detail tomorrow) to vectors by executing sequentially: tract <- oilspill$tract, density <- oilspill$density, and site <- oilspill$site.

2. Make a boxplot of the density in the control sites and the oil-spill sites separately using the following command: boxplot(density ~ site, col = "khaki"). Ask for overall summary statistics.

NoteNote: Use of ~ in a formula

In the previous question you have created side-by-side boxplots for density in the control and oil-spill sites. To do this you have specified a formula using the tilde operator, written in R with ~. You can read this operator as versus in a graphical representation. So basically you have asked R to make side-by-side boxplots of density versus site.

The col function argument, as specified in the command, can be used to specify the color of the box in the boxplot.

3. Ask for summary statistics for each site separately. First make two objects (dens_control and dens_oilspill) by means of subsetting in an appropriate way.

4. Carry out the t-test for \(\mbox{H}_0\mbox{: }\mu_1 \leq \mu_2\) against \(\mbox{H}_{\mbox{a}}\mbox{: }\mu_1 > \mu_2\) at \(\alpha = 0.05\). Do not forget to first check with Levene’s Test, leveneTest(y = density, group = as.factor(site), center = mean), whether you can assume variances to be equal or not. Give the outcome of the test-statistic, the p-value for the one-sided problem and the conclusion of the test (in words). Check the help (?t.test) on how to fill in the function arguments.

5. Determine the 0.95 Confidence Interval for \(\mu_{1}\) – \(\mu_{2}\) . Which conclusion follows for the test with \(\mbox{H}_{\mbox{a}}\mbox{: }\mu_1 - \mu_2 \neq 0\)?

6. Another way of performing the t-test is with: t.test(density ~ site). Do this and compare the output of d. with the one just generated. Is there difference, and if so, what is the difference?

2.3 Additional SWIRL Exercises

If you are finished with the statistical exercises before today’s lesson ends, please do the following exercises.

2.3.1 Exercise SWIRL: Workspace and Files

Start swirl() from the package swirl and work through lesson 2 about Workspace and Files in R. In this lesson you will learn more on how to deal with the workspace and do file management from the console.

2.3.2 Exercise SWIRL: Logic

Start swirl() from the package swirl and work through lesson 8 about Logic in R.

2.4 Command Reference

Lesson 2 commands, operators and constants

Command	Description
~	tilde operator used to separate the left- and right-hand sides in a model formula
all()	check if all values are TRUE
any()	check if some values are TRUE
detach()	detach object, o.a. package, from the search path
identical()	test objects for exact equality
is.na()	check for ‘Not Available’ / Missing Values
is.nan()	check for ‘Not a Number’ / result of invalid math operation
isTRUE()	check if object equals TRUE
leveneTest()	test for homogeneity of variance across groups (car package)
names()	get or set the names of an object
View()	Invoke RStudio’s data viewer
which()	check which indices are TRUE

Logical Operators

Command	Description
& or &&	logical and / intersection
| or ||	logical or / union
xor()	logical exclusive or
!	logical negation (NOT)

Relational Operators

Command	Description
<	less than
>	greater than
==	logical equality
<=	less than or equal to
>=	greater than or equal to
!=	not equal to

[1]

R.L. Ott, M. Longnecker, An introduction to statistical methods and data analysis, \(6^{\mbox{th}}\) International Student Edition, Brooks/Cole, Cengage Learning, 2010.

[2]

R.L. Ott, M. Longnecker, An introduction to statistical methods & data analysis, \(7^{\mbox{th}}\) Edition, Cengage Learning, 20 Channel Center Street, Boston, MA 0210, USA, 2016.

[3]

J. Fox, S. Weisberg, An R companion to applied regression, Second, Sage, Thousand Oaks, CA, USA, 2011. http://socserv.socsci.mcmaster.ca/jfox/Books/Companion.