---
title: "Fundamentals of R"
author: "Evan Carey, created for BH Analytics, LLC"
output: 
  # slidy_presentation:
  #   highlight: pygments
  html_document:
    toc: true
    toc_float: true
  pdf_document:
    keep_tex: yes
    latex_engine: xelatex
    number_sections: yes
    toc: yes
    toc_depth: 1

---

```{r setup, include=FALSE}
knitr::opts_knit$set(warning = FALSE,
                      message = FALSE)
knitr::opts_chunk$set(warning = FALSE,
                      message = FALSE)
```



Objectives 
===========================

-  R overview 
-  RStudio Environment 
-  R as a calculator 
-  Creating variables 
-  Intro to vectors 
-  Intro to dataframes 
-  Intro to graphics
- Structured text files, delimiters
- Importing a CSV with base R
- File paths in R and your working directory
- Introducing the readr package
- Reading Excel Files


Why R? 
===========================

-  'R is a system for statistical computation and graphics.' From the [R FAQ](https://cran.r-project.org/)
-  Open source (free!)
-  Works on Mac/Linux/Windows
-  Comprehensive Statistical/Analytics Platform 
-  Most popular open source software for statistics (or maybe Python)
-  New things implemented quickly.

Read Chapter 2 of [R Programming for Data Science](https://bookdown.org/rdpeng/rprogdatascience/) for more details on the history of R. 

Download and Install The R project for Statistical Computing 
===========================

-  Download  R at URL [www.r-project.org](www.r-project.org) 
-  Install R if you have not already done so. 
-  After installing R, launch it! 

Read Chapter 3 of [R Programming for Data Science](https://bookdown.org/rdpeng/rprogdatascience/) for more details on installation. 


Writing Basic R Code
===========================

Let's write some simple code in R, and learn how to execute it. 

```{r, eval=FALSE}
## Basic Addition
3 + 4

## Function
log(42)

## Save the result as a variable
x1 <- 3 + 4
## Print the result
x1
```


Thought on Formatting Your Code
===========================

WhyCan'tIWriteAllofMySentencesLikeThis?
ItSeemsInefficientToUseSpaces!

-  Grammar Rules for the computer 
    +  The R language expects very specific syntax 
    +  Deviation from that syntax will give errors 
-  Correct Syntax and Readable Code 
    +  We can have correct syntax that doesn't give errors 
    +  But it is a nightmare for humans to read 
-  Style Guides 
    +  Guidelines for formatting code 
    +  Poorly formatted but syntactically correct code will be evaluated by a computer without issue. However, humans maintaining the code will not find the work so easy. 

R Help? 
===========================
-   There is built in documentation for R available, you can open using the following:

```{r, eval=F}
?log
```

-  There is also a large community of online resources, check out these links!
-  Stack Overflow [stackoverflow.com](stackoverflow.com)
-  R-Bloggers.com [www.r-bloggers.com](www.r-bloggers.com)

RStudio - Fancy Front End for R
===========================

-  R is a command line program. Think of it as the analytic engine. 
-  But the Graphical User Interface (GUI) for R is pretty limited...
-  RStudio is a popular GUI to use for R. Think of RStudio as a fancy steering wheel for driving R, which still uses R as the engine. A steering wheel with no engine isn't useful...
-  The homepage for RStudio is [here](https://www.rstudio.com/). Download and install RStudio if you have not already done so. 
-  We will use RStudio from now on, but always remember you can use regular R without RStudio! Occasionally there may be an issue caused by RStudio. Running your code using 'regular' R is an option. 

EX2. Math with RStudio 
===========================

-  Open R studio 
-  Save this as a new script and put a header on it 
-  Establish a reasonable file directory for this training 
-  Let's explore the options ... 
-  Using  R as a nifty calculator 

```{r, echo=TRUE} 
# Simple sums 
4 + 9 
 
# products 
4 * 9 
``` 

Examples 
===========================

```{r, echo=TRUE} 
# log is the natural log (ln) 
log(1000) 
# Use log10 or log(x, base = 10) to get base 10 
log(1000, base = 10) 
 
log10(1000) 
``` 

EX3. More Math 
===========================

>  Question: What is $log_{4}$ of 42? 


EX3. More Math
===========================

>  Question: What is $log_{4}$ of 42? 

```{r, echo=TRUE} 
log(42, base = 4) 
``` 

>  Question: What is $\left(\frac{3 \sin\left( \frac{\pi}{6} \right) +
    32}{\sqrt{23}} + 2\right) - 0.9$ 

-  Hint:  `sin()`, `pi`, `sqrt()` 


EX3. More Math
===========================

>  Question: What is $log_{4}$ of 42? 

```{r, echo=TRUE} 
log(42, base = 4) 
``` 

>  Question: What is $\left(\frac{3 \sin\left( \frac{\pi}{6} \right) +
    32}{\sqrt{23}} + 2\right) - 0.9$ 

-  Hint:  `sin()`, `pi`, `sqrt()` 

```{r, echo=TRUE} 
((3 * sin(pi / 6) + 32) / sqrt(23) + 2) - 0.9 
``` 

-  Order of Operations will be respected 


Assigning Values to Variables 
===========================

-  What if we want to keep a value for later use? 
-  R has five assignment operators: `<-`, `=`, `->`, `<<-`, `->>`
-  In the Console window type `?'<-'` 
-  Avoid `<<-` and `->>` unless you are absolutely sure such an assignment is needed (more on this later in the course.)
-  Do no intermix left and right assignment operators 
-  `<-` is the most commonly used assignment operator 
-  Preferred by Google Style Guide 
-  Psychological implied meaning and difference between `<-` and `=`: 
    + `=` defines values of a function's argument 
    + `<-` defines an object 

Body Mass Index 
===========================

>  Question: What is the BMI of someone who is 6 ft 2 inches and weighs 210 pounds?

```{r, echo=TRUE} 
weight   <- 210 
height   <- 74 
constant <- 703 
bmi      <- (weight * constant)/height ^ 2 
``` 


BMI 
===========================

-  Where's the answer?
-  By default, no output to the console is provided when assignments are made. You may request the output to typing the object's name in the console. 
-  Answer 

```{r, echo=TRUE} 
bmi 
``` 

-   R is not verbose 


BMI extended
===========================

>  Question: What is the BMI of this person if they lost 20 pounds?

```{r, echo=TRUE} 
weight <- weight - 20 
bmi 
# What happened? 
bmi      <- (weight * constant)/height ^ 2 
bmi 
``` 

- What does this imply about R variables?
- If changes are made to intermediate variables, they must be recalculated


What variables, functions, are in my workspace? 
===========================

```{r, echo=TRUE,error=TRUE} 
# BMI will generate an error
BMI

# What happened? 
``` 

```{r}
bmi
```


```{r, echo=TRUE} 
ls() 
``` 

Removing Objects from the Workspace 
===========================

```{r, echo=TRUE} 
ls() 
bmi 
rm("bmi") 
ls()  
#bmi #ask for it, it will no longer be there and generate an error
``` 

Removing All Objects from the Workspace  
===========================

```{r, echo=TRUE} 
ls() 
 
# Etch-A-Sketch End of The World!  
# Clear the whole workspace! 
rm(list = ls()) 
ls() 
``` 


Writing a simple script 
===========================

-  Open RStudio and a new script 
-  Let's get into the habit in creating a meaningful header for scripts 
-  We can work with more than a single value at a time 
-  `help('c')` 
-  Let's look at a simple example of height and weight data 


```{r, echo=TRUE} 
# data input 
height <- c( 60,  62,  61,  65) 
 
weight <- c(135, 155, 145, 155)   
 
sex    <- c("f", "m", "f", "f") 
``` 


Use script to find BMI 
===========================

-  Find the body mass index BMI for each subject 

```{r, echo=TRUE} 
# bmi = weight (lb) per height (inches) squared 
constant <- 703 
bmi      <- (weight  * constant )/ height^2 
bmi
``` 

-  Note the differences in the length of the vector and the constant 

```{r, echo=TRUE} 
length(bmi) 
length(constant) 
``` 

- R *recycled* the constant term in the vector operation 

Some simple summary statistics 
===========================

There are simple summary statistics available (and many more complex ones!)

```{r, echo=TRUE} 
mean(weight) 
median(height) 
quantile(bmi) 
``` 

Object Types 
===========================

-  There are many types of objects and many data structures 
-  These concepts will be explored in more detail soon 

```{r, echo=TRUE} 
mean(sex) 
# find the mode of the objects 
mode(bmi) 
mode(sex) 
``` 

Subsetting / Indexing A Vector
===========================

-  What is the height of the 3rd subject 

```{r, echo=TRUE} 
height[3]  
``` 

-  What is the bmi of the 1st 2nd and 3rd subjects 

```{r, echo=TRUE} 
bmi[c(1,2,3)] 
bmi[1:3] 
``` 

Logical Vector 
===========================

-  Indexing by logical vectors is possible too 

```{r, echo=TRUE} 
males <- sex == "m" 
males 
height[males] 
 
# can combine the work into one line, e.g., height of females: 
height[sex == "f"] 
``` 

EX4. Using logical indexing 
===========================

>  Question: What is the bmi of males over 5'1"? 

```{r, echo=TRUE} 
# help("&") 
idx <- sex == "m" & height > 61 
bmi[idx] 
``` 

>  Question: What is the bmi of patients that are female or under 5'3"? hint: use `|` to represent or 
 

EX4. Using logical indexing 
===========================

>  Question: What is the bmi of males over 5'1"? 

```{r, echo=TRUE} 
# help("&") 
idx <- sex == "m" & height > 61 
bmi[idx] 
``` 

>  Question: What is the bmi of patients that are female or under 5'3"? hint: use `|` to represent or 
 
```{r, echo=TRUE} 
idx <- sex == "f" | height < 63 
bmi[idx] 
``` 


Simple Scatterplot 
===========================

Basic graphs are quickly available:

```{r, echo=TRUE} 
plot(height, weight) 
``` 

EX5. Make Simple Scatterplot 
===========================

>  Question: Make a plot of BMI versus height 
 
```{r, echo=TRUE} 
plot(bmi, height) 
``` 

Introduction to Data Frames 
===========================

-  The primary data analysis object in R is the dataframe 
-  A data frame is a list of vectors columns 
-  You might think of this like an Excel spreadsheet

Data Frame:

Height | Weight | Sex
---|---|---
70 | 170 | m
55 | 150 | f
66 | 160 | m

Introduction to Dataframes 
===========================

-  The primary data analysis object in R is the data frame 
-  A data frame is a list of vectors (columns) 
-  You may think of this like an Excel spreadsheet for now 
-  We can touch the columns of a dataframe using the dollar sign `$`
-  The columns are now locked together into one object 

```{r, echo=TRUE} 
my_df <- data.frame(weight, height) 
my_df
``` 

```{r}
# Calculate the mean height 
mean(my_df$height) 
```


Dataframes 2 
===========================

-  You can use tab completion in Rstudio following `$`

>  Question: Calculate the mean weight 
 
```{r, echo=TRUE} 
# Calculate the mean weight 
mean(my_df$weight) 
``` 

Adding a New Variable `$`
===========================

-  We can add a new variable by simple assignment using the `$`
-  Here is an example of adding `gender`

```{r, echo=TRUE} 
my_df$gender <- sex 
``` 

>  Question: Can you figure out how to create a BMI variable (redo the calculation using `$`)? 

```{r, echo=TRUE} 
my_df$bmi <- (my_df$weight * 703)/my_df$height ^ 2 
``` 

Save the script 
===========================

-  Save the  `01 _Introduction.R` file 
-  Close up RStudio 
-  Let's discuss the options for saving when closing  R 
-  Reopen R studio inspect the workspace 
-  Rerun your script 

Extending R 
===========================

-  As powerful as  R is the base install doesn't have everything 
-  Additional functionality can be found in packages 
-  See CRAN 
-  To install a package: (This only needs to be done once per R major version) 
-  When you see  `library(xxxx)` you should first install the package `xxxx` 

```{r}
# library(ggplot2)
# install.packages("ggplot2")
```


Loading packages
===========================

-  Load the extra code per session

```{r, echo=TRUE} 
# quick graph  
library(ggplot2) 
qplot(height,weight)  
``` 


Structured Text Files
==========================

A common method of storing data is to use delimited text file. Delimited text files store data one row at a time, with each column separate by a 'delimiter'. One example is a comma separated values file, often referred to as a CSV. In a CSV file, the delimiter is a comma. Usually the first row of the text file  has the names of each column, and subsequent rows of text contain the actual data. A table of data with this information:

![](../common/Dataframe_overview.png)

Would look like this as a csv file:

```{r, comment=""}
## create data frame
df1 <- 
  data.frame(age=c(56,24,35,42),
             gender=c('M','M','F','M'),
             salary=c('35k','35k','50k','25k'))
## write out dataframe
write.csv(df1,
          file = "test1.csv",
          row.names = F) # don't include row names
## Check where the file went!
getwd()
```

Most statistical programs can output data as a csv. This is one of the most common file formats for passing data between programs (like from SQL to R, or from SAS to R, etc). The main advantage is any program can read the data. The main disadvantage is there is no 'metadata' indicating what each column type should be. Programs must guess, or the programmer must specify. 

There are other options for delimiters you may encounter. We could use anything other than a comma. Sometimes files may be pipe delimited:


```{r, comment=""}
write.table(df1,
            file = "test2.csv",
            row.names = F,
            sep = "|")
```

And the variable names may or may not be present!

When you receive a file of data, you should always first open it with a text editor (not import it to R) and verify the delimiter, as well as if the first row is variable names or data. 

Importing Data Using Base R
===========================

* So how do we import data from a comma delimited file into R? We can simply use the `read.csv()` function, and the appropriate arguments. 
* Remember to first examine the data with a text editor, then try to import into R. 
* You will need to tell R where the file is on your file system by using a filepath. You cannot use single backslashes (like normal windows paths), because the backslash is a special command in R (the escape command). Blame Microsoft, this is only an issue in Windows...
* You can either use double backslashes (which escapes the escape, so it works...), or just use forward slashes. Here is an example of where this doesn't work:

```{r, error=TRUE}
# Made up file paths:
# doesn't work, would give error
"c:\users\evan\file1.txt"
# this works
"c:\\users\\evan\\file1.txt"
# this works
"c:/users/evan/file1.txt"
```

- `\u` is actually a special command for unicode. There are other special commands, like `\t` for tab, or `\n` for newline. Here I demonstrate them using the `cat()` function, which just sends (prints) the text to the console:

```{r}
# print hello world
cat('Hello world!')

# print some text with special commands
cat('Here is a tab\tdid you see it?')
cat('Here is a newline\ndid you see it?')
```


```{r}
# Import csv into R using base functions
# You must alter this path to point at wherever you have downloaded the file on your system!
df_sales <- 
  read.csv('../Data/customer_sales1.csv')
# Examine the file after you import (as always!)
head(df_sales)
str(df_sales)
```


* If you are importing data that has some other delimiter, you will need to use `read.table()`
* For example, here is a file with the pipe delimiter (which you would discover after you open the examine the file with a text editor). 
* Notice this file did not have any variable names! We have to add them afterwards then. 

```{r}
# import pipe delimited file
df2 <- 
  read.table(file='../data/chickweight.txt',
             sep = '|')
head(df2)
str(df2)
# Add names - we would need to know these from some other source. 
names(df2) <- 
  c("weight", "Time", "Chick", "Diet")
```

Shortcoming of the Base Import Functions
============================

* You can use those base functions to import data. However, R has to try and guess what each column should be (unless you specify it). Also, by default, R converts all the characters into factors (which some people don't like). Finally, it can be slow on larger data. 

* For those reasons, there are a few different packages implemented to make reading data bit easier. We will focus on the `readr` package, which is part of the tidyverse collection of packages. We will spend a bit of time going over all the packages in the tidyverse group in subsequent lectures. 

* For these simple cases, we can just use `read_csv()` to replace `read.csv()`, and `read_table()` to replace `read.table()`.

* The main difference you will notice is:
    + Character values remain character instead of factors
    + The print method for the dataframe is nicer (it won't print the whole dataframe and overwhelm your screen). This is because the return object is actually a `tbl` (pronounced tibble), which just a fancy version of a dataframe. 

```{r}
# using readr
library(readr)
# Import csv into R using readr functions
# You must alter this path to point at wherever you have downloaded the file on your system!
df_sales <- 
  read_csv('../data/customer_sales1.csv')
# Examine the file after you import (as always!)
df_sales
str(df_sales)
class(df_sales)
```


```{r}
# import pipe delimited file
df2 <- 
  read_delim(file='../data/chickweight.txt',
            delim = '|')
df2
```

Perform some of those data manipulations from PBI:
=================================

* replace med with medium  
* add a year column  
* Add the conditional income category 


```{r}
head(df_sales)
```

```{r}
# check unique values
unique(df_sales$activity)
```

```{r}
df_sales$activity[df_sales$activity=='Med'] <- 
  'Medium'
```

We can add the transaction year like so: 


```{r}
library(lubridate)
df_sales$transaction_year <- 
  year(df_sales$transaction_Date)
```


And lets explore the income category:


```{r}
# examine income 
summary(df_sales$income)

# create bins:
df_sales$income_cat <- 
  cut(df_sales$income,
      breaks = c(0,60,70,105),
      labels = c('low','medium','high'))
```


Conclusion
========================

We have covered a lot of ground of base R functionality! Does anyone have any questions about what we covered so far ?


let's step back into PowerBI and show how to use R as a data source, building off the script we just created. 