---
title: "Shell & git"
author: "Colin Rundel"
date: "2018-01-19"
output:
  xaringan::moon_reader:
    css: "slides.css"
    lib_dir: libs
    nature:
      highlightStyle: github
      highlightLines: true
      countIncrementalSlides: false
---

exclude: true

```{r, message=FALSE, warning=FALSE, include=FALSE}
options(
  htmltools.dir.version = FALSE, # for blogdown
  width=80
)

htmltools::tagList(rmarkdown::html_dependency_font_awesome())
```

---
class: middle
count: false

# Using the shell

---

## Before we start

I've created a small repo containing a simple file structure we will be navigating around today. Before we start you should obtain a copy of these files on the same filesystem as RStudio by selecting / entering the following:

* File > New Project... >   
Version Control >  
Git > `https://github.com/Sta523-Fa18/filesystem.git`

---

## Why the shell

* Programmatically interact with OS (file system)
* Origins in the 1960s and 1970s
* "designed by computer scientists for computer scientists" 
* Evolution - interactive command language to scripting programming language
* "quick and dirty" prototyping

---

## Unix Design Philosophy

> Even though the UNIX system introduces a number of innovative programs and techniques, no single program or idea makes it work well. Instead, what makes it effective is the approach to programming, a philosophy of using the computer. Although that philosophy can't be written down in a single sentence, at its heart is the idea that the power of a system comes more from the relationships among programs than from the programs themselves. Many UNIX programs do quite trivial things in isolation, but, combined with other programs, become general and useful tools."


Brian Kernighan and Rob Pike - The UNIX Programming Environment

---

## Anatomy of a Unix Command

```shell
cr173@saxon [filesystem]$ ls -l --all ./
total 3
drwxr-xr-x. 4 cr173 visitor  5 Aug 31 09:53 .
drwxr-xr-x. 4 cr173 visitor  4 Aug 31 09:53 ..
drwxr-xr-x. 2 cr173 visitor  5 Aug 29 22:38 data
-rwxr-xr-x. 1 cr173 visitor 88 Aug 29 22:38 haiku.txt
drwxr-xr-x. 4 cr173 visitor  5 Aug 29 22:38 users
```

* `cr173@saxon [filesystem]$` is the prompt

* `ls` is the command, list directory contents in this case

* `-l` is a short flag / option, `l` here means long listing format

* `--all` is a long flag / option, `--all` is equivalent to `-a`

* `./` is the argument, `./` here refers to the current directory

---

## Where am I? (`pwd`, `ls`, `cd`)

```shell
cr173@saxon [~]$ pwd
/home/vis/cr173
```

```shell
cr173@saxon [~]$ ls -l
total 5
drwx------+  2 cr173 visitor 11 Aug 25 19:53 mail
drwx------+  2 cr173 visitor  2 Dec 12 2011  Mail
drwxr-xr-x+  2 cr173 visitor  2 Aug 25 20:37 Sta523
```

```shell
cr173@saxon [~]$ cd Sta523
cr173@saxon [Sta523]$ pwd
/home/vis/cr173/Sta523
```

---

## Absolute vs relative paths

In unix paths can either be absolute or relative, and the difference is very important. For portability reasons you should almost never use absolute paths.

Absolute path examples:

```shell
/var/ftp/pub
/etc/samba.smb.conf
/boot/grub/grub.conf
```

Relative path examples:

```shell
Sta523/filesystem/
data/access.log
filesystem/nelle/pizza.cfg
```

---


## Special directories

```shell
cr173@saxon [Sta523]$ ls -l
total 1
drwxr-xr-x+ 2 cr173 visitor 2 Aug 25 20:41 filesystem
```

--

```shell
cr173@saxon [Sta523]$ ls -la
total 18
drwxr-xr-x+  3 cr173 visitor   3 Aug 25 20:41 .
drwxr-xr-x+ 69 cr173 visitor 104 Aug 25 20:37 ..
drwxr-xr-x+  2 cr173 visitor   2 Aug 25 20:41 filesystem
```

--

```shell
cr173@saxon [Sta523]$ pwd
/home/vis/cr173/Sta523
```

--

```shell
cr173@saxon [Sta523]$ cd .
cr173@saxon [Sta523]$ pwd
/home/vis/cr173/Sta523
```

--

```shell
cr173@saxon [Sta523]$ cd ..
cr173@saxon [~]$ pwd
/home/vis/cr173/
```

---

## Home directory and `~`

Tilde (`~`) is a special character that expands to the name of your home directory. If you append a user's login to `~`, it then refers to that user's home directory.

```shell
cr173@saxon [filesystem]$ cd ~
cr173@saxon [~]$ pwd
/home/vis/cr173
```

--

```shell
cr173@saxon [~]$ cd ~mc301
cr173@saxon [mc301]$ pwd
/home/fac/mc301
```

---

## Copying, moving and deleting (`cp`, `mv`, `rm`)

```shell
cr173@saxon [filesystem]$ ls -l
total 4
drwxr-xr-x+ 2 cr173 visitor  5 Aug 25 20:56 data
-rw-r--r--+ 1 cr173 visitor 88 Aug 25 21:07 haiku.txt                            
drwxr-xr-x+ 4 cr173 visitor  5 Aug 25 20:53 users
```

--

```shell
cr173@saxon [filesystem]$ cp haiku.txt awesome_haiku.txt
cr173@saxon [filesystem]$ ls -l
total 5
-rw-r--r--+ 1 cr173 visitor 88 Aug 25 21:07 awesome_haiku.txt
drwxr-xr-x+ 2 cr173 visitor  5 Aug 25 20:56 data
-rw-r--r--+ 1 cr173 visitor 88 Aug 25 21:07 haiku.txt
drwxr-xr-x+ 4 cr173 visitor  5 Aug 25 20:53 users
```

--

```shell
cr173@saxon [filesystem]$ rm awesome_haiku.txt
cr173@saxon [filesystem]$ ls -l
total 4
drwxr-xr-x+ 2 cr173 visitor  5 Aug 25 20:56 data
-rw-r--r--+ 1 cr173 visitor 88 Aug 25 21:07 haiku.txt
drwxr-xr-x+ 4 cr173 visitor  5 Aug 25 20:53 users
```

---

```shell
cr173@saxon [filesystem]$ mv haiku.txt better_haiku.txt
cr173@saxon [filesystem]$ ls -l
total 4
-rw-r--r--+ 1 cr173 visitor 88 Aug 25 21:07 better_haiku.txt
drwxr-xr-x+ 2 cr173 visitor  5 Aug 25 20:56 data
drwxr-xr-x+ 4 cr173 visitor  5 Aug 25 20:53 users
```

--

```shell
cr173@saxon [filesystem]$ mv better_haiku.txt haiku.txt
cr173@saxon [filesystem]$ ls -l
total 4
-rw-r--r--+ 1 cr173 visitor 88 Aug 25 21:07 haiku.txt
drwxr-xr-x+ 2 cr173 visitor  5 Aug 25 20:56 data
drwxr-xr-x+ 4 cr173 visitor  5 Aug 25 20:53 users
```

---

## Wildcards and the shell (globs)

These are characters that the shell will try to expand to match existing paths and files.

* `*` -  matches any number of characters in a filename, including none.
* `?` -  matches any single character.
* `[ ]` - set of characters that may match a single character at that position.

Note - These are similar but distinct from regular expressions which we will discuss later.

---

## Exercise

Make sure that you have cloned the filesystem repository to your home directory and change to `filesystem` directory

```shell
cr173@saxon [~]$ cd ~/filesystem
```

Along with your neighbors, explore and map out all of the files and subdirectories that are contained within `filesystem`. It will probably be easiest to do this by drawing a tree on a piece of paper.


---
class: middle
count: false

# git

---
class: center

## Why version control?

```{r echo=FALSE}
knitr::include_graphics("imgs/phd_comics_vc.gif")
```

---

## Why git?

* Distributed
    * Work online or offline
    * Collaborate with large groups

* Popular
    * Active development
    * Shiny new tools and ecosystems
    * Fast

* Tracks any type of file
    * Works best with text 

* Branching
    * Smart merges
---

## Some initial configuration

We want to let `git` know who we are so there are some simple configuration options we should set.

Let's first tell `git` who we are, and what editor we want to use.

```bash
$ git config --global user.name "Colin Rundel"
$ git config --global user.email "rundel@gmail.com"
$ git config --global push.default simple
$ git config --global core.editor nano
$ git config --global credential.helper 'cache --timeout=600000'
```

Note - these options are specific to the machine on which they are set, you will need to do this on any other machine you want to use with git.

On Windows the credential.helper should instead be,
```bash
git config --global credential.helper wincred
```


---

## Cloning a repository

```bash
cr173@saxon [Sta523]$ git clone https://github.com/Sta523-Fa18/hw1-rundel.git
Cloning into 'hw1-rundel'...
Username for 'https://github.com': rundel
Password for 'https://rundel@github.com':
remote: Counting objects: 30, done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 30 (delta 14), reused 30 (delta 14), pack-reused 0
Unpacking objects: 100% (30/30), done.
```

```bash
cr173@saxon [Sta523]$ ls
exercises  filesystem  hw1-rundel
cr173@saxon [Sta523]$ ls -a hw1-rundel/
fizzbuzz.png  hw1.Rmd  hw1.Rproj  hw1_whitelist.R  README.md  wercker.yml
```

---

## Checking the status

```bash
cr173@saxon [Sta523]$ cd hw1-rundel
cr173@saxon [hw1-rundel]$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
```

---

## Making changes

Lets create a `README.md` file to include team member names and email addresses.


```bash
cr173@saxon [hw1-rundel]$ nano hw1.Rmd
```

```bash
cr173@saxon [hw1-rundel]$ cat hw1.Rmd
Homework 1 - Colin Rundel
========================================================

### Task 1 - Implementation
...
```

---

## Checking the status again

```bash
cr173@saxon [hw1-rundel]$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   hw1.Rmd

no changes added to commit (use "git add" and/or "git commit -a")
```

---

## Adding a new file

```bash
cr173@saxon [hw1-rundel]$ nano TODO
```

```bash
cr173@saxon [hw1-rundel]$ cat TODO
- Add other team member's info
- Fix formatting
```

---

## Checking the status yet again

```bash
cr173@saxon [hw1-rundel]$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   README.md

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    TODO

no changes added to commit (use "git add" and/or "git commit -a")
```

---

## Lets save our changes

* One file `README.md` is tracked and modified (repo is already has this file but our current version differs from the previously saved version)

* The other file `TODO` is untracked (this file does not exist in the repo)

Our next step is the same for both files - we want to *stage* our changes using `git add`.

```bash
cr173@saxon [hw1-rundel]$ git add hw1.Rmd
cr173@saxon [hw1-rundel]$ git add TODO
```

--

```bash
cr173@saxon [hw1-rundel]$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    modified:   README.md
    new file:   TODO

```

---

## What is staging?

By using `git add` we have made git aware of the current version of both files, but we have not actually saved the changes yet.

To save the changes (locally) we need to *commit* them using `git commit`, since this change will be saved it is customary to add a message about the nature of the changes being made (for future reference).

```bash
cr173@saxon [hw1-rundel]$ git commit -m "Added Colin's information to README; Added TODO file."
[master f9c548c] Added Colin's information to README; Added TODO file.
 2 files changed, 7 insertions(+), 2 deletions(-)
 create mode 100644 TODO
```

--

```bash
cr173@saxon [hw1-rundel]$ git status
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
  (use "git push" to publish your local commits)
nothing to commit, working directory clean
```

We've saved our changes but we're now out of sync with the repository on github.

---

## Making the problem worse

What if at the same time I was making these edits my teammate John Doe was making changes to `README.md` on github?

.center[
![github README edit](imgs/github_HW1_edit.png)
![github README commit](imgs/github_HW1_commit.png)
]

---

## Syncing repos

We can attempt to send our changes back to github by using `git push`

```bash
cr173@saxon [hw1-rundel]$ git push
To git@github.com:Sta523-Sp17/hw1-rundel.git
 ! [rejected]        master -> master (fetch first)
error: failed to push some refs to 'git@github.com:Sta523-Sp17/hw1-rundel.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
```

---

## Merging remote changes

```bash
cr173@saxon [hw1-rundel]$ git pull
remote: Counting objects: 3, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From github.com:Sta523-Sp17/hw1-rundel
   6b79df7..804e091  master     -> origin/master
Auto-merging README.md
CONFLICT (content): Merge conflict in README.md
Automatic merge failed; fix conflicts and then commit the result.
```

---

```bash
cr173@saxon [hw1-rundel]$ git status
On branch master
Your branch and 'origin/master' have diverged,
and have 1 and 1 different commit each, respectively.
  (use "git pull" to merge the remote branch into yours)
You have unmerged paths.
  (fix conflicts and run "git commit")

Unmerged paths:
  (use "git add <file>..." to mark resolution)

    both modified:   README.md

no changes added to commit (use "git add" and/or "git commit -a")
```

---

```bash
cr173@saxon [hw1-rundel]$ cat README.md 
hw1-rundel Repo
=======

## Team Members

<<<<<<< HEAD
Homework 1 - Colin Rundel
=======
Homework 1 - John Doe
>>>>>>> 804e09178910383c128035ce67a58c9c1df3f558
```

* `<<<<<<<` - Indicates the start of the merge conflict.
* `=======` - Indicates the break point used for comparison. 
* `>>>>>>>` - Indicates the end of the lines that had a merge conflict.

---

## Resolving merge conflicts

Edit the merged file to reflect the changes you actually want.

```bash
cr173@saxon [hw1-rundel]$ nano README.md
```

```bash
cr173@saxon [hw1-rundel]$ cat README.md
Homework 1 - John Doe & Colin Rundel
========================================================

### Task 1 - Implementation

...
```

---

## Stage and commit changes

```bash
cr173@saxon [hw1-rundel]$ git add hw1.Rmd 
cr173@saxon [hw1-rundel]$ git status
On branch master
Your branch and 'origin/master' have diverged,
and have 1 and 1 different commit each, respectively.
  (use "git pull" to merge the remote branch into yours)
All conflicts fixed but you are still merging.
  (use "git commit" to conclude merge)

Changes to be committed:

    modified:   README.md

$ git commit -m "Fix Merge conflict"
```

---

```bash
cr173@saxon [hw1-rundel]$ git status
On branch master
Your branch is ahead of 'origin/master' by 2 commits.
  (use "git push" to publish your local commits)
nothing to commit, working directory clean
```

```bash
cr173@saxon [hw1-rundel]$ git push
Counting objects: 7, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (7/7), 791 bytes | 0 bytes/s, done.
Total 7 (delta 1), reused 0 (delta 0)
To git@github.com:Sta523-Sp17/hw1-rundel.git
   804e091..9c4a5e7  master -> master
```

---

## Acknowledgments
Above materials are derived in part from the following sources:

* Karl Broman - [Wisc's Tools4RR](http://kbroman.org/Tools4RR/)
* [Software Carpentry Project](http://software-carpentry.org/)