Unit testing & continuous integration

# Unit testing & continuous integration
## How to find bugs as soon as you create them
### David A. Selby
### R-thritis Computing Group
### 19 November 2021

---

<div>
<style type="text/css">.xaringan-extra-logo {
width: 67px;
height: 50px;
z-index: 0;
background-image: url(https://www.cfe.manchester.ac.uk/media/mhs/bmh-faculty/images/CfE-logo-150px.jpg);
background-size: contain;
background-repeat: no-repeat;
position: absolute;
bottom:.3em;left:0.3em;
}
</style>
<script>(function () {
  let tries = 0
  function addLogo () {
    if (typeof slideshow === 'undefined') {
      tries += 1
      if (tries < 10) {
        setTimeout(addLogo, 100)
      }
    } else {
      document.querySelectorAll('.remark-slide-content:not(.title-slide):not(.inverse):not(.hide_logo)')
        .forEach(function (slide) {
          const logo = document.createElement('div')
          logo.classList = 'xaringan-extra-logo'
          logo.href = null
          slide.appendChild(logo)
        })
    }
  }
  document.addEventListener('DOMContentLoaded', addLogo)
})()</script>
</div>

---

Typically, we test code to ensure _its output meets our expectations_.

---

### Great Expectations

```r
is_plausible <- function(value, minimum = -Inf, maximum = Inf)
  value >= minimum & value <= maximum
```

What do you .primary[**expect**] the output to be?

```r
is_plausible(c(10, 37, -999), min = 0, max = 112)
```

```
# [1]  TRUE  TRUE FALSE
```

```r
is_plausible("2", min = 0, max = 10)
```

```
# [1] FALSE
```

???

Should `is_plausible.character`:

1. always return `NA`?
2. throw an error?
3. try to coerce to numeric?
4. always return `FALSE`? (But is it really _implausible_?)

As an exercise, try sorting the vector `c(2, 3, "a", 11, "b", "+4")`

---

# Unit testing

---

## Unit testing in R

---

**Unit testing** is the process in which the smallest testable parts of source code are tested individually and independently, usually in an automated way.

- Formalises the testing of code
- Makes it easier to identify bugs when they are introduced
- Helps ensure that the code meets all necessary criteria
- Reassures the user that the code works correctly

---

**testthat** is an R package created by Hadley Wickham for the purpose of writing unit tests for R code.
It is available on CRAN.

Other unit testing packages are available (but not covered here), e.g. [**RUnit**](https://cran.r-project.org/package=RUnit), [**testrmd**](https://github.com/rmflight/testrmd).

---

### Unit testing with **testthat**

---

The **testthat** framework comprises three parts:

<dl><dt>Expectations</dt>
<dd>
the core functions. They all have prefix <code>expect_</code>
</dd>
<dt>Tests</dt>
<dd>
a series of expectations about one feature, wrapped in <code>test_that()</code>
</dd>
<dt>Files</dt>
<dd>containing a set of tests of related functionality</dd>
</dl>

---

.primary[Expectations] in **testthat** compare an object with a reference value or property.
If they .secondary[**do not match**], an error is thrown.

- `expect_equal`
- `expect_is`
- `expect_length`
- `expect_true`
- `expect_error`
- ...

---

`expect_equal` compares a number with a reference value

```r
hyp <- 3^2 + 4^2
expect_equal(hyp, 25) # Runs without error
expect_equal(hyp, 26) # Throws an error:
```

```
# Error: `hyp` not equal to 26.
# 1/1 mismatches
# [1] 25 - 26 == -1
```

```r
expect_equal(sqrt(2), 1.41) # Throws an error:
```

```
# Error: sqrt(2) not equal to 1.41.
# 1/1 mismatches
# [1] 1.41 - 1.41 == 0.00421
```

```r
expect_equal(sqrt(2), 1.41, tolerance = .01) # No error thrown
```

---

`expect_is` checks the data type of an object

```r
val <- 43.7
expect_is(val, 'numeric')   # Runs without error
expect_is(val, 'character') # Produces an error:
```

```
# Error: `val` inherits from `'numeric'` not `'character'`.
```

`expect_length` checks the number of elements in a vector

```r
expect_length(letters, 26) # Runs without error
expect_length(letters, 25) # Throws an error:
```

```
# Error: `letters` has length 26, not length 25.
```

---

Some expectation functions check against a (fixed) condition, rather than a custom reference value.

```r
codes <- c(no = F, yes = T)
expect_true(codes['yes'])    # Runs without error
expect_true('yes')           # Throws an error:
```

```
# Error: "yes" is not TRUE
# 
# `actual` is a character vector ('yes')
# `expected` is a logical vector (TRUE)
```

```r
expect_named(codes)          # Runs without error
expect_false(as.logical(0))  # Runs without error
```

---

Other such functions include `expect_error`, which checks that a function call throws an error when evaluated.

```r
expect_error(3.14 + 'hello')  # Runs without error
expect_error(3.14 < 'hello')  # Throws an error:
```

```
# Error: `3.14 < "hello"` did not throw an error.
```

---

### Back to our example

```r
expect_true( is_plausible(5, min = 0, max = 10) )   # No error
expect_false( is_plausible(-1, min = 0, max = 1) )  # No error
expect_true( is_plausible('2', min = 0, max = 10) ) # Error:
```

```
# Error: is_plausible("2", min = 0, max = 10) is not TRUE
# 
# `actual`:   FALSE
# `expected`: TRUE
```

Now decide:

1. Fix the code just enough to satisfy the expectation,
2. Or: have a rethink?

---

### Back to our example

```r
is_plausible <- function(value, minimum = -Inf, maximum = Inf) {
* value <- as.numeric(value)
  value >= minimum & value <= maximum
}
expect_true( is_plausible('2', min = 0, max = 10) ) # No error
```

Or .primary[rethink], .small[by expecting stricter handling of inputs]:

```r
is_plausible <- function(value, minimum = -Inf, maximum = Inf) {
* stopifnot(is.numeric(value))
  value >= minimum & value <= maximum
}
*expect_error( is_plausible('2', min = 0, max = 10) ) # No error
```

---

### Writing tests

```r
test_that('Scalar integers', {
  expect_true(is_plausible(5, min = 0, max = 10))
  expect_false(is_plausible(-1, min = 0, max = 1))
})
```

```
# Test passed
```

```r
test_that('Integer vectors', {
  expect_equal(is_plausible(0:10, min = 0, max = 10),
               rep(TRUE, 11))
  expect_equal(is_plausible(-1:2, min = 0, max = 1),
               c(F, T, T, F))
})
```

```
# Test passed
```

---

### Writing tests

```r
test_that('Handle unusual or missing inputs', {
  expect_error(is_plausible('2', min = 0, max = 10))
  expect_error(is_plausible(min = 0, max = 10))
  expect_error(is_plausible(2, min = "0", max = 10))
  expect_error(is_plausible(2, min = 0, max = "10"))
})
```

```
# -- Failure (<text>:4:3): Handle unusual or missing inputs ----------------------
# `is_plausible(2, min = "0", max = 10)` did not throw an error.
# 
# -- Failure (<text>:5:3): Handle unusual or missing inputs ----------------------
# `is_plausible(2, min = 0, max = "10")` did not throw an error.
```

---

## Unit testing workflows

- Add a `tests/` folder to an R package
    - run tests with `<Ctrl> + <Shift> + T`
    - tests will also run during `R CMD CHECK`
    
- Or run tests locally in analysis folder:
    - `test_file()` / `test_dir()`
    - Re-test with every edit: `auto_test()`
    
- R Markdown 'test chunks' with `error=TRUE` or [**testrmd**](https://github.com/rmflight/testrmd)

.footnote[Note: most documentation for **testthat** assumes you are writing a package.
Future talk: how to do analysis as an R package...]

---

## Test-driven development

1. Write a test **before** any other code.

2. Check that the test **fails**.

3. Write **enough code to make it pass**.

4. Add another test and iterate steps 1–3.

5. _Refactor_ the code, while ensuring it passes all tests.

???

In a way, `R CMD CHECK` could be considered test-driven development, because the tests your package has to pass were written long before the package.

---

# Continuous integration

---

## Continuous integration

---

In software engineering, .primary[**continuous integration (CI)**] is the practice of merging all changes to a central repository, and .secondary[automatically rebuilding & testing] the code after every change.

- Use version control to track/manage changes
- Use CI software to automatically rerun/retest code
- Identify bugs/conflicts as soon as they're created

**Tools:** <del style="text-decoration: line-through;">Travis CI</del>, GitHub Actions, Bitbucket Pipelines, Gitlab CI/CD, Jenkins

???

People _used to_ recommend Travis CI, but the company sold out and stopped offering a free service.
If browsing the web you might still find old articles talking about an R + GitHub + Travis workflow.

---

Make life easier for yourself:

1. Project metadata in a (dummy) `DESCRIPTION` file
2. R code in `R/` subfolder
3. Tests in a `tests/testthat/` subfolder
4. Run tests quickly: `test_local()` / `<Ctrl>+<Shift>+T`
5. Use Git for version control & CI for automatic testing

```yaml
Package: blah
Version: 0.1
```

---

# Thanks!

Based on a [talk by Lewis Rendell](https://warwick.ac.uk/fac/sci/wdsi/events/wrug/resources/unittesting.pdf)  
at Warwick R User Group, 2017.

---

## Next meeting

Friday 3 December

https://adventofcode.com/

Attempt days 1–2 and share your approach