Home / Learn / 14 Testing

Release v0.4.0Edition 0.2

14.1 Examples: do and expect

Page 2 of 10

14.1.1 What an example is

A recipe book shows, next to each recipe, a photo of the finished dish. You follow the steps, then compare what is on your plate with the photo. If they differ, something went wrong, and you know it before the guests arrive.

An example is that photo, for a function. It names a case in plain words, runs the code once with chosen arguments, and states what must be true of what came back. A test is any check that runs code and compares the outcome with what you expect; in Una, examples are the tests.

You have written examples since section 1.7, and every chapter since has used them, mostly in the same three lines: example "…" with a name, a do line that makes one call, and expect result == …. That is the simplest form. This section starts from it and covers every line an example can hold, and every condition an expect line can state. A few words name the parts, and the rest of the chapter uses them:

14.1.2 Why tests live next to the code

Each step of this chapter is a package, made as chapter 2 explains; the chapter's download, 14-testing.tar.gz, holds them all. Here is a cart that adds up its lines. It has a bug: it counts one unit per line, instead of the line's quantity:

steps/01-unchecked/src/cart.unacheckrununa
module cart

--- One line of a cart: a SKU, how many units, and the price of one unit in cents.
pub kind Line
  sku: Text
  quantity: Int
  unit_cents: Int

--- What a whole cart holds: its units, and their price in cents.
pub kind Summary
  units: Int
  cents: Int

--- Adds up the units and the price of every line.
pub summarize(lines: List[Line]) -> Summary
=
  let edit units = 0
  let edit cents = 0
  for line in lines
    units = units + 1
    cents = cents + line.quantity * line.unit_cents
  Summary(units: units, cents: cents)

pub main() -> Unit
  uses @console: Console[print]
=
  let cart = [Line(sku: "LAMP-01", quantity: 2, unit_cents: 1990), Line(sku: "MUG-03", quantity: 1, unit_cents: 890)]
  let summary = summarize(cart)
  @console.print("{summary.units} units, {summary.cents} cents")

una check . passes, and una run . prints:

Output
2 units, 4870 cents

Two lamps and a mug are 3 units, not 2. Nothing in the program says so. The header of summarize promises a Summary, and it returns one; the checker has no way to know that units should count every unit. una check proves many things, such as types, labels and contracts, but not what you meant. You would notice the wrong number only by reading the output, on the one cart that main happens to try, and only if you knew the right answer.

An example writes the right answer down, once, next to the function, and una test compares it on every run. When a later change breaks the count, the example fails, and says where. Because the example sits in the same module, a reader who opens summarize finds, a few lines down, what it must give for a real cart.

14.1.3 How to write an example

An example is a top-level declaration of a module, like a function. Its lines come in this order, each under example indented two spaces:

  1. example "…". The keyword, and a name in double quotes. The name is what una test prints, so write it as the fact the example checks. Names are unique within a module.
  2. given lines, if the code needs services. Each installs a fake for one capability. Section 14.3 covers them; the examples of this section call functions that need none.
  3. One do line. The code under test, run once.
  4. One or more expect lines. Each states something that must hold.

An expect line takes one of four forms: a condition, expect fails, expect performed and expect not performed. This section covers the first. Section 14.2 covers expect fails, and section 14.3 the other two.

The next step shows each way to write the do line and a condition, on a shelf of products:

steps/02-example-forms/src/shelf.unathe code under testuna
--- A product on a shelf: its SKU, its price in cents, and how many units are left.
pub kind Product
  sku: Text
  cents: Int
  units: Int

--- The price of `quantity` units, in cents.
pub price_of(product: Product, quantity: Int) -> Int = product.cents * quantity

--- Whether a product has at least one unit left.
pub in_stock(product: Product) -> Bool = product.units > 0

--- The SKUs of the products that have run out, in shelf order.
pub sold_out(products: List[Product]) -> List[Text] = products.filter(keep: fn(product) -> product.units == 0).map(each: fn(product) -> product.sku)

--- The product with the SKU `sku`, if the shelf holds it.
pub find(products: List[Product], sku: Text) -> Option[Product] = products.first_where(keep: fn(product) -> product.sku == sku)

--- A label for the shelf edge. Private: only this module calls it.
edge_label(product: Product) -> Text = "{product.sku} {product.cents}"

14.1.3.1 A comparison with ==

The most common condition compares result with the value you expect. It is the form of chapter 1:

steps/02-example-forms/src/shelf.unauna
example "three lamps cost 59.70"
  do price_of(Product(sku: "LAMP-01", cents: 1990, units: 4), quantity: 3)
  expect result == 5970

14.1.3.2 Other comparisons, and several expect lines

Any condition that is true or false works: !=, <, <=, >, >=, and and, or, not to combine them (section 3.3). Use them when only a bound matters. An example may hold several expect lines, and every one must hold:

steps/02-example-forms/src/shelf.unauna
example "a price is never below the price of one unit"
  do price_of(Product(sku: "LAMP-01", cents: 1990, units: 4), quantity: 3)
  expect result >= 1990
  expect result != 0

14.1.3.3 A result that is already a condition

When the function returns a Bool, result is the condition itself. expect result states that it is true, and expect not result that it is false:

steps/02-example-forms/src/shelf.unauna
example "a lamp with units left is in stock"
  do in_stock(Product(sku: "LAMP-01", cents: 1990, units: 4))
  expect result

example "a mug with no units left is not"
  do in_stock(Product(sku: "MUG-03", cents: 890, units: 0))
  expect not result

14.1.3.4 Fields, lengths and variants

result is a value like any other, so you can read its fields, its .len, or test its variant with is (section 4.2):

steps/02-example-forms/src/shelf.unauna
example "only the mug has run out"
  do sold_out([Product(sku: "LAMP-01", cents: 1990, units: 4), Product(sku: "MUG-03", cents: 890, units: 0)])
  expect result.len == 1
  expect result == ["MUG-03"]

example "a SKU the shelf does not hold is not found"
  do find([Product(sku: "LAMP-01", cents: 1990, units: 4)], sku: "DESK-02")
  expect result is None

expect result is None holds when find found nothing. A whole value can be compared at once too: expect result == ["MUG-03"] compares the list element by element.

14.1.3.5 Any expression on the do line

The do line is usually one call, but it may be any expression. Here it adds two prices, and result is the sum:

steps/02-example-forms/src/shelf.unauna
example "two lamps and a mug cost 48.70"
  do price_of(Product(sku: "LAMP-01", cents: 1990, units: 4), quantity: 2) + price_of(Product(sku: "MUG-03", cents: 890, units: 0), quantity: 1)
  expect result == 4870

Keep the expression short: the name of the example says what it computes, and a long do line is harder to read than two examples.

14.1.3.6 A call in an expect line

An expect line may call functions too, those of the prelude or of the module, as long as the whole line is a Bool. contains (section 10.1) asks whether the list holds an element:

steps/02-example-forms/src/shelf.unauna
example "the mug is among the SKUs that have run out"
  do sold_out([Product(sku: "LAMP-01", cents: 1990, units: 4), Product(sku: "MUG-03", cents: 890, units: 0)])
  expect result.contains(item: "MUG-03")

14.1.3.7 A private function

An example belongs to its module, so it may call the module's private functions, those without pub, such as a helper you want to pin down on its own:

steps/02-example-forms/src/shelf.unauna
example "the shelf edge shows the SKU and the price"
  do edge_label(Product(sku: "LAMP-01", cents: 1990, units: 4))
  expect result == "LAMP-01 1990"

14.1.3.8 Running the forms

una test . prints one line per example, then a count:

Terminal
$ una test .
example shelf "three lamps cost 59.70" ... ok
example shelf "a price is never below the price of one unit" ... ok
example shelf "a lamp with units left is in stock" ... ok
example shelf "a mug with no units left is not" ... ok
example shelf "only the mug has run out" ... ok
example shelf "a SKU the shelf does not hold is not found" ... ok
example shelf "two lamps and a mug cost 48.70" ... ok
example shelf "the mug is among the SKUs that have run out" ... ok
example shelf "the shelf edge shows the SKU and the price" ... ok

9 examples: 9 passed, 0 failed

14.1.4 Two lamps and a mug

Now the cart of Why tests live next to the code, with the bug fixed and two examples:

steps/03-examples/src/cart.unacheckruntestuna
module cart

--- One line of a cart: a SKU, how many units, and the price of one unit in cents.
pub kind Line
  sku: Text
  quantity: Int
  unit_cents: Int

--- What a whole cart holds: its units, and their price in cents.
pub kind Summary
  units: Int
  cents: Int

--- Adds up the units and the price of every line.
pub summarize(lines: List[Line]) -> Summary
=
  let edit units = 0
  let edit cents = 0
  for line in lines
    units = units + line.quantity
    cents = cents + line.quantity * line.unit_cents
  Summary(units: units, cents: cents)

pub main() -> Unit
  uses @console: Console[print]
=
  let cart = [Line(sku: "LAMP-01", quantity: 2, unit_cents: 1990), Line(sku: "MUG-03", quantity: 1, unit_cents: 890)]
  let summary = summarize(cart)
  @console.print("{summary.units} units, {summary.cents} cents")

example "two lamps and a mug"
  do summarize([Line(sku: "LAMP-01", quantity: 2, unit_cents: 1990), Line(sku: "MUG-03", quantity: 1, unit_cents: 890)])
  expect result.units == 3
  expect result.cents == 4870

example "an empty cart holds nothing"
  do summarize([])
  expect result == Summary(units: 0, cents: 0)
  1. example "two lamps and a mug". The text names the case. It is what una test prints, so write it as a sentence a reader can follow.
  2. do summarize([...]). The do line runs the code under test, once. An example has exactly one do line.
  3. expect result.units == 3. Each expect line is a Bool that must be true. result is the value the do line gave, so you can read its fields. Here there is one expect line for each field.
  4. expect result == Summary(units: 0, cents: 0). Records compare field by field (section 4.1), so one line can state a whole value. The empty cart is the edge case: a loop over no lines must still give a correct summary.

14.1.5 Running the examples with una test

una run . runs main only, and ignores the examples:

Output
3 units, 4870 cents

una test . checks the package first, as una check does, then runs every example of every module and prints one line per example. It runs nothing else: main runs only if an example calls it.

Terminal
$ una test .
example cart "two lamps and a mug" ... ok
example cart "an empty cart holds nothing" ... ok

2 examples: 2 passed, 0 failed

Each line names the module, cart, and the example. ok means every expect line held. The last line counts them.

14.1.6 How an example behaves

14.1.6.1 When an example fails

Put the bug back. Count one unit per line, instead of the line's quantity:

src/cart.unadiff
--- a/src/cart.una
+++ b/src/cart.una
@@ -19,3 +19,3 @@
   for line in lines
-    units = units + line.quantity
+    units = units + 1
     cents = cents + line.quantity * line.unit_cents

The package still checks. una test . finds the bug:

Terminal
$ una test .
example cart "two lamps and a mug" ... FAILED
  src/cart.una:33: expect result.units == 3
  it is false, with `result` = `Summary(units: 2, cents: 4870)`
example cart "an empty cart holds nothing" ... ok

2 examples: 1 passed, 1 failed

Read the failure from the top. FAILED marks the example. The next line gives the file and line of the expect that did not hold, src/cart.una:33, and its text. The last line says why: it is false, with result = …, followed by the whole value the do line gave. Here units is 2, so the count is wrong, and cents is right, so the price is not. una test exits with code 1 when any example fails, so a build script, or a continuous-integration job that checks every change automatically, stops there.

14.1.6.2 The checks run in order

The expect lines of an example are checked from the top, and the first that does not hold fails the example. The lines below it are not reported, so put the most telling check first.

14.1.6.3 Each example runs alone

Each example runs in a fresh machine, as if the program had just started, with nothing left over from another example or from main. The order of examples in the file does not matter, and an example cannot depend on another.

14.1.6.4 An example is code, and is checked

una check . checks examples like any other code: names, types, labels and proofs. An example that compares a number with a text does not run at all:

src/shelf.unadiff
--- a/src/shelf.una
+++ b/src/shelf.una
@@ -31,3 +31,3 @@
   do price_of(Product(sku: "LAMP-01", cents: 1990, units: 4), quantity: 3)
-  expect result == 5970
+  expect result == "59.70"
 
Diagnostic
[U2001] Error: `==` needs operands of one type, and these are `Int` and `Text`
    ╭─[ src/shelf.una:32:10 ]
    │
 32 │   expect result == "59.70"
    │          ────────┬────────
    │                  ╰────────── `==` needs operands of one type, and these are `Int` and `Text`

== compares values of one type, and price_of returns an Int. una test . reports the same diagnostic and runs nothing: a package that does not check has no examples to run.

14.1.6.5 One do line, then at least one expect line

An example with a do line and nothing to expect checks nothing, and Una refuses it:

src/shelf.unadiff
--- a/src/shelf.una
+++ b/src/shelf.una
@@ -65,2 +65 @@
   do edge_label(Product(sku: "LAMP-01", cents: 1990, units: 4))
-  expect result == "LAMP-01 1990"
Diagnostic
[U0101] Error: unexpected end of block, expected `expect`
    ╭─[ src/shelf.una:65:65 ]
    │
 65 │   do edge_label(Product(sku: "LAMP-01", cents: 1990, units: 4))
    │                                                                 │
    │                                                                 ╰─ unexpected end of block, expected `expect`

The mark points at the end of the do line, where Una expected an expect line. The order of the lines is fixed too. A second do line is the same error, U0101, with the message unexpected keyword `do`, expected `expect` : an example tests one piece of code. A given line after the do line is refused the same way.

14.1.7 One fact, stated exactly

A good example states one fact, in its name and in its expect lines. When it fails, its name tells you what broke before you read any code.

Prefer exact values over loose ones. expect result == 4870 catches a price that is off by one cent, which expect result > 0 would miss. A bound is right when the bound is the rule, as in "a price is never below the price of one unit"; section 14.4 shows that such rules usually belong in a contract.

When an example fails, the report alone cannot tell you which side is wrong: the code, or the example. Read the function's doc comment and decide which states what the store means, then fix that side. Never change an example only to make it pass.

14.1.8 An example, a print in main, or a contract

Which to use:

You want to… Use
See what a function gives, once, while you write it a print in main, with una run .
State the right answer, and be told when it changes an example, with una test .
Pin down an edge case, such as an empty cart an example for that case alone
Know that the code has the right types and authority una check ., which needs no example
A rule that must hold for every input a contract (section 14.4)

14.1.9 Check yourself: examples

The cart gains a third example. Its author thinks of units as the number of lines:

steps/07-predict/src/cart.unauna
example "three mugs on one line"
  do summarize([Line(sku: "MUG-03", quantity: 3, unit_cents: 890)])
  expect result.units == 1
  expect result.cents == 2670

What does una test . print for it?

Show the answer
Output
example cart "three mugs on one line" ... FAILED
  src/cart.una:42: expect result.units == 1
  it is false, with `result` = `Summary(units: 3, cents: 2670)`

summarize counts units, so three mugs are 3, and the first expect fails. The second line, about cents, is not reported: the first failing expect ends the example. Here the code is right and the example is wrong, which the report alone cannot tell you: read the function's doc comment, then fix whichever side is wrong.

Next:14.2 Testing failures: expect fails