Home / Learn / 14 Testing

Release v0.4.0Edition 0.2

14.3 Fakes: given, expect performed and expect not performed

Page 4 of 10

14.3.1 What a fake is

A pilot trains in a flight simulator before flying a real plane. The simulator answers like the plane: it climbs, it stalls, it can lose an engine on demand. Nothing is at risk, and the same exercise can be run again and again.

Most code worth testing talks to something: a warehouse, a card processor, a mail service. In Una that something is a capability (chapter 12), a declared service with operations, and the code that does its work is a handler (section 12.3). A fake is a handler written for tests: it answers like the real service, from values you choose, and touches nothing outside the program. A program installs a handler for one call with with (section 12.4); an example installs a fake with a given line.

14.3.2 Why an example cannot use a real service

An example must give the same result on every machine and every day. A real mail service sends mail, a real clock moves, and a real warehouse changes its stock. So an example never reaches the host, and every capability its do line needs must come from a fake.

Here is a function that needs two services. The warehouse says how many units of a SKU are left, and the mail service sends the buyer a message when the stock runs low:

steps/14-no-given/src/restock.unauna
--- The warehouse's stock.
pub capability Stock
  op left(sku: Text) -> Int
    ensures result >= 0

--- The store's mail service.
pub capability Mailer
  op send(to: Text, body: Text) -> Unit

--- Mails the buyer when a SKU runs low, with fewer than 3 units left. Says whether it did.
pub warn_if_low(sku: Text) -> Bool
  uses @stock: Stock[left]
  uses @mailer: Mailer[send]
=
  let units = @stock.left(sku)
  if units < 3
    @mailer.send("buyer@example.invalid", body: "{sku}: {units} left")
    true
  else
    false

An example of it calls it with nothing standing in for the warehouse or the mail service:

steps/14-no-given/src/restock.unauna
example "two units left mails the buyer"
  do warn_if_low("LAMP-01")
  expect result

una check . refuses it, once for each service:

Diagnostic
error U3001 at src/restock.una:43:6: `warn_if_low` needs `@stock: Stock[left]`, which no `given` line binds
error U3001 at src/restock.una:43:6: `warn_if_low` needs `@mailer: Mailer[send]`, which no `given` line binds

Both diagnostics are U3001: warn_if_low needs @stock and @mailer, and nothing in the example provides them. The words given and binds in the message name the line that would: a given line, which installs a fake for one service and is the subject of the next subsection. An example never falls back to a real service. A function that needs a host capability, such as the console, gets a fake of it too (A fake of a host capability).

14.3.3 How to write a fake and install it

14.3.3.1 A given line

given @name = handler(arguments) installs a handler for one capability binding, for this example only. @name is the binding the do call's uses clause names, such as @catalog for uses @catalog: Catalog[price]. One given line per binding, before the do line. The right side names a handler and its arguments: an installation, not any expression.

The next step quotes prices from a catalog service, and has one fake per kind of answer a test needs:

steps/15-fake-forms/src/quotes.unathe code under testuna
--- How the catalog service can fail.
pub kind CatalogError = | CatalogDown

--- Why a quote could not be made.
pub kind QuoteError = NoPrice(sku: Text) | Unavailable

--- The store's catalog, a service on the network.
pub capability Catalog
  op price(sku: Text) -> Option[Int]
    fails CatalogError

--- The price of `quantity` units of a SKU, in cents, as the catalog says today.
pub quote(sku: Text, quantity: Int) -> Int
  uses @catalog: Catalog[price]
  fails QuoteError
=
  let cents = @catalog.price(sku)
    if error fail Unavailable
    if none fail NoPrice(sku)
  cents * quantity

14.3.3.2 A fake that always answers the same

steps/15-fake-forms/src/quotes.unauna
--- A catalog that gives every SKU the same price.
handler flat_price(cents: Int) for Catalog
  op price(sku) = Some(cents)

The handler's parameter, cents, is given where it is installed, so each example chooses the price: given @catalog = flat_price(1990).

14.3.3.3 A fake that holds data

steps/15-fake-forms/src/quotes.unauna
--- A catalog that knows the prices in `prices`, and no others.
handler price_list(prices: Map[Text, Int]) for Catalog
  op price(sku) = prices.get(key: sku)

A map (section 10.4) of the prices it knows. A SKU outside the map gives None, which is the path where the catalog has no price.

14.3.3.4 A fake that fails

steps/15-fake-forms/src/quotes.unauna
--- A catalog that never answers.
handler catalog_down() for Catalog
  op price(sku) = fail CatalogDown

A service that never answers is a case you cannot easily make with a real one. A fake makes it with one line.

14.3.3.5 A fake that decides by its arguments

steps/15-fake-forms/src/quotes.unauna
--- A catalog that decides by the SKU it is asked about: mugs cost 890, and it is down for
--- anything else.
handler mugs_only() for Catalog
  op price(sku) = if sku.starts_with(prefix: "MUG") then Some(890) else fail CatalogDown

An operation of a fake can branch on what it receives, so one handler can answer some calls and fail others.

14.3.3.6 A fake of a host capability

A handler may implement a host capability too, such as Console. This one prints nothing:

steps/15-fake-forms/src/quotes.unauna
--- A console that prints nothing: a fake of a host capability.
handler quiet() for Console
  op print(line) = unit

With it, an example can run a function that prints, main included, without reaching the terminal. A function that holds the console under another name gets the same fake through that name, as A console held under another name shows.

14.3.4 What the code did: expect performed

Each run of an example keeps a trace: the list of operations the code performed, in order, each with its arguments and its outcome. Two forms of expect read it:

The binding is the one a given line installed, and the operation must exist on its capability. Neither line mentions result, so both are checked even when the do line failed.

14.3.4.1 Running the fake forms

steps/15-fake-forms/src/quotes.unathe examplesuna
example "two lamps at a flat price"
  given @catalog = flat_price(1990)
  do quote("LAMP-01", quantity: 2)
  expect result == 3980
  expect performed @catalog.price

example "a SKU the list does not hold has no price"
  given @catalog = price_list(empty_map().with(key: "LAMP-01", value: 1990))
  do quote("DESK-02", quantity: 1)
  expect fails NoPrice

example "a catalog that does not answer"
  given @catalog = catalog_down()
  do quote("LAMP-01", quantity: 1)
  expect fails Unavailable
  expect performed @catalog.price

example "three mugs from a catalog that knows only mugs"
  given @catalog = mugs_only()
  do quote("MUG-03", quantity: 3)
  expect result == 2670

example "main prints, and nothing reaches the terminal"
  given @console = quiet()
  do main()
  expect performed @console.print
Terminal
$ una test .
example quotes "two lamps at a flat price" ... ok
example quotes "a SKU the list does not hold has no price" ... ok
example quotes "a catalog that does not answer" ... ok
example quotes "three mugs from a catalog that knows only mugs" ... ok
example quotes "main prints, and nothing reaches the terminal" ... ok

5 examples: 5 passed, 0 failed

The third example states both what the call returned, a failure, and what it did before failing: it asked the catalog.

14.3.5 Faking the warehouse and the mail service

This step mails the buyer when a SKU runs low. The warehouse promises, in the ensures of its operation, that it never reports fewer than 0 units:

steps/16-fakes/src/restock.unacheckruntestuna
module restock

--- A count of units, never below zero.
kind Units = Int where it >= 0

--- The warehouse's stock.
pub capability Stock
  op left(sku: Text) -> Int
    ensures result >= 0

--- The store's mail service.
pub capability Mailer
  op send(to: Text, body: Text) -> Unit

--- Mails the buyer when a SKU runs low, with fewer than 3 units left. Says whether it did.
pub warn_if_low(sku: Text) -> Bool
  uses @stock: Stock[left]
  uses @mailer: Mailer[send]
=
  let units = @stock.left(sku)
  if units < 3
    @mailer.send("buyer@example.invalid", body: "{sku}: {units} left")
    true
  else
    false

--- A fake warehouse that holds `units` of every SKU.
handler flat_stock(units: Units) for Stock
  op left(sku) = units

--- A fake mail service: it accepts every message and delivers none.
handler outbox() for Mailer
  op send(to, body) = unit

pub main() -> Unit
  uses @console: Console[print]
=
  let low = warn_if_low("LAMP-01") with @stock = flat_stock(2), @mailer = outbox()
  let plenty = warn_if_low("DESK-02") with @stock = flat_stock(40), @mailer = outbox()
  @console.print("LAMP-01 warned: {low}, DESK-02 warned: {plenty}")

example "two units left mails the buyer"
  given @stock = flat_stock(2)
  given @mailer = outbox()
  do warn_if_low("LAMP-01")
  expect result
  expect performed @mailer.send

example "forty units left mails nobody"
  given @stock = flat_stock(40)
  given @mailer = outbox()
  do warn_if_low("LAMP-01")
  expect not result
  expect not performed @mailer.send
  1. handler flat_stock(units: Units) for Stock. A fake warehouse that holds units of every SKU. Units is a refined kind (section 5.3): an Int that is never below zero. Its parameter is given where it is installed, so each example sets the stock it needs: flat_stock(2), flat_stock(40).
  2. handler outbox() for Mailer. A fake mail service that accepts every message. It delivers nothing, and it does not need to: the example only needs to know that a message was sent.
  3. given @stock = flat_stock(2). Installs the handler as @stock for this example only, as with does for one call in main. Every capability the do line needs must have a given line.
  4. expect result and expect not result. warn_if_low returns a Bool, so result is already the condition.
  5. expect performed @mailer.send. This line holds when the trace has at least one send on @mailer. expect not performed @mailer.send holds when it has none.
Terminal
$ una test .
example restock "two units left mails the buyer" ... ok
example restock "forty units left mails nobody" ... ok

2 examples: 2 passed, 0 failed

14.3.6 A console held under another name

The dispatch step of section 12.5 writes its log through @log, the console under a name that says what it is for. Section 12.6 left dispatch without an example, since it prints. With a fake console it can have one:

steps/17-dispatch/src/notices.unauna
--- Sends a notice for every shipment and logs each outcome. One failure does not stop the batch.
pub dispatch(shipments: List[Shipment]) -> Unit
  uses @customers: Customers[find]
  uses @mailer: Mailer[send]
  uses @log: Console[print]
=
  for shipment in shipments
    let outcome = attempt notify(shipment)
    @log.print(log_line(shipment, outcome: outcome))

This step is the program of section 12.5 with two handlers added, quiet and printed, and with its examples replaced by the two of this subsection:

steps/17-dispatch/src/notices.unauna
--- A console that prints nothing: a fake of a host capability.
handler quiet() for Console
  op print(line) = unit

--- A mail service for a developer's machine: it prints each message instead of sending it.
handler printed(prefix: Text) for Mailer
  uses @console: Console[print]
  op send(to, subject, body) = @console.print("{prefix} to {to}: {subject}")

A given line names the binding of the function's header, not its capability. dispatch holds the console as @log, so its example installs the fake console with given @log = quiet():

steps/17-dispatch/src/notices.unauna
example "a skipped shipment still writes a log line"
  given @customers = directory(sample_customers())
  given @mailer = mail_down()
  given @log = quiet()
  do dispatch([Shipment(order: "A-1005", customer: "C-17", carrier: "Rapida")])
  expect performed @log.print
  expect performed @mailer.send

The mail service never answers, so notify fails for the one shipment, and attempt keeps that failure as a value. dispatch itself does not fail, so the example has no expect fails line. expect performed @mailer.send shows that the mail service was asked, and expect performed @log.print that the log was written even though the mail failed.

14.3.6.1 A fake with capabilities of its own

printed is the mail service of section 12.3 that prints each message instead of sending it, so it holds the console itself. A fake's own uses come from the given lines above it. In this example, given @console = quiet() comes first, and printed takes its console from it:

steps/17-dispatch/src/notices.unauna
example "a notice goes out through a mail service that prints"
  given @customers = directory(sample_customers())
  given @console = quiet()
  given @mailer = printed("[mail]")
  do notify(Shipment(order: "A-1001", customer: "C-17", carrier: "Parcelo"))
  expect result == "rosa@example.invalid"
  expect performed @console.print

So the order of the given lines matters here. With given @mailer = printed("[mail]") first, no given line above it binds @console, and Una refuses the example with U3014: printed would need the host's console, which an example never uses.

Terminal
$ una test .
example notices "a skipped shipment still writes a log line" ... ok
example notices "a notice goes out through a mail service that prints" ... ok

2 examples: 2 passed, 0 failed

14.3.7 An order line that never reaches the inventory

The order intake of section 7.7 read its stock from a function, on_hand. In a store, the stock lives in a service. This step replaces that function with the inventory service of section 12.3: the same Inventory capability, the same failure kind, StoreError, and the same two handlers, warehouse and offline. Here StoreError and Inventory are declared pub, since the public functions of the module use them.

steps/18-order-intake/src/orders.unauna
--- How the inventory service can fail.
pub kind StoreError = | Timeout

--- The inventory service: how many units of a SKU are on hand, or `None` for a SKU it does not
--- stock.
pub capability Inventory
  op on_hand(sku: Text) -> Option[Int]
    fails StoreError
steps/18-order-intake/src/orders.unauna
--- An inventory held in memory: the units on hand of each SKU.
handler warehouse(stock: Map[Text, Int]) for Inventory
  op on_hand(sku) = stock.get(key: sku)

--- An inventory that never answers.
handler offline() for Inventory
  op on_hand(sku) = fail Timeout

Order, OrderError and parse_order are those of section 7.7, unchanged. The rest changes in three places:

steps/18-order-intake/src/orders.unauna
--- Confirms that the inventory holds enough units of the order's SKU.
pub check_stock(order: Order) -> Order
  uses @inventory: Inventory[on_hand]
  fails OrderError
=
  let left = @inventory.on_hand(order.sku)
    if error fail Unavailable(sku: order.sku)
    if none fail UnknownSku(sku: order.sku)
  if order.quantity > left
    fail OutOfStock(sku: order.sku, wanted: order.quantity, left: left)
  else
    order

--- One line, from text to an order that can be shipped.
pub accept(line: Text) -> Order
  uses @inventory: Inventory[on_hand]
  fails OrderError
=
  let order = parse_order(line)
    if error pass
  check_stock(order)
    if error pass

--- Every line, accepted or rejected: one bad line does not stop the batch.
pub process(lines: List[Text]) -> List[Result[Order, OrderError]]
  uses @inventory: Inventory[on_hand]
=
  let edit results: List[Result[Order, OrderError]] = []
  for line in lines
    let outcome = attempt accept(line)
    results = results.append(item: outcome)
  results

accept parses the line first, then asks the inventory. A malformed line must be rejected before anyone is asked about stock. An example can state that order of the steps:

steps/18-order-intake/src/orders.unauna
example "a malformed line never reaches the inventory"
  given @inventory = offline()
  do accept("LAMP-01")
  expect fails Malformed
  expect not performed @inventory.on_hand

expect fails Malformed says what the caller sees. expect not performed @inventory.on_hand says what the code did not do: it never asked the inventory. The fake is offline() on purpose. Had the code asked it, the call would have failed with Unavailable, and the first line would fail too. The example of a quantity that is not a number states the same order:

steps/18-order-intake/src/orders.unauna
example "a quantity that is not a number is BadQuantity, before the inventory is asked"
  given @inventory = offline()
  do accept("LAMP-01, two")
  expect fails BadQuantity
  expect not performed @inventory.on_hand

The other examples of the step test the paths that reach the inventory: a line in stock, too many units, an inventory that does not answer, and a SKU it does not know:

Output
example orders "a line in stock is accepted" ... ok
example orders "too many units are out of stock" ... ok
example orders "a malformed line never reaches the inventory" ... ok
example orders "an inventory that does not answer is Unavailable" ... ok
example orders "a SKU the inventory does not know is UnknownSku" ... ok
example orders "a quantity that is not a number is BadQuantity, before the inventory is asked" ... ok

6 examples: 6 passed, 0 failed

14.3.8 A fallback, tested with fakes

The product page of section 12.3 must offer a number even when the inventory does not answer, so orderable turns every failure into 0:

steps/19-orderable/src/orderable.unauna
--- How many units of a SKU a customer may order on the product page. The page must show a
--- number even when the inventory does not answer, and must never promise stock it cannot
--- confirm, so a failure and an unknown SKU both give 0.
orderable(sku: Text) -> Int
  uses @inventory: Inventory[on_hand]
=
  let units = @inventory.on_hand(sku)
    if error use None
    if none use 0
  units

A fallback is a path like any other, and it is easy to break without noticing: the page still shows a number. This step is the package of section 12.3 with two examples added. Its fakes are the handlers of section 12.3, warehouse and offline, and each example reaches one fallback:

steps/19-orderable/src/orderable.unauna
example "an unknown SKU offers 0"
  given @inventory = warehouse(empty_map().with(key: "LAMP-01", value: 4))
  do orderable("CHAIR-09")
  expect result == 0

example "an inventory that does not answer offers 0, and does not fail"
  given @inventory = offline()
  do orderable("LAMP-01")
  expect result == 0
  expect performed @inventory.on_hand
Output
example orderable "an unknown SKU offers 0" ... ok
example orderable "an inventory that does not answer offers 0, and does not fail" ... ok

2 examples: 2 passed, 0 failed

The second example has no expect fails line, and passes: the clause if error use None turns the inventory's failure into a value, so no failure leaves orderable. expect performed @inventory.on_hand shows that the inventory was asked, so the 0 comes from the fallback. If a later change let the failure through, this example would fail and name it.

14.3.9 How fakes behave

14.3.9.1 Fakes are checked too

A fake is a handler, and the verifier checks every handler against the operations it implements. Give flat_stock a plain Int instead of Units, the count that is never below zero:

src/restock.unadiff
--- a/src/restock.una
+++ b/src/restock.una
@@ -27,3 +27,3 @@
 --- A fake warehouse that holds `units` of every SKU.
-handler flat_stock(units: Units) for Stock
+handler flat_stock(units: Int) for Stock
   op left(sku) = units
Diagnostic
[U5001] Error: postcondition of `left` is refuted
    │ Note 2: counterexample: result = -1, units = -1

The counterexample, units = -1, is a warehouse no real store has. A test that passed against it would prove nothing about the real one. Una rejects the fake before any example runs, so every fake keeps the promises that the code under test relies on.

14.3.9.2 A failure nobody expected

expect performed runs even when the do line failed, but the failure still has to be expected. Here is the example of the catalog that does not answer, without its expect fails line:

steps/21-unexpected-failure/src/quotes.unauna
example "a catalog that does not answer was asked"
  given @catalog = catalog_down()
  do quote("LAMP-01", quantity: 1)
  expect performed @catalog.price
Output
example quotes "a catalog that does not answer was asked" ... FAILED
  the `do` line failed with `Unavailable`, and no `expect fails` line expects a failure

The report names the failure, and says that no expect fails line expects it. Add expect fails Unavailable, and the example passes.

14.3.9.3 What expect performed does not say

expect performed says that an operation ran, not how often or with which arguments. When those matter, return them: a value the example can compare is the clearest evidence. una --json test . also lists every operation with its arguments, in the trace of each example, whose fields For agents names.

14.3.9.4 A fake holds no state

Note

A handler holds no state between operations: outbox cannot count the messages it was given, and a fake cannot fail once and then answer. Each given line installs a fresh copy of the handler, and each example runs in a fresh machine, so no example sees what another did.

14.3.10 Which fake to write

Which to use:

The test needs a service that… Write a handler that…
Answers, and the answer does not matter returns a fixed value, or unit
Answers with a value the example chooses takes it as a parameter, such as flat_price(1990)
Knows some data and not other data takes a map or a list, such as price_list(…)
Does not answer fails, such as catalog_down()
Answers some calls and refuses others branches on its arguments, such as card(limit: 10000) in section 14.5
Must not be reached at all fails, such as offline(), and expect not performed

14.3.11 Check yourself: fakes

The restock step gains an example with exactly 3 units left:

steps/22-three-units/src/restock.unauna
example "three units left mails the buyer"
  given @stock = flat_stock(3)
  given @mailer = outbox()
  do warn_if_low("LAMP-01")
  expect performed @mailer.send

Does it pass?

Show the answer

No:

Output
example restock "three units left mails the buyer" ... FAILED
  src/restock.una:60: expect performed @mailer.send
  the trace has no `@mailer.send`

warn_if_low mails when fewer than 3 units are left, units < 3, so 3 units send nothing, and the trace has no send. The fake did its job: it gave the stock the example chose, and the trace shows what the code did with it. A boundary like this one is where examples earn their place.

Next:14.4 What the verifier proves, and what examples show