Home / Learn / 14 Testing
14.3 Fakes: given, expect performed and expect not performed
Page 4 of 10
14.3.1 What a fake is
A pilot trains in a flight simulator before flying a real plane. The simulator answers like the plane: it climbs, it stalls, it can lose an engine on demand. Nothing is at risk, and the same exercise can be run again and again.
Most code worth testing talks to something: a warehouse, a card processor, a mail service. In Una that something is a capability (chapter 12), a declared service with operations, and the code that does its work is a handler (section 12.3). A fake is a handler written for tests: it answers like the real service, from values you choose, and touches nothing outside the program. A program installs a handler for one call with with (section 12.4); an example installs a fake with a given line.
14.3.2 Why an example cannot use a real service
An example must give the same result on every machine and every day. A real mail service sends mail, a real clock moves, and a real warehouse changes its stock. So an example never reaches the host, and every capability its do line needs must come from a fake.
Here is a function that needs two services. The warehouse says how many units of a SKU are left, and the mail service sends the buyer a message when the stock runs low:
--- The warehouse's stock.
pub capability Stock
op left(sku: Text) -> Int
ensures result >= 0
--- The store's mail service.
pub capability Mailer
op send(to: Text, body: Text) -> Unit
--- Mails the buyer when a SKU runs low, with fewer than 3 units left. Says whether it did.
pub warn_if_low(sku: Text) -> Bool
uses @stock: Stock[left]
uses @mailer: Mailer[send]
=
let units = @stock.left(sku)
if units < 3
@mailer.send("buyer@example.invalid", body: "{sku}: {units} left")
true
else
falseAn example of it calls it with nothing standing in for the warehouse or the mail service:
example "two units left mails the buyer"
do warn_if_low("LAMP-01")
expect resultuna check . refuses it, once for each service:
error U3001 at src/restock.una:43:6: `warn_if_low` needs `@stock: Stock[left]`, which no `given` line binds
error U3001 at src/restock.una:43:6: `warn_if_low` needs `@mailer: Mailer[send]`, which no `given` line bindsBoth diagnostics are U3001: warn_if_low needs @stock and @mailer, and nothing in the example provides them. The words given and binds in the message name the line that would: a given line, which installs a fake for one service and is the subject of the next subsection. An example never falls back to a real service. A function that needs a host capability, such as the console, gets a fake of it too (A fake of a host capability).
14.3.3 How to write a fake and install it
14.3.3.1 A given line
given @name = handler(arguments) installs a handler for one capability binding, for this example only. @name is the binding the do call's uses clause names, such as @catalog for uses @catalog: Catalog[price]. One given line per binding, before the do line. The right side names a handler and its arguments: an installation, not any expression.
The next step quotes prices from a catalog service, and has one fake per kind of answer a test needs:
--- How the catalog service can fail.
pub kind CatalogError = | CatalogDown
--- Why a quote could not be made.
pub kind QuoteError = NoPrice(sku: Text) | Unavailable
--- The store's catalog, a service on the network.
pub capability Catalog
op price(sku: Text) -> Option[Int]
fails CatalogError
--- The price of `quantity` units of a SKU, in cents, as the catalog says today.
pub quote(sku: Text, quantity: Int) -> Int
uses @catalog: Catalog[price]
fails QuoteError
=
let cents = @catalog.price(sku)
if error fail Unavailable
if none fail NoPrice(sku)
cents * quantity14.3.3.2 A fake that always answers the same
--- A catalog that gives every SKU the same price.
handler flat_price(cents: Int) for Catalog
op price(sku) = Some(cents)The handler's parameter, cents, is given where it is installed, so each example chooses the price: given @catalog = flat_price(1990).
14.3.3.3 A fake that holds data
--- A catalog that knows the prices in `prices`, and no others.
handler price_list(prices: Map[Text, Int]) for Catalog
op price(sku) = prices.get(key: sku)A map (section 10.4) of the prices it knows. A SKU outside the map gives None, which is the path where the catalog has no price.
14.3.3.4 A fake that fails
--- A catalog that never answers.
handler catalog_down() for Catalog
op price(sku) = fail CatalogDownA service that never answers is a case you cannot easily make with a real one. A fake makes it with one line.
14.3.3.5 A fake that decides by its arguments
--- A catalog that decides by the SKU it is asked about: mugs cost 890, and it is down for
--- anything else.
handler mugs_only() for Catalog
op price(sku) = if sku.starts_with(prefix: "MUG") then Some(890) else fail CatalogDownAn operation of a fake can branch on what it receives, so one handler can answer some calls and fail others.
14.3.3.6 A fake of a host capability
A handler may implement a host capability too, such as Console. This one prints nothing:
--- A console that prints nothing: a fake of a host capability.
handler quiet() for Console
op print(line) = unitWith it, an example can run a function that prints, main included, without reaching the terminal. A function that holds the console under another name gets the same fake through that name, as A console held under another name shows.
14.3.4 What the code did: expect performed
Each run of an example keeps a trace: the list of operations the code performed, in order, each with its arguments and its outcome. Two forms of expect read it:
expect performed @mailer.sendholds when the trace has at least onesendon the binding@mailer, whether it completed or failed.expect not performed @mailer.sendholds when it has none.
The binding is the one a given line installed, and the operation must exist on its capability. Neither line mentions result, so both are checked even when the do line failed.
14.3.4.1 Running the fake forms
example "two lamps at a flat price"
given @catalog = flat_price(1990)
do quote("LAMP-01", quantity: 2)
expect result == 3980
expect performed @catalog.price
example "a SKU the list does not hold has no price"
given @catalog = price_list(empty_map().with(key: "LAMP-01", value: 1990))
do quote("DESK-02", quantity: 1)
expect fails NoPrice
example "a catalog that does not answer"
given @catalog = catalog_down()
do quote("LAMP-01", quantity: 1)
expect fails Unavailable
expect performed @catalog.price
example "three mugs from a catalog that knows only mugs"
given @catalog = mugs_only()
do quote("MUG-03", quantity: 3)
expect result == 2670
example "main prints, and nothing reaches the terminal"
given @console = quiet()
do main()
expect performed @console.print$ una test .
example quotes "two lamps at a flat price" ... ok
example quotes "a SKU the list does not hold has no price" ... ok
example quotes "a catalog that does not answer" ... ok
example quotes "three mugs from a catalog that knows only mugs" ... ok
example quotes "main prints, and nothing reaches the terminal" ... ok
5 examples: 5 passed, 0 failedThe third example states both what the call returned, a failure, and what it did before failing: it asked the catalog.
14.3.5 Faking the warehouse and the mail service
This step mails the buyer when a SKU runs low. The warehouse promises, in the ensures of its operation, that it never reports fewer than 0 units:
module restock
--- A count of units, never below zero.
kind Units = Int where it >= 0
--- The warehouse's stock.
pub capability Stock
op left(sku: Text) -> Int
ensures result >= 0
--- The store's mail service.
pub capability Mailer
op send(to: Text, body: Text) -> Unit
--- Mails the buyer when a SKU runs low, with fewer than 3 units left. Says whether it did.
pub warn_if_low(sku: Text) -> Bool
uses @stock: Stock[left]
uses @mailer: Mailer[send]
=
let units = @stock.left(sku)
if units < 3
@mailer.send("buyer@example.invalid", body: "{sku}: {units} left")
true
else
false
--- A fake warehouse that holds `units` of every SKU.
handler flat_stock(units: Units) for Stock
op left(sku) = units
--- A fake mail service: it accepts every message and delivers none.
handler outbox() for Mailer
op send(to, body) = unit
pub main() -> Unit
uses @console: Console[print]
=
let low = warn_if_low("LAMP-01") with @stock = flat_stock(2), @mailer = outbox()
let plenty = warn_if_low("DESK-02") with @stock = flat_stock(40), @mailer = outbox()
@console.print("LAMP-01 warned: {low}, DESK-02 warned: {plenty}")
example "two units left mails the buyer"
given @stock = flat_stock(2)
given @mailer = outbox()
do warn_if_low("LAMP-01")
expect result
expect performed @mailer.send
example "forty units left mails nobody"
given @stock = flat_stock(40)
given @mailer = outbox()
do warn_if_low("LAMP-01")
expect not result
expect not performed @mailer.sendhandler flat_stock(units: Units) for Stock. A fake warehouse that holdsunitsof every SKU.Unitsis a refined kind (section 5.3): anIntthat is never below zero. Its parameter is given where it is installed, so each example sets the stock it needs:flat_stock(2),flat_stock(40).handler outbox() for Mailer. A fake mail service that accepts every message. It delivers nothing, and it does not need to: the example only needs to know that a message was sent.given @stock = flat_stock(2). Installs the handler as@stockfor this example only, aswithdoes for one call inmain. Every capability thedoline needs must have agivenline.expect resultandexpect not result.warn_if_lowreturns aBool, soresultis already the condition.expect performed @mailer.send. This line holds when the trace has at least onesendon@mailer.expect not performed @mailer.sendholds when it has none.
$ una test .
example restock "two units left mails the buyer" ... ok
example restock "forty units left mails nobody" ... ok
2 examples: 2 passed, 0 failed14.3.6 A console held under another name
The dispatch step of section 12.5 writes its log through @log, the console under a name that says what it is for. Section 12.6 left dispatch without an example, since it prints. With a fake console it can have one:
--- Sends a notice for every shipment and logs each outcome. One failure does not stop the batch.
pub dispatch(shipments: List[Shipment]) -> Unit
uses @customers: Customers[find]
uses @mailer: Mailer[send]
uses @log: Console[print]
=
for shipment in shipments
let outcome = attempt notify(shipment)
@log.print(log_line(shipment, outcome: outcome))This step is the program of section 12.5 with two handlers added, quiet and printed, and with its examples replaced by the two of this subsection:
--- A console that prints nothing: a fake of a host capability.
handler quiet() for Console
op print(line) = unit
--- A mail service for a developer's machine: it prints each message instead of sending it.
handler printed(prefix: Text) for Mailer
uses @console: Console[print]
op send(to, subject, body) = @console.print("{prefix} to {to}: {subject}")A given line names the binding of the function's header, not its capability. dispatch holds the console as @log, so its example installs the fake console with given @log = quiet():
example "a skipped shipment still writes a log line"
given @customers = directory(sample_customers())
given @mailer = mail_down()
given @log = quiet()
do dispatch([Shipment(order: "A-1005", customer: "C-17", carrier: "Rapida")])
expect performed @log.print
expect performed @mailer.sendThe mail service never answers, so notify fails for the one shipment, and attempt keeps that failure as a value. dispatch itself does not fail, so the example has no expect fails line. expect performed @mailer.send shows that the mail service was asked, and expect performed @log.print that the log was written even though the mail failed.
14.3.6.1 A fake with capabilities of its own
printed is the mail service of section 12.3 that prints each message instead of sending it, so it holds the console itself. A fake's own uses come from the given lines above it. In this example, given @console = quiet() comes first, and printed takes its console from it:
example "a notice goes out through a mail service that prints"
given @customers = directory(sample_customers())
given @console = quiet()
given @mailer = printed("[mail]")
do notify(Shipment(order: "A-1001", customer: "C-17", carrier: "Parcelo"))
expect result == "rosa@example.invalid"
expect performed @console.printSo the order of the given lines matters here. With given @mailer = printed("[mail]") first, no given line above it binds @console, and Una refuses the example with U3014: printed would need the host's console, which an example never uses.
$ una test .
example notices "a skipped shipment still writes a log line" ... ok
example notices "a notice goes out through a mail service that prints" ... ok
2 examples: 2 passed, 0 failed14.3.7 An order line that never reaches the inventory
The order intake of section 7.7 read its stock from a function, on_hand. In a store, the stock lives in a service. This step replaces that function with the inventory service of section 12.3: the same Inventory capability, the same failure kind, StoreError, and the same two handlers, warehouse and offline. Here StoreError and Inventory are declared pub, since the public functions of the module use them.
--- How the inventory service can fail.
pub kind StoreError = | Timeout
--- The inventory service: how many units of a SKU are on hand, or `None` for a SKU it does not
--- stock.
pub capability Inventory
op on_hand(sku: Text) -> Option[Int]
fails StoreError--- An inventory held in memory: the units on hand of each SKU.
handler warehouse(stock: Map[Text, Int]) for Inventory
op on_hand(sku) = stock.get(key: sku)
--- An inventory that never answers.
handler offline() for Inventory
op on_hand(sku) = fail TimeoutOrder, OrderError and parse_order are those of section 7.7, unchanged. The rest changes in three places:
check_stockasks the service. It calls@inventory.on_handwhere 7.7 calledon_hand, so it declaresuses @inventory: Inventory[on_hand]. So doesaccept, which calls it, since a caller holds at least what its callees use (section 12.1).Unavailablemeans that the inventory did not answer. It still carries the SKU, andreasonnow says "the inventory did not answer about" that SKU.- A batch function replaces
report.processruns every line of a batch and keeps each outcome withattempt, so one bad line does not stop the others.describewrites the report line of one outcome, andmainruns the batch.
--- Confirms that the inventory holds enough units of the order's SKU.
pub check_stock(order: Order) -> Order
uses @inventory: Inventory[on_hand]
fails OrderError
=
let left = @inventory.on_hand(order.sku)
if error fail Unavailable(sku: order.sku)
if none fail UnknownSku(sku: order.sku)
if order.quantity > left
fail OutOfStock(sku: order.sku, wanted: order.quantity, left: left)
else
order
--- One line, from text to an order that can be shipped.
pub accept(line: Text) -> Order
uses @inventory: Inventory[on_hand]
fails OrderError
=
let order = parse_order(line)
if error pass
check_stock(order)
if error pass
--- Every line, accepted or rejected: one bad line does not stop the batch.
pub process(lines: List[Text]) -> List[Result[Order, OrderError]]
uses @inventory: Inventory[on_hand]
=
let edit results: List[Result[Order, OrderError]] = []
for line in lines
let outcome = attempt accept(line)
results = results.append(item: outcome)
resultsaccept parses the line first, then asks the inventory. A malformed line must be rejected before anyone is asked about stock. An example can state that order of the steps:
example "a malformed line never reaches the inventory"
given @inventory = offline()
do accept("LAMP-01")
expect fails Malformed
expect not performed @inventory.on_handexpect fails Malformed says what the caller sees. expect not performed @inventory.on_hand says what the code did not do: it never asked the inventory. The fake is offline() on purpose. Had the code asked it, the call would have failed with Unavailable, and the first line would fail too. The example of a quantity that is not a number states the same order:
example "a quantity that is not a number is BadQuantity, before the inventory is asked"
given @inventory = offline()
do accept("LAMP-01, two")
expect fails BadQuantity
expect not performed @inventory.on_handThe other examples of the step test the paths that reach the inventory: a line in stock, too many units, an inventory that does not answer, and a SKU it does not know:
example orders "a line in stock is accepted" ... ok
example orders "too many units are out of stock" ... ok
example orders "a malformed line never reaches the inventory" ... ok
example orders "an inventory that does not answer is Unavailable" ... ok
example orders "a SKU the inventory does not know is UnknownSku" ... ok
example orders "a quantity that is not a number is BadQuantity, before the inventory is asked" ... ok
6 examples: 6 passed, 0 failed14.3.8 A fallback, tested with fakes
The product page of section 12.3 must offer a number even when the inventory does not answer, so orderable turns every failure into 0:
--- How many units of a SKU a customer may order on the product page. The page must show a
--- number even when the inventory does not answer, and must never promise stock it cannot
--- confirm, so a failure and an unknown SKU both give 0.
orderable(sku: Text) -> Int
uses @inventory: Inventory[on_hand]
=
let units = @inventory.on_hand(sku)
if error use None
if none use 0
unitsA fallback is a path like any other, and it is easy to break without noticing: the page still shows a number. This step is the package of section 12.3 with two examples added. Its fakes are the handlers of section 12.3, warehouse and offline, and each example reaches one fallback:
example "an unknown SKU offers 0"
given @inventory = warehouse(empty_map().with(key: "LAMP-01", value: 4))
do orderable("CHAIR-09")
expect result == 0
example "an inventory that does not answer offers 0, and does not fail"
given @inventory = offline()
do orderable("LAMP-01")
expect result == 0
expect performed @inventory.on_handexample orderable "an unknown SKU offers 0" ... ok
example orderable "an inventory that does not answer offers 0, and does not fail" ... ok
2 examples: 2 passed, 0 failedThe second example has no expect fails line, and passes: the clause if error use None turns the inventory's failure into a value, so no failure leaves orderable. expect performed @inventory.on_hand shows that the inventory was asked, so the 0 comes from the fallback. If a later change let the failure through, this example would fail and name it.
14.3.9 How fakes behave
14.3.9.1 Fakes are checked too
A fake is a handler, and the verifier checks every handler against the operations it implements. Give flat_stock a plain Int instead of Units, the count that is never below zero:
--- a/src/restock.una
+++ b/src/restock.una
@@ -27,3 +27,3 @@
--- A fake warehouse that holds `units` of every SKU.
-handler flat_stock(units: Units) for Stock
+handler flat_stock(units: Int) for Stock
op left(sku) = units[U5001] Error: postcondition of `left` is refuted
│ Note 2: counterexample: result = -1, units = -1The counterexample, units = -1, is a warehouse no real store has. A test that passed against it would prove nothing about the real one. Una rejects the fake before any example runs, so every fake keeps the promises that the code under test relies on.
14.3.9.2 A failure nobody expected
expect performed runs even when the do line failed, but the failure still has to be expected. Here is the example of the catalog that does not answer, without its expect fails line:
example "a catalog that does not answer was asked"
given @catalog = catalog_down()
do quote("LAMP-01", quantity: 1)
expect performed @catalog.priceexample quotes "a catalog that does not answer was asked" ... FAILED
the `do` line failed with `Unavailable`, and no `expect fails` line expects a failureThe report names the failure, and says that no expect fails line expects it. Add expect fails Unavailable, and the example passes.
14.3.9.3 What expect performed does not say
expect performed says that an operation ran, not how often or with which arguments. When those matter, return them: a value the example can compare is the clearest evidence. una --json test . also lists every operation with its arguments, in the trace of each example, whose fields For agents names.
14.3.9.4 A fake holds no state
Note
A handler holds no state between operations: outbox cannot count the messages it was given, and a fake cannot fail once and then answer. Each given line installs a fresh copy of the handler, and each example runs in a fresh machine, so no example sees what another did.
14.3.10 Which fake to write
Which to use:
| The test needs a service that… | Write a handler that… |
|---|---|
| Answers, and the answer does not matter | returns a fixed value, or unit |
| Answers with a value the example chooses | takes it as a parameter, such as flat_price(1990) |
| Knows some data and not other data | takes a map or a list, such as price_list(…) |
| Does not answer | fails, such as catalog_down() |
| Answers some calls and refuses others | branches on its arguments, such as card(limit: 10000) in section 14.5 |
| Must not be reached at all | fails, such as offline(), and expect not performed |
14.3.11 Check yourself: fakes
The restock step gains an example with exactly 3 units left:
example "three units left mails the buyer"
given @stock = flat_stock(3)
given @mailer = outbox()
do warn_if_low("LAMP-01")
expect performed @mailer.sendDoes it pass?
Show the answer
No:
example restock "three units left mails the buyer" ... FAILED
src/restock.una:60: expect performed @mailer.send
the trace has no `@mailer.send`warn_if_low mails when fewer than 3 units are left, units < 3, so 3 units send nothing, and the trace has no send. The fake did its job: it gave the stock the example chose, and the trace shows what the code did with it. A boundary like this one is where examples earn their place.