Home / Learn / 14 Testing
14.4 What the verifier proves, and what examples show
Page 5 of 10
14.4.1 Two kinds of evidence
A contract (chapter 5) and an example both state what a function must do, and Una checks both. A contract or an example, in section 5.2, compared them in a table. This section shows the difference on real bugs:
- A contract,
requiresandensuresin the header, is proven by the verifier for every input, before the program runs. It states a rule, such as "the fee is never above 4.90". - An example is checked by running the code on one input. It states one exact case, such as "an order of 49.99 pays 4.90".
Neither is weaker. A contract covers inputs no one thought to try. An example states the exact answer, which a contract that gives a range does not.
14.4.2 Why one kind is not enough
Take the store's shipping fee: 4.90, and free from 50.00 on. The rule comes from the store's terms, so the step records it as a requirement and ties the contract to it with derived from, as section 5.5 teaches:
module shipping
requirement SHIPPING_1
text "Orders of 50.00 or more ship for free"
source policy "Store terms, section 4"
owner checkout
status active
--- The shipping fee of an order, in cents: 4.90, and nothing from 50.00 on.
pub shipping_fee(subtotal: Int) -> Int
ensures result >= 0 and result <= 490
ensures subtotal >= 5000 implies result == 0
derived from SHIPPING_1
=
if subtotal >= 5000 then 0 else 490
pub main() -> Unit
uses @console: Console[print]
=
@console.print("49.99 pays {shipping_fee(4999)}, 50.00 pays {shipping_fee(5000)}")
example "an order of 49.99 pays 4.90"
do shipping_fee(4999)
expect result == 490
example "an order of exactly 50.00 ships for free"
do shipping_fee(5000)
expect result == 0Now compare two bugs.
14.4.2.1 A bug the verifier catches: off by one
In the first, the threshold is off by one, > instead of >=:
--- a/src/shipping.una
+++ b/src/shipping.una
@@ -15 +15 @@
- if subtotal >= 5000 then 0 else 490
+ if subtotal > 5000 then 0 else 490error U5001 at src/shipping.una:15:34: postcondition of `shipping_fee` is refuted
counterexample: result = 490, subtotal = 5000The verifier rejects it before anything runs, and its counterexample, subtotal = 5000, is exactly the order the bug charges.
14.4.2.2 A bug only an example catches: a dropped zero
In the second bug, a zero is missing from the fee:
--- a/src/shipping.una
+++ b/src/shipping.una
@@ -15 +15 @@
- if subtotal >= 5000 then 0 else 490
+ if subtotal >= 5000 then 0 else 49The package checks clean: 49 cents is between 0 and 490, so every promise still holds. Only the example sees it:
$ una test .
example shipping "an order of 49.99 pays 4.90" ... FAILED
src/shipping.una:24: expect result == 490
it is false, with `result` = `49`
example shipping "an order of exactly 50.00 ships for free" ... ok
2 examples: 1 passed, 1 failedThe contract states a range, and the example states one exact value. Neither replaces the other.
14.4.3 The parts of the shipping rule
requirement SHIPPING_1. The rule in words, where it came from, and who answers for it (section 5.5).ensures. Two promises, proven for everysubtotal: the fee is between 0 and 4.90, and an order of 50.00 or more pays nothing.derived from SHIPPING_1. Ties the contract to the requirement.- The examples. Two chosen cases, one on each side of the threshold: 49.99 pays 4.90, and exactly 50.00 pays nothing.
14.4.4 How proofs and examples work together
14.4.4.1 una test proves before it runs
una test . checks the package first, proofs included. A bug that breaks a contract therefore stops every example of the package: the verifier reports it, with a counterexample, and nothing runs. A bug that keeps every contract reaches the examples, which are the only check left.
14.4.4.2 A counterexample becomes an example
The counterexample of the off-by-one bug, subtotal = 5000, is the order of the second example, "an order of exactly 50.00 ships for free". From a counterexample to an example, in section 5.2, keeps such a case as an example. The proof already covers it; the example shows a reader the edge the rule turns on, in the words of a real order.
14.4.4.3 What each one covers
A contract covers every input, but only the facts it states. An example covers one input, but states the whole answer, and with fakes, the effects. So write the rule as a contract, and add an example at each edge of the rule, such as each side of a threshold, and for each exact value a reader should see.
14.4.5 Choosing a contract or an example
Which to use:
| You want to… | Use |
|---|---|
| A rule that must hold for every input, such as a bound or a threshold | ensures in the header, proven by una check |
| A precondition that callers must meet | requires |
| An exact value for a case a reader should see | an example with expect result == … |
| A failure path | an example with expect fails |
| What a call did, or did not do, to a service | an example with fakes and expect performed |
| Where a rule came from | a requirement and derived from (section 5.5) |
14.4.6 Check yourself: proofs and examples
A third bug: the fee becomes 5.00 instead of 4.90.
--- a/src/shipping.una
+++ b/src/shipping.una
@@ -15 +15 @@
- if subtotal >= 5000 then 0 else 490
+ if subtotal >= 5000 then 0 else 500Which catches it first: una check, or the example "an order of 49.99 pays 4.90"?
Show the answer
una check, before any example runs:
error U5001 at src/shipping.una:15:35: postcondition of `shipping_fee` is refuted
counterexample: result = 500, subtotal = 4999500 is above the bound of the first ensures, result <= 490, so the verifier refutes it, with an order of 49.99 as the counterexample. The example would catch it too, but una test checks the package first and runs nothing when it does not check. A bug that breaks a contract is caught for every input; a bug that stays inside the contract, like the dropped zero, needs an example.