Automated tests, explained to developers
Code that works today is not the question. The question is what happens at the next change, and who notices first.
What is a test?
A small program that creates data, runs a scenario through Odoo and checks the result: “a confirmed order does create its delivery”. In Odoo, it is written in Python in the module's tests/ directory, and one command runs them all in seconds, with nobody at the screen. A module holds dozens of them.
The rules
-
Written with the code, from the request
Tests start from what was asked, not from the code written. They cover the normal case, the edge cases, the errors, and the access rights: a user without the right must indeed be refused. Each test bears a name that reads like a sentence:
test_a_cancelled_order_releases_its_stock. -
A bug: the test first
When a defect is reported, the test that reproduces it is written first, seen to fail on the current code, and only then comes the fix. A test that never failed proves nothing. Before touching old code without tests, what it does today is first pinned down in tests.
-
A test is never bent
A red test means the code gets fixed, not the test. A test that stays green when the code is wrong (a check always true, a case never run) is a defect, blocking at review, just like a bug.
-
No network
Outside services (a bank, a carrier) are never called during tests: they are replaced by a double, which answers what is expected of them. The result is the same on every machine, at every run.
-
Run again on every change, like the CI
The CI (continuous integration) is a server that runs all the tests and quality checks again on every push. Tests are run with the command the repository documents, and the summary line is what gets read: “0 failed, 0 error(s) of 18 tests”. A summary that ran no test is worth nothing.
-
A review by someone else
Before a change is merged, someone who did not write it reads it: is the request covered, is a behaviour broken, can a test fail, are the access rights there. Each remark cites the file and the line. The delivery says what ran, or “nothing run”.
An example: a reported bug
Four steps, always in this order.
-
The test first
A test reproduces the defect, as it was described, with the data of the case.
-
Red on the old code
The proof that the test does see the defect, and nothing else.
-
The smallest fix
The code changes, the test passes, nothing else moves.
-
Green at every push
The test stays and the CI runs it again: the defect does not come back unseen.
In practice
A test, named for what it guarantees: it creates its data, runs the scenario, and checks one thing.
# timesheet_billing/tests/test_billing.py
from odoo.tests.common import TransactionCase
class TestBilling(TransactionCase):
def test_a_billable_line_becomes_an_invoice_line(self):
# The data of the case: two hours, billable.
line = self._timesheet(hours=2, billable=True)
# The scenario.
invoice = line.bill()
# A single check.
self.assertEqual(invoice.amount_untaxed, 2 * line.rate)
The summary line, the one read at the end: how many tests ran, how many failed. Here, eighteen tests, no failure.
0 failed, 0 error(s) of 18 tests when loading database 'test'
What it costs
Writing the tests takes time, and it is in the estimate. A full suite takes several minutes: write the batch, run once, fix, run once more. And sometimes, a refusal to deliver “quickly, without tests”.
Do you do it differently?
A disagreement, a better way, a case that does not fit these rules: write to me, I read everything.
The other methodologies
-
Modular development, explained to developers
The rules, an example in three modules and two code samples, for whoever starts out with Odoo.
-
OCA standards, explained to developers
A summary of the conventions, the three questions before writing, and the links to the OCA pages that prevail.