Tests¶
What CI ran in the build that produced this site, down to each test and how long it took. Every lane that runs tests leaves its results behind, and the report below is generated from them: unit tests under each compiler and sanitiser, integration tests against real upstreams, and the BDD scenarios against every target, each organised by lane, executable and suite, with a view across lanes for the question "where is this tested".
Tested from main at 9ba7f64.
20209 tests across 19 lanes: 20109 passed, none failed, 100 skipped, 47 min 57 s of test time in all.
| Kind | Lanes | Tests | Passed | Failed | Skipped | Time |
|---|---|---|---|---|---|---|
| Unit | 5 | 16893 | 16893 | 0 | 0 | 15 min 7 s |
| Integration | 5 | 279 | 279 | 0 | 0 | 6 min 55 s |
| Build | 1 | 2349 | 2349 | 0 | 0 | 997 ms |
| BDD | 8 | 688 | 588 | 0 | 100 | 25 min 54 s |
- Overview: headline counts, failures first, then each kind of testing with its lanes, and each pack.
- Unit: its 5 lanes, and which lanes ran each suite.
- Integration: its 5 lanes, and which lanes ran each suite.
- Build: its 1 lanes, and which lanes ran each suite.
- BDD: every feature as its file describes it, and every scenario against every target.
- Failures: each with its message and a link to the line.
- Skipped: each with the tags that decided it.
- Slowest: test time by lane, and the slowest suites and tests.
Reading the report¶
The report drills down. The overview leads with failures, then each kind of testing with its lanes. A kind page holds the view across its lanes: for unit and integration testing, which lanes ran each suite and how many of its tests; for BDD, every feature as its file describes it, with every scenario against every target. A lane is one CI job, named by what it does and what it exercises; the CI pipeline describes each.
A unit or integration lane page lists its executables, each executable its suites, and each suite opens to its tests with status, duration and a link to the test's source at the commit that ran. A BDD lane page is the living documentation of that target: each feature's prose, its scenarios with their tags and outcome, and each scenario opening to its steps as they ran, with the time each took. The Skipped page says what a skip means and which tags decided each one.
Failures have a page of their own with the message each left. A pack page gathers the tests that exercise one pack, attributed by the test file that holds them, lane by lane, with the upstream version each lane built the pack against. The overview says how test time is counted.
The overview ends with what each lane compiled against: the compiler, and the version each selected pack's upstream states in its own header, with the tree's git tag beside it where the tag names something the header does not, or the tag or commit alone for a tree whose header states no version. All of it is read at configure time rather than from an image tag. A lane page carries the tag and commit of each tree. The platform pages' "tested against" claims are held to this record by the documentation build, so a version a page names is one CI built against.
To run the same tests locally, build the debug preset's junit target as
Building & testing describes; the BDD scenarios are run as
BDD testing describes.