Tests
Tested from main at 9ba7f64.
On this page: Failures · Unit · Integration · Build · BDD · By pack · Versions
Test time is the sum of every test's own duration, across lanes that ran in parallel; it is not the run's wall clock.
Failures
No failures in this run.
Unit 16893 tests, 16893 passed, 15 min 7 s
CppUTest against the fakes, under each compiler and sanitiser the lanes run. The Unit page has the view across these lanes.
| Lane | Executables | Suites | Tests | Passed | Failed | Skipped | Time |
|---|---|---|---|---|---|---|---|
| unit-linux-clang | 19 | 194 | 2736 | 2736 | 0 | 0 | 3.5 s |
| unit-linux-gcc | 64 | 289 | 4330 | 4330 | 0 | 0 | 3 min 58 s |
| unit-linux-gcc-sanitize | 63 | 271 | 4126 | 4126 | 0 | 0 | 10 min 56 s |
| unit-linux-gcc-tunables | 30 | 213 | 3186 | 3186 | 0 | 0 | 4.0 s |
| unit-windows-msvc | 19 | 173 | 2515 | 2515 | 0 | 0 | 4.9 s |
Integration 279 tests, 279 passed, 6 min 55 s
CppUTest against a real upstream, where a fake would prove too little. The Integration page has the view across these lanes.
| Lane | Executables | Suites | Tests | Passed | Failed | Skipped | Time |
|---|---|---|---|---|---|---|---|
| integration-linux-littlefs | 1 | 2 | 14 | 14 | 0 | 0 | 2 ms |
| integration-linux-mbedtls | 1 | 4 | 77 | 77 | 0 | 0 | 6 min 48 s |
| integration-linux-openssl | 1 | 3 | 45 | 45 | 0 | 0 | 2.1 s |
| integration-linux-wolfssl | 2 | 6 | 98 | 98 | 0 | 0 | 299 ms |
| integration-windows-openssl | 1 | 3 | 45 | 45 | 0 | 0 | 4.2 s |
Build 2349 tests, 2349 passed, 997 ms
A build lane that also runs the unit tests it builds. The Build page has the view across these lanes.
| Lane | Executables | Suites | Tests | Passed | Failed | Skipped | Time |
|---|---|---|---|---|---|---|---|
| build-linux-wolfssl | 14 | 174 | 2349 | 2349 | 0 | 0 | 997 ms |
BDD 688 tests, 588 passed, 100 skipped, 25 min 54 s
Gherkin scenarios driving a real target against the syslog oracle. The BDD page has the view across these lanes.
| Lane | Executables | Suites | Tests | Passed | Failed | Skipped | Time |
|---|---|---|---|---|---|---|---|
| bdd-cmsisrtos-lwipsocket-mbedtls-littlefs-syslogng | 1 | 22 | 86 | 72 | 0 | 14 | 4 min 56 s |
| bdd-cmsisrtos-lwipsocket-wolfssl-littlefs-syslogng | 1 | 22 | 86 | 72 | 0 | 14 | 5 min 1 s |
| bdd-freertos-lwipraw-mbedtls-fatfs-syslogng | 1 | 22 | 86 | 72 | 0 | 14 | 4 min 48 s |
| bdd-freertos-plustcp-mbedtls-plusfat-syslogng | 1 | 22 | 86 | 72 | 0 | 14 | 4 min 11 s |
| bdd-linux-openssl-syslogng | 1 | 22 | 86 | 81 | 0 | 5 | 2 min 3 s |
| bdd-linux-openssl-syslogng-passthrough | 1 | 22 | 86 | 58 | 0 | 28 | 46.1 s |
| bdd-linux-wolfssl-syslogng | 1 | 22 | 86 | 81 | 0 | 5 | 2 min 3 s |
| bdd-windows-openssl-otel | 1 | 22 | 86 | 80 | 0 | 6 | 2 min 6 s |
By pack
Tests attributed by the test file that holds them. A pack's figure counts every lane that ran it.
| Pack | Lanes | Tests | Passed | Failed | Skipped | Time |
|---|---|---|---|---|---|---|
| BDD scenarios | 8 | 688 | 588 | 0 | 100 | 25 min 54 s |
| BDD target | 1 | 202 | 202 | 0 | 0 | 1 ms |
| CmsisRtos | 5 | 170 | 170 | 0 | 0 | 1 ms |
| Core | 6 | 8104 | 8104 | 0 | 0 | 1 min 56 s |
| FatFs | 2 | 124 | 124 | 0 | 0 | 1 ms |
| FreeRtos | 2 | 72 | 72 | 0 | 0 | 1 ms |
| LittleFs | 3 | 128 | 128 | 0 | 0 | 10 ms |
| LwipRaw | 2 | 276 | 276 | 0 | 0 | 5 ms |
| LwipSocket | 2 | 236 | 236 | 0 | 0 | 5 ms |
| MbedTls | 3 | 763 | 763 | 0 | 0 | 19 min 43 s |
| OpenSsl | 7 | 2045 | 2045 | 0 | 0 | 21.1 s |
| PlusFat | 2 | 100 | 100 | 0 | 0 | 0 ms |
| PlusTcp | 2 | 276 | 276 | 0 | 0 | 3 ms |
| Posix | 5 | 1520 | 1520 | 0 | 0 | 18 ms |
| StdAtomic | 5 | 55 | 55 | 0 | 0 | 0 ms |
| Test support | 6 | 1751 | 1751 | 0 | 0 | 14 ms |
| VxWorks64 | 5 | 1720 | 1720 | 0 | 0 | 78 ms |
| Windows | 1 | 81 | 81 | 0 | 0 | 11 ms |
| WolfSsl | 5 | 1898 | 1898 | 0 | 0 | 2.1 s |
Versions
What each lane compiled against, from the record its configure wrote: the compiler, and the version each selected pack's upstream states in its own header. A lane page has the tag and commit of each tree.
| Lane | Compiler | Upstreams |
|---|---|---|
| bdd-cmsisrtos-lwipsocket-mbedtls-littlefs-syslogng | GNU 14.2.1 (arm-none-eabi-gcc) | CmsisRtos v6.4.0 / v11.3.0, LittleFs 2.11 (v2.11.3), LwipSocket 2.2.1, MbedTls 3.6.7 |
| bdd-cmsisrtos-lwipsocket-wolfssl-littlefs-syslogng | GNU 14.2.1 (arm-none-eabi-gcc) | CmsisRtos v6.4.0 / v11.3.0, LittleFs 2.11 (v2.11.3), LwipSocket 2.2.1, WolfSsl 5.9.4 |
| bdd-freertos-lwipraw-mbedtls-fatfs-syslogng | GNU 14.2.1 (arm-none-eabi-gcc) | FatFs R0.16, FreeRtos V11.3.3, LwipRaw 2.2.1, MbedTls 3.6.7 |
| bdd-freertos-plustcp-mbedtls-plusfat-syslogng | GNU 14.2.1 (arm-none-eabi-gcc) | CmsisRtos v6.4.0 / v11.3.0, FatFs R0.16, FreeRtos V11.3.3, LittleFs 2.11 (v2.11.3), LwipRaw 2.2.1, LwipSocket 2.2.1, MbedTls 3.6.7, PlusFat 9ca465c, PlusTcp V4.4.1 (V4.4.2), WolfSsl 5.9.4 |
| bdd-linux-openssl-syslogng | GNU 14.2.0 | OpenSsl 3.5.7, WolfSsl 5.9.4 |
| bdd-linux-openssl-syslogng-passthrough | GNU 14.2.0 | OpenSsl 3.5.7, WolfSsl 5.9.4 |
| bdd-linux-wolfssl-syslogng | GNU 14.2.0 | WolfSsl 5.9.4 |
| bdd-windows-openssl-otel | MSVC 19.51.36260.0 | OpenSsl 3.6.3 |
| build-linux-wolfssl | GNU 14.2.0 | WolfSsl 5.9.4 |
| integration-linux-littlefs | GNU 14.2.0 | CmsisRtos v6.4.0 / v11.3.0, FatFs R0.16, FreeRtos V11.3.3, LittleFs 2.11 (v2.11.3), LwipRaw 2.2.1, LwipSocket 2.2.1, MbedTls 3.6.7, OpenSsl 3.5.7, PlusFat 9ca465c, PlusTcp V4.4.1 (V4.4.2), WolfSsl 5.9.4 |
| integration-linux-mbedtls | GNU 14.2.0 | CmsisRtos v6.4.0 / v11.3.0, FatFs R0.16, FreeRtos V11.3.3, LittleFs 2.11 (v2.11.3), LwipRaw 2.2.1, LwipSocket 2.2.1, MbedTls 3.6.7, OpenSsl 3.5.7, PlusFat 9ca465c, PlusTcp V4.4.1 (V4.4.2), WolfSsl 5.9.4 |
| integration-linux-openssl | GNU 14.2.0 | OpenSsl 3.5.7, WolfSsl 5.9.4 |
| integration-linux-wolfssl | GNU 14.2.0 | OpenSsl 3.5.7, WolfSsl 5.9.4 |
| integration-windows-openssl | MSVC 19.51.36260.0 | OpenSsl 3.6.3 |
| unit-linux-clang | Clang 19.1.7 | OpenSsl 3.5.7 |
| unit-linux-gcc | GNU 14.2.0 | CmsisRtos v6.4.0 / v11.3.0, FatFs R0.16, FreeRtos V11.3.3, LittleFs 2.11 (v2.11.3), LwipRaw 2.2.1, LwipSocket 2.2.1, MbedTls 3.6.7, OpenSsl 3.5.7, PlusFat 9ca465c, PlusTcp V4.4.1 (V4.4.2), WolfSsl 5.9.4 |
| unit-linux-gcc-sanitize | GNU 14.2.0 | CmsisRtos v6.4.0 / v11.3.0, FatFs R0.16, FreeRtos V11.3.3, LittleFs 2.11 (v2.11.3), LwipRaw 2.2.1, LwipSocket 2.2.1, MbedTls 3.6.7, OpenSsl 3.5.7, PlusFat 9ca465c, PlusTcp V4.4.1 (V4.4.2), WolfSsl 5.9.4 |
| unit-linux-gcc-tunables | GNU 14.2.0 | OpenSsl 3.5.7, WolfSsl 5.9.4 |
| unit-windows-msvc | MSVC 19.51.36260.0 | OpenSsl 3.6.3 |