Test Flakiness

Loading data...

Tests per day, by state

Share of tests, per day

Each test is classified once per day, over the runs it had that day: In the charts above, failing beats being skipped, so a test that failed on Linux and is disabled on Windows counts once, as flaky. The table counts it in both columns — “how much is skipped here” is a question about skipping, and answering it with only the tests that never failed hides most of them. So the table’s Flaky and Skipped columns overlap and do not sum to Tests. Tests annotated run-if are explicitly scoped to another platform, so not running elsewhere is not a skip. A test that did not run at all on a given day is left out of that day entirely. Opening a folder lists only its flaky and skipped tests — the ones that passed everywhere are counted in the percentages but not listed, since there is nothing to do about them.