Alamofire
24of 808 tests proven unable to fail
All 24 are compile-time API checks — a call with no assertion, verifying a static member exists and type-checks. Real, and deliberate.
Which of your tests cannot fail?
A test that passes and a test that checks something are different things, and nothing in your pipeline tells them apart. A green suite containing a test with no assertion, or a snapshot suite left in record mode, reports exactly the same colour as one that works.
We prove a subset of your tests cannot fail.
We do not claim the rest are good.
100% precision on a holdout sealed before it ever ran.
100%
Blocker precision
0
False positives
38
Unseen cases
17/17
Clean cases correct
How the holdout was sealed
Most accuracy numbers in this category are training-set scores: the rules get corrected until the labels agree, and the result is quoted as accuracy. That is a regression gate wearing a measurement's clothes.
So the split was written before any rule had been run against it, and its SHA-256 recorded at that moment. The evaluator refuses to score if the digest no longer matches, and a test asserts the same — editing a label after seeing a result fails the build rather than quietly improving the number. Tuning happened on a separate visible split. The sealed one was opened once.
On 38 unseen cases it claimed proven unable to fail nine times and was right nine times, with no false positives across 17 correct tests. One case it got wrong in the conservative direction — and that has deliberately not been fixed, because changing a rule to satisfy a holdout case is exactly what the seal prevents.
Every rule is decidable from the source.
No model call, no heuristic scoring. The same input gives the same answer on every run, which is what a CI gate needs and what an LLM asked the same question cannot provide.
| Rule | Severity | What a finding proves |
|---|---|---|
| snapshot-recording | blocker | isRecording = true or record: .all is committed. Every snapshot assertion in scope passes unconditionally and rewrites its own reference. The suite is green by construction. |
| no-assertion | blocker | The test reaches no assertion, directly or through any local helper. At runtime it can only fail by crashing or hanging. |
| never-runs | serious | A .disabled trait or an unconditional throw XCTSkip. The test still counts toward a green suite. |
| tautology | serious | XCTAssertTrue(true) and friends — an outcome fixed at compile time, independent of the code under test. |
| unproven | never counted | No assertion found, but a callee is declared elsewhere and might assert. Reported separately and excluded from every count. |
the rule that pays for the tool
// One line in a base class, committed by accident. // Every snapshot test below it passes forever — and rewrites // its own reference, so the regression becomes the baseline. override func setUp() { isRecording = true }
Three well-maintained libraries. Three low numbers.
These are good suites, and the results say so — 3% and 0.2% of tests flagged. Published because a checker that screamed at mature projects would be telling you about itself rather than about the code.
Kingfisher
1of 444 tests proven unable to fail
One concurrency stress test that detects races by crashing rather than asserting. Three advisories for self-comparisons.
swift-snapshot-testing
6of 92 tests proven unable to fail
Its 16 recording findings are all in RecordTests.swift and the test base classes — the library's own machinery for testing record mode. Exactly where the rule should fire.
Four false positives were found this way, by opening every finding against the source that produced it before publishing any of them. Test discovery originally found 74 of Alamofire's 790 tests because it required a literal XCTestCase in the inheritance clause. Conditional guard #available skips were reported as unconditional. record: .failed borrowed .all's wording and overclaimed. And XCTAssertEqual(identity, identity) was called a defect when, with NaN, reflexivity is precisely the property under test. Each now has a named regression test.
What we cannot tell you yet.
We built this because of AI-written tests. We have not measured any.
The motivation was a team describing their AI-generated UI suite collapsing on the third or fourth attempt. But every finding on this page comes from mature, human-written open source. Whether agent-written suites fail this check more often is a claim we have no data for, and will not make until we do.
The labels and the checker share an author
Sealing the holdout removes tuning bias. It cannot remove conception bias — a case neither the rules nor the labels imagined is missed by both. The real-repo results above are the counterweight, since that is code nobody here wrote.
Swift only, single-file resolution
XCTest and Swift Testing. Kotlin is next on the same parser. A helper declared in another file degrades to unproven rather than being guessed at.
A test that can fail may still be a bad test
Whether it would catch a real regression needs mutation testing, which is dynamic and out of scope. That boundary is why the headline counts what was proven rather than scoring what was not.
Run it on your suite. Nothing uploads.
It is a local checker over source you already have — no SDK, no account, no telemetry. If the number surprises you, that is worth a conversation. If it comes back clean, that is worth knowing too, and costs you a single command.
Repository figures measured on 17 September 2026 with lv-test-integrity v0.1 against shallow clones of the public default branches, and reproducible from that commit. They describe those suites on that date and are not a general assessment of the projects, all of which are actively maintained and score well here. The holdout figures come from a 38-case split sealed by SHA-256 before first execution; the corpus was authored in-house, which is stated above as a limit on what it proves.