Changelog
Every correction, regrade, removed case and recomputation is logged here with its date and the record ids it touched. The methodology lock dates from 2026-11-11.
- 2026-11-12 regrade
Owner-approved reading applied to all vendors: the stated duration of the whole engagement counts as the period for metrics reporting that engagement's outcome (data/supplements-method/, 78 cases, 201 metrics). Cases regraded and scores recomputed.
- 2026-11-11 methodology lock
Methodology version 1.0 hashed into config/methodology.lock before collection.
- 2026-11-11 verification
First verification run: 20 collected cases removed (page gone, development rather than testing, or no listed framework named on the page); figures dropped from 5 cases for same-page contradictions.
Records: qasource-1, qasource-2, qasource-3, qa-wolf-3, qa-wolf-4, qa-wolf-5, kualitatem-1, kualitatem-2, kualitatem-3, betterqa-2, betterqa-3, qa-madness-3 and 13 more
- 2026-11-11 recomputation
Scores computed for 17 gate-passing vendors; 12 ranked.
- 2026-11-11 recollection
12-case collection cap lifted for every vendor that hit it (deviqa, a1qa, codoid, testfort); framework tags re-verified against case content with site menus removed; scores recomputed.
- 2026-11-11 regrade
Before/after column blocks (Without DeviQA / With DeviQA, BugRaptors Before vs After) re-read in 46 cases: paired rows added as metrics with a stated baseline (data/supplements/), duplicates replaced; cases regraded and scores recomputed.
Related pages
Scores and grades describe published evidence as of the verifiedOn date on each item. They do not measure service quality. Verify with the vendor before contracting.