Skip to content
Test Automation Vendors / 263 cases

Cross-case count

Coverage claims in automation cases: how many define the denominator

Research desk, testautomationvendors.com · published 2026-11-12 · data as of 2026-11-12 · 76 records

88
coverage claims
25
name the denominator
39
state a starting value

The dataset holds 88 coverage claims. 25 say in the quoted text what the figure is a share of (test cases, regression scenarios, features or code). 39 state a starting value.

What stands out

  1. 01DeviQA publishes 32 of the 76 cases with a coverage claim; the three vendors with the most publish 52, so a reader is looking at a concentrated set of sources.
  2. 02149 of 326 metrics in these cases state a before value; across all 639 metrics in the dataset the count is 209.

Counts come from verified records. A baseline is the before value a vendor states for a metric; quoted claims carry their grade and baseline flag.

baseline:
The measured value before the work started, against which a later value is compared. source

Claims that name the denominator

25 quotes say what the figure is a share of.

  • “helped automate 20% of app regression tests”before not stated after 20% a1qa a1qa-14 grade C, claim only baseline no method yes
  • “helped cover 95% of tests without any tool limitations”before not stated after 95% a1qa a1qa-22 grade C, claim only baseline no method no
  • “Only 15% of tests were automated”before 15% after 90-95% DeviQA deviqa-2 grade A, verified baseline baseline yes method yes
  • “90-95% of tests are automated”before Only 15% of tests were automated after 90-95% of tests DeviQA deviqa-2 grade A, verified baseline baseline yes method no
  • “20% of the regression tests automated”before No automation testing after 20% of the regression tests DeviQA deviqa-5 grade B, partial baseline yes method no
  • “more than 90% automated coverage of the application”before not stated after over 90% DeviQA deviqa-9 grade A, verified baseline baseline no method yes
  • “Test cases covered more than 90% of the application”before No comprehensive test documentation after more than 90% of the application DeviQA deviqa-15 grade A, verified baseline baseline yes method yes
  • “99% of the application was covered with test cases”before The test documentation was incomplete after 99% of the application DeviQA deviqa-23 grade B, partial baseline yes method no
  • “>90% of test cases are automated”before Only smoke tests were automated on BE after >90% of test cases DeviQA deviqa-27 grade B, partial baseline yes method no
  • “98% of the test cases were automated”before There were no automated tests after 98% of the test cases DeviQA deviqa-30 grade B, partial baseline yes method no

Claims that do not

63 quotes give a figure without its base; the first 10 are shown.

  • “65% of a critical scope automated in 10 weeks”before no automated suite after 65% of the prioritized critical scope a1qa a1qa-3 grade B, partial baseline yes method yes
  • “Reached 66% test automation coverage”before not stated after 66% a1qa a1qa-4 grade B, partial baseline no method yes
  • “85-90% of pre-release regression automated”before not stated after 85-90% a1qa a1qa-5 grade B, partial baseline no method yes
  • “achieved 90% automated test coverage across critical user flows”before not stated after 90% a1qa a1qa-29 grade B, partial baseline no method yes
  • “100% API functionality covered with automated tests”a1qa a1qa-62 grade C, claim only baseline no method yes
  • “75% coverage for UI regression testing and 93% for back-end regression testing”a1qa a1qa-66 grade C, claim only baseline no method yes
  • “4x test coverage boost in 1.5 years”a1qa a1qa-66 grade C, claim only baseline no method yes
  • “34 integrations covered during testing”a1qa a1qa-75 grade C, claim only baseline no method yes
  • “Achieved 100% of the planned mobile automation”before not stated after 100% of planned mobile automation Abstracta abstracta-2 grade B, partial baseline no method no
  • “Reached 100% automation of the targeted web regression suite ahead of schedule”before not stated after 100% of targeted web regression suite Abstracta abstracta-2 grade B, partial baseline no method no

Coverage claims by vendor

Counts of coverage claims and of those with a starting value, per vendor.

VendorCoverage claimsWith a starting value
DeviQA4134
TestFort120
a1qa101
Qualitest41
Zuci Systems40
Codoid30
QAwerk30
ThinkSys31
Abstracta20
Testrig Technologies20
BugRaptors11
QA Madness10
QA Wolf10
TestingXperts11

How the split was made

A claim counts as defining its denominator when the quote contains a phrase such as of test cases, of regression scenarios, of features or of code. The rule is mechanical and reads only the quote.

Questions

Why does the denominator matter?

A coverage figure can be a share of test cases, of manual regression, of features or of code lines, and each gives a different number for the same suite.

How many claims state a starting value?

39 of 88.

Are coverage figures scored?

They count as metrics for outcome verifiability only when baseline and method are both stated.

Where can I see each claim?

In the claim ledger of each case file.

Records used

This post reads 76 records; the newest verifiedOn date among them sets the data-as-of date.

Show record ids

a1qa-3, a1qa-4, a1qa-5, a1qa-14, a1qa-22, a1qa-29, a1qa-62, a1qa-66, a1qa-75, abstracta-2, bugraptors-6, codoid-12, codoid-20, codoid-21, deviqa-1, deviqa-2, deviqa-3, deviqa-5, deviqa-7, deviqa-8, deviqa-9, deviqa-11, deviqa-12, deviqa-13, deviqa-15, deviqa-18, deviqa-19, deviqa-23, deviqa-24, deviqa-27, deviqa-28, deviqa-29, deviqa-30, deviqa-32, deviqa-33, deviqa-35, deviqa-37, deviqa-38, deviqa-40, deviqa-41, deviqa-42, deviqa-43, deviqa-44, deviqa-46, deviqa-47, deviqa-52, qa-madness-2, qa-wolf-1, qawerk-2, qawerk-6, qawerk-8, qualitest-4, qualitest-8, qualitest-10, qualitest-12, testfort-7, testfort-8, testfort-9, testfort-12, testfort-17, testfort-19, testfort-20, testfort-22, testfort-24, testfort-26, testfort-30, testingxperts-3, testrig-1, testrig-11, thinksys-1, thinksys-2, thinksys-5, zuci-systems-3, zuci-systems-4, zuci-systems-5, zuci-systems-8

More from the blog

Scores and grades describe published evidence as of the verifiedOn date on each item. They do not measure service quality. Verify with the vendor before contracting.