The Direct Answer: “Enough” Is a Residual-Risk Decision, Not a Universal Percentage

Production test coverage is sufficient when the product’s credible manufacturing defects and selected production-critical behaviors have explicit detection or evidence paths, the remaining blind spots are understood, and the residual risk is acceptable for the product and application.

That is deliberately different from saying that 90%, 95%, 98%, or any other number is “enough.” Without a defined denominator, a percentage cannot tell an OEM what has actually been proven.

Why “95% Test Coverage” Does Not Tell an OEM Enough

A coverage percentage is a ratio. Before an engineering team treats that ratio as evidence, it has to ask: coverage of what?

Depending on the tool or organization, the denominator may be based on testable nodes, components, interconnect opportunities, structural fault categories, or functional requirements. Coverage-analysis tools such as Keysight’s Test Coverage Consultant model coverage against defined fault categories and test strategies rather than treating “coverage” as one universal property of a board. The engineering lesson is straightforward: the denominator controls the interpretation.

For example, a program may have excellent structural coverage of opens, shorts, and component-value faults while providing limited evidence about an at-speed interface, a firmware/configuration mismatch, a load-dependent function, or a product behavior that only appears during final system operation. The structural number may be accurate within its defined model. It is simply not the whole product-release argument.

Start With the Failure Universe, Not the Test Equipment

A common planning mistake is to start with equipment: “We have AOI, flying probe, ICT and FCT - what coverage does that give us?” A more useful sequence starts with the product.

Identify the failure modes and required behaviors that matter for the released configuration, then ask which station can generate convincing evidence for each one. The list will vary by product, but it commonly spans multiple categories:

• assembly and workmanship defects such as missing, wrong, reversed or misaligned parts;
• opens, shorts and interconnect faults;
• component-value or basic parametric errors;
• programming, firmware, identity or configuration errors;
• interface and subsystem behavior that requires active stimulus;
• product-specific safety or end-of-line checks;
• behaviors that belong to validation or qualification rather than production test.

This last distinction matters. Production test is not qualification. A production station can verify selected unit-level attributes against released criteria. It does not, by itself, prove environmental durability, lifecycle performance, EMC behavior, vibration resistance, humidity tolerance, or every field operating condition. Those are separate bodies of evidence.

Different Test Methods Produce Different Evidence

There is no single production-test technology that proves every relevant condition. Visual and automated inspection, structural electrical test, boundary scan, programming verification, functional test and end-of-line test answer different questions. IEEE 1149.1 boundary scan, for example, was developed in part to provide standardized test access to board-level interconnects when physical probing becomes difficult.

The right question is not “Which method is best?” It is “What does this method prove that the rest of the strategy does not?”

Indic's existing technical articles describe its production test and traceability strategy, in-house test engineering, and DFT and test-point planning. This article addresses a different question: how an OEM should judge the combined evidence before release.

When More Test Coverage Adds Very Little

More access, more probes, more stations and more test steps do not automatically create proportionate risk reduction. iNEMI’s Test Strategy Project examined the relationship between physical access, fault coverage and economics, and reported that some ICT probe-removal approaches can reduce probe count significantly with only a small reduction in defect coverage.

The broader engineering lesson is not “remove probes.” It is that test access and useful coverage are not the same thing.

Two test stages may both detect the same short. A downstream functional station may also fail because of that short, but its failure message may be “unit does not boot,” which can be much less useful for isolation. The additional station still has value if it proves important behavior, but counting the same structural fault twice does not necessarily double the protection against field escapes.

Before adding another test step, ask:

• Which new failure class does it detect?
• Does it improve fault isolation or debug speed?
• Does it provide evidence for a product behavior that was previously untested?
• Does it improve process feedback early enough to prevent repeated defects?
• What cycle-time, fixture, maintenance and change-control burden does it add?

The Missing Question: What Can Still Escape?

A test strategy becomes release-ready when its blind spots are explicit rather than accidental.

Suppose two products both report 95% coverage. Product A has extensive coverage of low-consequence structural faults but leaves one field-critical configuration condition unverified. Product B has a slightly lower headline percentage, but every high-consequence failure mode has a deliberate detection or evidence path and the remaining gaps are low consequence and consciously accepted.

The second strategy may be the better release decision even though its headline percentage is lower.

Questions OEM Teams Should Ask Before Approving Test Coverage

1. What is the denominator? Which faults, nodes, components, requirements or functions are included in the stated coverage?
2. What important failures are outside that denominator?
3. Which test stage uniquely detects or isolates each high-consequence failure mode?
4. Where are we paying for duplicate evidence, and is the duplication intentional?
5. Which blind spots are accepted, and who owns that acceptance?
6. Does the actual released test program match the product, firmware, fixture and limit revisions being shipped?
7. Are we confusing production test with qualification? Field reliability may depend on environmental, safety or lifecycle evidence that belongs outside the manufacturing station.

What “Enough” Actually Means

Enough test coverage does not mean every imaginable failure has been reproduced on the production line. It means the team has built a defensible chain from credible product/manufacturing risks to observable evidence, made the blind spots visible, and consciously decided what to do about the residual risk.

That is a stronger release argument than an arbitrary threshold because it remains useful as the product changes. A layout revision can alter access. A component alternate can change what needs to be measured. Firmware can change the functional evidence required. A new fixture or limit file can change the achieved test result without changing the underlying design. The coverage review should therefore be treated as a controlled engineering artifact, not a one-time percentage.

For OEM teams evaluating an EMS partner, this is also a useful discriminator. The more revealing question is not simply, “What test equipment do you have?” It is, “How do you show what each test proves, what remains untested, and how that evidence stays aligned with product revisions?”

Technical Sources and Further Reading

• iNEMI Test Strategy Project - relationship among test access, fault coverage and test-strategy economics.
• Keysight Test Coverage Consultant - example of test coverage being modeled against defined fault categories and strategies.
• IEEE 1149.1 Working Group - boundary-scan architecture and standardized test access for assembled-board interconnects and device functions.
• iNEMI Functional Test Coverage Assessment - methodology motivation for comparing functional-test coverage consistently.