Why many effectiveness checks are weak
A weak CAPA closure says “no recurrence observed” two weeks after the fix. Two weeks of a 250 ppm problem may contain no defect even if nothing changed. An effectiveness check is only meaningful if the criterion was set before the action and the volume is large enough to tell a real drop from luck.
Step 1: write the criterion first
- Which metric: rejects at the customer, internal scrap, a measured characteristic, or complaints.
- The baseline, with counts: 12 rejects in 48,000 is 250 ppm, not just “250 ppm”.
- The target and the minimum volume or time before you judge it.
- Who checks it and where the data comes from.
Step 2: choose the volume (rule of three)
If you see zero defects in n parts, the true rate is plausibly below about 3/n at the 95 percent level. So zero in 12,000 parts supports “below about 250 ppm”, the old level. Zero in 30,000 supports “below about 100 ppm”. Fewer parts than 3 / baseline rate cannot show the problem is gone, however clean the data looks.
Seen from the other side: if the rate had stayed at 250 ppm, the chance of 30,000 parts with zero defects is about 0.00055, roughly 6 in 10,000 (Poisson model). Zero in 30,000 is therefore strong evidence that something changed.
Worked timeline
| Week 0 | Complaint: 12 rejects in 48,000 delivered = 250 ppm. Containment starts (100 percent sort). |
| Week 1 | Effectiveness criterion written into the CAPA before the fix: zero rejects in at least 12,000 parts, no recurrence over 3 months, sort data confirms. |
| Week 3 | Root cause verified (switched on and off), permanent action implemented, PFMEA and control plan updated. |
| Week 4 to 8 | 30,000 parts produced; 0 rejects counted before the sort removes them. Sort still running. |
| Week 8 | Criterion met (0 in 30,000, below 100 ppm): containment lifted, date and reason recorded. |
| Month 3 | Customer field data since lifting: 0 complaints; recurrence check and audit confirm the action is in place. CAPA closed. |
Two traps in this example. First, if the 30,000 parts were still being 100 percent sorted, the zero says something about the sort, not the fix. Count defects found by the sort, before it removes them. Second, if the root cause was tied to a tool change every 5,000 parts, the volume must cover several tool changes, not just a count.
Effectiveness checklist
- Criterion, baseline and minimum volume written before implementation.
- Root cause verified, not assumed (see how to write D4).
- Data counted upstream of any containment sort.
- Volume at least 3 / baseline rate, and long enough to cover the triggering conditions.
- Same measurement method before and after, with an adequate gauge (see Gage R&R basics).
- Containment removed only after the criterion is met, with a record of when.
- Read-across to similar parts and lines done (see D7 prevent recurrence).
- Recurrence check scheduled, with an owner.
In an 8D this is D6; in an ISO 9001 CAPA it is the effectiveness review of the corrective action. Same evidence either way: see CAPA vs 8D and the CAPA form template.
FAQ
- What is the difference between an implementation check and an effectiveness check?
- An implementation check confirms the action was done as planned (the poka-yoke is installed, the instruction is released). An effectiveness check confirms the problem stopped and stayed down. A CAPA can be fully implemented and still not effective.
- How many parts do I need to show the problem is gone?
- With zero defects in n parts, the rule of three says the true rate is plausibly below about 3/n at the 95 percent level. To show a rate below the old 250 ppm you need about 12,000 parts with zero defects; to show below 100 ppm, about 30,000.
- How long should I wait before closing a CAPA?
- Long enough to cover the conditions that caused the problem: shifts, tool changes, material batches, seasons. Many companies use 3 months or a defined volume, whichever is later. Common practice, not a rule. Your procedure or customer may set a fixed period.
- Is a passed audit proof of effectiveness?
- No. An audit shows the action is in place. Effectiveness needs data on the outcome: defect rate, complaints, scrap, or the measured characteristic.