Shifting the Paradigm: Does AQL Testing Answer the Wrong (Not Best) Question?
Sometimes we might be wasting too many resources attempting to answer the wrong question with much accuracy, or to the third decimal place.
In the blogs, "AQL (Acceptable Quality Level) Sampling Can Be Deceptive" and "Example That Highlights Challenges With Acceptable Quality Level (AQL) testing", I described shortcomings of the AQL methodology.
This blog and others to follow will describe an alternative technique that can often be used in lieu of AQL testing. This methodology provides more information with less effort and can lead to activities that result in improved future product quality. In addition, this methodology offers predictive statements that can be used not only in AQL test situations but also as a scorecard methodology thorough-out the enterprise.
With AQL testing, sampling is to provide a decision making process as to whether a lot is satisfactory or not relative to a specification; however, this is often a very difficult, if not impossible, task to accomplish. When one is confronted with the desire to answer a question that is not realistically achievable, we should first step back to determine whether we are attempting to answer the wrong (or at least not the best) question.
Sometimes we might be wasting much resource attempting to answer the wrong question with much accuracy, or to the third decimal place.
When we are examining an AQL sample lot of parts, population statements are being made about each lot. However, in most situations, a lot could be considered a time series sample of parts produced from the process. With this type of thinking, our sampling can lead to a statement about the process that produces the lots of parts. With this approach, our sample size is effectively larger since we would be including data in our decision-making process from previous sampled lots.
Scoping the situation using this frame of reference has another advantage. If a process non-conformance rate is unsatisfactory, the statement is made about the process, not about an individual lot. The customer can then state to its supplier that the process needs to be improved, which can lead to specific actions that result in improved future product quality. This does not typically occur with AQL testing since focus in lot sampling is given to what it would take to get the current lot to pass the test. For this situation, it might end up being, without the customer knowing it, a resample of the same lot. This second sample of the same lot could pass because of the test uncertainty, as described in the previous blog.
When determining an approach for assessing incoming part quality, the analyst needs to address the question of process stability. If a process is not stable, the test methods and confidence statements cannot be interpreted with much precision. Process control charting techniques can be used to determine the stability of a process.
Consider also what actions will be taken when a failure occurs in a particular attribute-sampling plan. Will the failure be "talked away"? Often, no knowledge is obtained about the "good" parts. Are these "good parts" close to "failure"? What direction can be given to fixing the source of failure so that failure will not occur in a customer environment? One should not play games with numbers. Only tests that give useful information for continually improving the manufacturing process should be considered.
Fortunately, however, many problems that are initially defined as attribute tests can be redefined to continuous response output tests. For example, a tester may reject an electronic panel if the electrical resistance of any circuit is below a certain resistance value. In this example, more benefit could be derived from the test if actual resistance values are evaluated. With this information, percent of population projections for failure at the resistance threshold could then be made using probability plotting techniques. After an acceptable level of resistance is established in the process, resistance could then be monitored using control chart techniques for variables. These charts then indicate when the resistance mean and standard deviation are decreasing or increasing with time, an expected indicator of an increase in the percentage builds that are beyond the threshold requirement.
Additionally, design of experiments (DOE) techniques could be used as a guide to manufacture test samples that represent the limits of the process. This test could perhaps yield parts that are more representative of future builds and future process variability. These samples will not be "random" from the process, but this technique can potentially identify future process problems that a random sample from an initial "batch" lot would miss.
With the typical low failure rates of today, AQL sampling is not an effective approach to identify lot defect problems. However, often it can be difficult to convince others that AQL does not add much value and should be replaced by a better process monitoring system.
Reference: The content of this blog was taken from Section 21.12 of IEE Volume III
In the blogs, "AQL (Acceptable Quality Level) Sampling Can Be Deceptive" and "Example That Highlights Challenges With Acceptable Quality Level (AQL) testing", I described shortcomings of the AQL methodology.
This blog and others to follow will describe an alternative technique that can often be used in lieu of AQL testing. This methodology provides more information with less effort and can lead to activities that result in improved future product quality. In addition, this methodology offers predictive statements that can be used not only in AQL test situations but also as a scorecard methodology thorough-out the enterprise.
With AQL testing, sampling is to provide a decision making process as to whether a lot is satisfactory or not relative to a specification; however, this is often a very difficult, if not impossible, task to accomplish. When one is confronted with the desire to answer a question that is not realistically achievable, we should first step back to determine whether we are attempting to answer the wrong (or at least not the best) question.
Sometimes we might be wasting much resource attempting to answer the wrong question with much accuracy, or to the third decimal place.
When we are examining an AQL sample lot of parts, population statements are being made about each lot. However, in most situations, a lot could be considered a time series sample of parts produced from the process. With this type of thinking, our sampling can lead to a statement about the process that produces the lots of parts. With this approach, our sample size is effectively larger since we would be including data in our decision-making process from previous sampled lots.
Scoping the situation using this frame of reference has another advantage. If a process non-conformance rate is unsatisfactory, the statement is made about the process, not about an individual lot. The customer can then state to its supplier that the process needs to be improved, which can lead to specific actions that result in improved future product quality. This does not typically occur with AQL testing since focus in lot sampling is given to what it would take to get the current lot to pass the test. For this situation, it might end up being, without the customer knowing it, a resample of the same lot. This second sample of the same lot could pass because of the test uncertainty, as described in the previous blog.
When determining an approach for assessing incoming part quality, the analyst needs to address the question of process stability. If a process is not stable, the test methods and confidence statements cannot be interpreted with much precision. Process control charting techniques can be used to determine the stability of a process.
Consider also what actions will be taken when a failure occurs in a particular attribute-sampling plan. Will the failure be "talked away"? Often, no knowledge is obtained about the "good" parts. Are these "good parts" close to "failure"? What direction can be given to fixing the source of failure so that failure will not occur in a customer environment? One should not play games with numbers. Only tests that give useful information for continually improving the manufacturing process should be considered.
Fortunately, however, many problems that are initially defined as attribute tests can be redefined to continuous response output tests. For example, a tester may reject an electronic panel if the electrical resistance of any circuit is below a certain resistance value. In this example, more benefit could be derived from the test if actual resistance values are evaluated. With this information, percent of population projections for failure at the resistance threshold could then be made using probability plotting techniques. After an acceptable level of resistance is established in the process, resistance could then be monitored using control chart techniques for variables. These charts then indicate when the resistance mean and standard deviation are decreasing or increasing with time, an expected indicator of an increase in the percentage builds that are beyond the threshold requirement.
Additionally, design of experiments (DOE) techniques could be used as a guide to manufacture test samples that represent the limits of the process. This test could perhaps yield parts that are more representative of future builds and future process variability. These samples will not be "random" from the process, but this technique can potentially identify future process problems that a random sample from an initial "batch" lot would miss.
With the typical low failure rates of today, AQL sampling is not an effective approach to identify lot defect problems. However, often it can be difficult to convince others that AQL does not add much value and should be replaced by a better process monitoring system.
Reference: The content of this blog was taken from Section 21.12 of IEE Volume III
