Process validation is becoming ever more important within regulated and non-regulated industries. The FDA defines Process Validation as a means established by objective evidence, a process that consistently produces a result, or product meeting its predetermined specifications. The Global Harmonization Task Force (GHTF) defines process validation as a term used in the medical device industry to indicate that a process has been put to such scrutiny that the result of the process (a product, a service or other outcome) can be practically guaranteed.

One must also be aware of the definition of verification. The FDA defines verification as a confirmation, by examination and provision of objective evidence, that specified requirements have been fulfilled. Because verification cannot be performed on every critical to quality characteristic (CTQ), validation is essential for cases in which the predetermined requirements of the product can only be assured by destructive testing (sterilized products, tensile strength of a forged part, etc.)

Regulatory and certification bodies have requirements for the validation of processes. 21 CFR Part 820.75 (a) Process validation states, ”Where the results of a process cannot be fully verified by subsequent inspection and test, the process shall be validated with a high degree of assurance and approved according to established procedures.” Additionally, ISO 9001:2008 7.5.2 Validation of processes for production and service provision states ”The organization shall validate any processes for production and service provision where the resulting output cannot be verified by subsequent monitoring or measurement and, as a consequence, deficiencies become apparent only after the product is in use or the service has been delivered.”

Process Validation is comprised of three interrelated steps: Installation Qualification (IQ), Operational Qualification (OQ), and Performance Qualification (PQ). Installation Qualification verifies that the equipment, and its ancillary systems or sub-systems have been installed in accordance with installation drawings and or specifications. Additional information is usually generated at this time including generation of a recommended spare parts list and maintenance schedule. Without proper installation of the equipment and supporting systems, operational and performance qualifications will not yield accurate information which can be used to document consistent and predictable performance. Operational Qualification is defined as establishing confidence that process equipment and sub-systems are capable of consistently operating within established limits and tolerances. Performance Qualification provides documented evidence that the integrated system or process is capable of consistently producing the intended product in a high quality and safe manner. The PQ documents and provides objective evidence that a process consistently operates with in predefined acceptance criteria.

According to the FDA website from Inspections conducted from October 1st, 2009 – September 30th, 2010, 174,483 Observations were issued related to process validation activities. Process validation observations accounted for 21% of the observations issued by the Agency. Although process validation is not the top observation category, it is very significant and should be addressed to ensure compliance. In fact, process validation activities are some of the easiest requirements that can be addressed to ensure compliance.

Prior to performing process validations, it is important to have well defined and documented specifications and acceptance criteria. Additionally, a well documented engineering study (design of experiments) should be conducted to find the operating window that defines the combination of parameters which when utilized, will consistently produce products within the allowed specifications.

When performing process validations the question is frequently asked “How many do I need?” The answer is- it depends. Depends on what? There are several factors that must be considered when determining the appropriate sample size including risks associated with the product, costs associated with producing the product, and costs associated with inspection, measuring, and testing. We will first consider the risks associated with the product.

For the sake of discussion, we will use the following generic definitions of risk in relation to patient safety:

The Bayes Success-Run Theorem (based on the binomial distribution) is one useful method that can be used to determine an appropriate risk-based sample size for process validations. The Bayes Success-Run Theorem is as follows:

R = (1-C) ^ (1/n)
where: R = Reliability (or probability of success)
C = confidence level
n = sample size for "0" failures allowed on test
Transposed the formula becomes n= ln(1-C)/ln(R)

For example, if we want to be 95% confident that a process is 95% reliable how many parts do we need to produce that are defect-free?

n= ln (1-.95)/ln(.95) = 59 parts with "0" failures allowed on test (note: always round up to the next whole number)

Now the question becomes: what confidence and reliability level should be used to determine the appropriate sample size? Again, the answer is "it depends." It depends on the risks associated with the product.

To put this in perspective let’s assume the product is a medical device. The following can be used as a guideline to establish confidence and reliability levels based on patient risk.

To determine a confidence statement around the true defect rate and the probability of finding one or more defects in the sample, please refer to the November 2007 Quality Progress article, “Zero Defect Sampling” by Tony Gojanovic.

For the case above with 59 samples (95% confident that a process is 95% reliable) we can state thqt we are 95% confident the true defect rate is between 0 and 5%. Additionally, iwe have a 95% probability of finding one or more defects in the sample (299 samples, the true defect rate, is between 0 and 1%, and with 29 samples, the true defect rate is between 0 and 5%).

Of course, different confidence and reliability levels can and should be utilized based upon the organizations' risk acceptance and determination threshold, industry practice, guidance documents and regulatory requirements.

The author believes this method is the most appropriate for the Operational Qualification (OQ) phase of process validation. Remember that parts produced during an operational qualification may not be used in a finished medical device. When parts are destroyed during the testing process, or when they are expensive to produce, a smaller sample size may be used, as long as appropriate statistical justification and rationale are documented.

According to the FDA’s Current Good Manufacturing Practices (CGMP) Revision 1 issued January 2011, “Samples must represent the batch under analysis and the sampling plan must result in statistical confidence." Additionally, the sampling plan, including sampling points, number of samples, and the frequency of sampling for each unit operation and attribute, should be defined prior to execution of the validation protocol. The number of samples should be adequate to provide sufficient statistical confidence of quality both within a batch and between batches.

The confidence level selected can be based on risk analysis as it relates to the particular attribute under examination. Sampling during this stage should be more extensive than is typical during routine production.

The second method to determine the sample size is acceptance sampling methodologies. With acceptance sampling an acceptable quality level (AQL) must be selected. This method is most appropriate for the Performance Qualification phase of process validation. Again the AQL should be selected based upon product risk. To put this in perspective let’s assume the product is a medical device. The following can be used as a guideline to establish confidence and reliability levels based on patient risk using the Zero Acceptance Number Sampling Plans developed by Nicholas Squeglia.
 

Using this method, the number of items to sample is dependent upon the size of the lot produced. For example, if the lot consists of 5000 pieces, 192 pieces would need to be inspected without a defect for high risk parts, 68 pieces would need to be inspected without a defect for medium risk parts, and 38 pieces would need to be inspected without a defect for low risk parts.

Of course different AQL levels can and should be utilized based upon the organizations risk acceptance determination threshold, industry practice, guidance documents regulatory requirements. The author believes this method is the most appropriate for the Performance Qualification (PQ) phase of process validation. Remember the lots produced during Performance Qualification should be similar the expected lot sized used for production.

It also isgood practice to consider the process yield when getting the ‘magical’ number of good parts needed to conduct the process validation. In our litigious society, sorting to compliance is an unacceptable and potentially costly practice. As the hidden costs of scrap and rework climb, the process quickly becomes economically unfeasible. If the process is producing an unacceptably high defect rate, process improvements should be studied and deployed prior to the process being validated.

The last question regarding “How many do I need?” revolves around the number of discrete lots necessary to demonstrate the process is validated within the stated ranges. Again the answer is it depends. Let us assume that production engineering has determined there are three critical factors that drive the process; time, temperature, and pressure. Hopefully a designed experiment (DOE) was conducted to find the low and high limits of the process parameters as well as a determination of process factor interactions. By performing “replicate cycles (runs) represent the required operational range of the equipment to demonstrate that the processes have been operated within the prescribed parameters for the process and that the output or product consistently meets predetermined specifications for quality and function”. Although not specifically defined by regulations, generally three lots are required to demonstrate the process and produce product consistently which meets predetermined specifications for quality and function.

Remember, the sole purpose of process validation is to ultimately demonstrate with a high degree of assurance the process can produce products that can be consistently manufactured meeting predetermined specifications within stated parameters (settings).

In summary, one must consider costs when determining the sample size (costs associated with producing the product and costs associated with inspection, measuring, and testing). This is especially important when producing complex low volume products or products that are destroyed during testing. No matter what sample size is ultimately used, properly document the justification for selecting the selected sample size. The justification could reference standard operating procedures (SOP’s), industry standards and guidance documents. So the next time someone asks, “How many do I need?” you can confidently say, “It depends!”

BIBLIOGRAPHY

Dovich, Robert A., Reliability Statistics, Milwaukee: Quality Press, 1990

Gojanovic, Tony, “Zero Defect Sampling”, Quality Progress, November 2007: 70

ISO 9001:2008

21 CFR Part 820

Squeglia, Nicholas L., Zero Acceptance Number Sampling Plans, 5th Ed, Milwaukee:, Quality Press, 2008