After all of these years “under the umbrella” of The Quality System Regulation and ISO 13485, there is still much confusion in the medical device industry differentiating between validation and verification (even in its most unassuming format). Personally, I had to work hands-on with these process tools until the puzzle pieces finally came together. There are many variables and twice as many approaches. From my observations, academicians who vicariously discuss these subjects are commonly confused and spread their misperceptions without regret.
Quite simply, and in a very general sense, verification and validation is a corroborated process of ensuring that a product design meets deliberately planned requirements. Requirements can come from customers, the Federal government, sales and marketing departments, design engineers, competent authorities, quality engineers, manufacturing associates, etc. Let’s break this enigma down to practical simplicity so that we have an overarching understanding of the two terms before moving forward. I realize that there are a lot of definitions and examples to initialize this conversation, but having a basic and calibrated understanding of the QS Regulation and the harmonized 13485 Standard is an important first step for all of us to embrace.
1. Verification confirms that the device properly reflects the requirements specified, ensuring that you built it right.
2. Validation confirms that the device will fulfill its intended use, ensuring that you built the right thing.
Note: It is entirely possible that a device passes prescribed testing when verified, but fails when validated. This can happen when a product is built as per the specifications, but the specifications themselves fail to address the functional needs of the product or process. Validation and verification walk down the road together and even hold hands on occasion, but they are very different in so many ways.
Definitions from 21 CFR, Part 820 (as harmonized with ISO 8402):
Note: FDA adopted the ISO 8402 definitions for validation and verification. These two basic definitions are also consistent with the definitions in ISO 9000.
- Verification means confirmation by examination and provision of objective evidence that specified requirements have been fulfilled.
- Validation means confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use can be consistently fulfilled.
…and now, more specifically:
Process Validation means establishing by objective evidence that a process consistently produces a result or product meeting its predetermined specifications. Process validation ensures that the process consistently produces conforming results and products when it is controlled appropriately. Validation of the process results in establishing and confirming adequate process control methods on the basis of scientifically valid rationale.
Discussion – Production units must be included in the design validation and tested under conditions similar to those that are expected to be experienced in the user environment. Design validation may require coordination with process validation activities, i.e. produce devices for Design validation activities using a validated process foundation and qualified machinery.
Examples of traditional processes that have been considered by FDA to require validation are test methods, welding, injection molding, sanitization, mixing, sterilization, aseptic processing, software-controlled processes, water systems, extrusion, cleaning, lyophilization and sealing.
Design Validation means establishing by objective evidence that device specifications conform to user needs and intended use(s). Design validation ensures that the finished device meets the users’ needs and the requirements for its intended use. Design validation follows successful design verification.
Discussion – Design verification is not a substitute for Design validation. Design validation should be performed under defined operating conditions and on multiple initial production lots or batches. Design validation may also be carried out in earlier stages before initial production. It may be necessary to perform multiple validations if there are different intended uses. Design validation requirements include product software validation and completion of risk analysis.
Examples that have been considered by FDA as Design validation activities are clinical studies, use tests, simulated use tests and clinical search/experience reports.
Design Verification shall confirm that the design output meets the design input requirements.
Discussion – Design verification consists of specific activities performed during the design process that ensure that the defined process is being followed correctly and that the design inputs are met. Typical verification activities are documented inspections, tests, analyses and objective evaluations.
Examples that have been considered by FDA as design verification activities are animal model tests, biocompatibility tests, material and device compatibility tests, functional tests, reliability testing and performance tests. For software, typical verification activities include code reviews, schematic reviews, unit and component tests and integration tests.
A Simple Analogy
Verification and validation are intertwined engineering tools with very important differences. To illustrate the concepts, consider a building (e.g. an office building) design analogy. In a typical scenario, the senior architect establishes the design input requirements and sketches the general appearance and construction of the building, but associates or contractors typically elaborate the details of the various mechanical systems. Verification is the process of checking at each stage whether the output conforms to requirements for that stage. For example: does the air conditioning system deliver the specified cooling capacity to each room? Is the roof rated to withstand so many newtons per square meter of wind loading? Is a fire alarm located within 50 meters of each location in the building?
At the same time, the architect has to keep in mind the broader question of whether the results are consistent with the ultimate user requirements. That’s why architects and engineers sometimes don’t see eye to eye. Does the air conditioning system keep the occupants comfortable throughout the building? Will the roof withstand weather extremes expected at the building site? Can the fire alarm be heard throughout the building? Those broader concerns are the essence of validation.
In the initial stages of design, verification is a key quality assurance technique. As the design effort progresses, verification activities can become progressively more comprehensive and detailed. For example, heat or cooling delivery can be calculated and verified by the air conditioning designer, but the resultant air temperature can only be estimated. Occupant comfort is a function not only of delivered air temperature, but also humidity, heat radiation to or from nearby thermal masses, heat gain or loss through adjacent windows, etc. During the latter design phases, the interaction of these complex factors may be considered during verification of the design.
Validation follows successful verification, and ensures that each requirement for a particular use is fulfilled. Validation of user needs is possible only after the building is built. The air conditioning and fire alarm performance may be validated by testing and inspection, while the strength of the roof will probably be validated by some sort of analysis linked to building codes, which are accepted as meeting the needs of the user-subject to possible confirmation during a subsequent severe storm.
The Manufacturing Factors
Some manufacturers equate production testing with verification. That is not accurate. Whereas verification testing establishes conformance of design output with design input, the goal of production testing is to determine whether the unit being tested has been correctly manufactured. In other words, production testing is designed to efficiently screen out manufacturing process errors and perhaps also to detect early process failures. Typically, a small subset of functional and performance tests accomplish this objective with a high degree of accuracy. Therefore, production testing is rarely comprehensive enough to verify the design. For example, a leakage test may be used during production to ensure that a hermetically-sealed enclosure was properly assembled. However, the leakage test may not be sensitive enough to detect long-term diffusion of gas through the packaging material, e.g. as early as with ethylene oxide sterilization. Permeability of the packaging material is an intrinsic property of the material rather than an assembly issue, and would likely be verified using a more specialized test than is used during production.
Whereas verification is a detailed examination of aspects of a design at various stages in the development, design validation is a cumulative summation of all efforts to assure that the design will conform to user needs and intended use(s), given expected variations in components, materials, manufacturing processes and the use environment.
Definition: Equipment Qualification (as part of Process Validation)
- The Installation Qualification (IQ) is the documented proof that facilities and equipment have been delivered and installed in accordance with the requirements and statutory safety regulations stipulated in the design qualification
- The Operational Qualification (OQ) is a demonstration that the process will produce acceptable results and establishment of the limits (worst case) of the process parameters
- The Performance Qualification (PQ) is the establishment of long-term process stability
The Practice of Process Validation
Planning for validation should begin early in the design process. The performance characteristics that are to be assessed should be identified, and validation methods and acceptance criteria should be established. For complex designs, a schedule of validation activities and organizational or individual responsibilities will facilitate maintaining control over the process. The validation plan should be reviewed for appropriateness, completeness and to ensure that user needs and intended uses are addressed. Validation may expose deficiencies in the original assumptions concerning user needs and intended uses. A formal review process should be used to resolve any such deficiencies. As with verification, the perception of a deficiency might be judged insignificant or inaccurate, or a corrective action may be required to mitigate.
Historically, some manufacturers have used their most competent assembly workers or skilled lab technicians to fabricate test articles (medical devices), but this practice can hide problems in the manufacturing process. Pilot production should simulate as closely as possible the actual manufacturing conditions.
Validation should also address product packaging and labeling. These components of the design may have significant human factor implications, and may affect product performance in unexpected ways. For example, packaging materials have been known to cause electrostatic discharge (ESD) failures in electronic devices. If the unit under test is delivered to the test site in the test engineer’s briefcase, the packaging problem may not become evident until after release to market.
Validation is a compilation of the results of all validation activities. For a complex design, the detailed results may be contained in a variety of separate documents and summarized in a validation report. Supporting information should be explicitly referenced in the validation report and either included as an appendix or available in the design history file.
Now that Product is Produced from a Validated Process
Since the nature of some types of device use or use environments may be particularly challenging or poorly understood, it might be necessary to validate a device under conditions of actual use in a clinical study. These studies should follow the same general guidelines as simulated validation testing. Validation under clinical conditions (clinical evaluation) will involve actual use conditions and should include representative users. The clinical environments that will be used in the evaluation should be representative of the actual use environment, and the validation testing process should affect the clinical environment and use conditions as little as possible. Validation performed under clinical conditions should be preceded by appropriate simulated-use testing to ensure that the device is sufficiently well designed to be safe in actual use (to the degree afforded by simulated-use testing).
Good engineering practice proposes that the overall philosophy of validation is the same whether it is for a device or its associated manufacturing process equipment. Validation answers the question: Have we built the right thing? For a device, this is ultimately achieved by showing that the final device meets the original user needs and intended uses. Process validation represents the same concept and shows that the process equipment meets the original user needs and intended uses. Validation can be shown as it applies to the design and development of a medical device.
While device validation encompasses all activities within the large “Device Validation V,” process validation involves all of the activities within the “Process Validation V,” including process design and production development. Although process design and production development are shown serially to simplify the diagram, production development usually occurs within the activity of process design. For instance, the prototype manufacturing equipment used for verification during process design ultimately becomes the final manufacturing equipment at the end of production development. Before the medical device regulations focused on design control, process validation was often viewed by designers as only being applicable to production development and not process design.
Questions for the Validation Model
The validation model should outline the major stages of verification and validation that should occur during a medical device design and development project. The model can also be used as a tool to help manufacturers audit their existing system of design and development. Here are some questions that should be given focused attention:
- Does your design team understand the difference between verification and validation?
- Do you attempt to capture all of the user needs and all of the intended uses of the device before commencing with the design and development?
- Have you set up a design validation strategy? Have you set up a process validation strategy?
- Have you set up verification requirements for each essential design output?
- Have you verified each design output against a design input? Have you identified the essential outputs as a result of this testing?
- Have you set up the process qualification requirements?
- Have you qualified the final process equipment? Is there a calibration master plan in place?
- Have you proven that the final device meets its initial device requirements (is verification complete and accurate)?
- Have you proven that the final device meets all of the user needs and intended use requirements?
Design verification means demonstrating that all of the device’s design inputs, usually taken directly from the Product Specification, are demonstrated to have been met. This is usually summarized in the design matrix, wherein every design input is matched with a specific document or engineer’s notebook reference that proves that the design output has been proven and deemed essential (or not). The documentation can be in the format of an experiment report, a metrology report, a validation protocol and report, etc. It’s your call as long as the design validation is geared to prove that the device provides for the user needs, when the device is used as intended.
Design verification is not a substitute for design validation. Design validation should be performed under defined operating conditions and on multiple initial production lots or batches. More importantly, production units must be included for the design validation and tested under conditions similar to those that are expected to be experienced in the user environment. Design validation may require coordination with process validation activities, i.e. produce devices for design validation activities using a validated process foundation and qualified machinery. Quite simply and in a very general sense, verification and validation is a corroborated process of ensuring that a product design meets deliberately planned requirements.
John Gagliardi has had success over the past 43 years in the Medical Device and Pharmaceutical industries because of his practical approach to process-orientation and business. He has been actively involved in research and development, quality assurance, training, operations, process architecture, FDA inspections and regulatory affairs. John specializes in building systems in a compliant and business-ready manner. John can be reached by email.
MidWest Process Innovation, LLC