Medical Device Verification Protocols: An effective approach to avoid common pitfalls
Verification of a medical device you could say is the ‘certificate’ of the device’s successful function. It proves, sometimes through many means, that the device functions as intended by comparing the design inputs to the design outputs. The verification process itself is by no means simple and it is vital that this process is seamless. This is primarily achieved by efficient planning in the form of an exemplary comprehensive protocol.
Unfortunately, without experience, it is easy to oversee key elements of the protocol and these mistakes can easily snowball into increased project costs (usually due to an increase in expensive testing conducted by third party certified test houses), delayed timescales/project launch or even project cancellation. All of which leads to projects being dragged-out with copious documentation, frustrated clients and flustered engineers.
To avoid these protocol mistakes, certain questions must be answered which address the most critical elements of the verification process.
When should verification of a device be addressed?
Realistically, the actual verification of the device will happen towards the end of the design development when the physical device has been produced. However, it is best to consider critical elements early, even during initial prototyping and testing. Many engineers will put off the design verification planning, hoping that these design requirements will just make themselves known as development is carried out. Do not fall for this illusion, it is a rare occurrence that approaching it this way will lead to everything being captured. If requirements are discovered too late, the design may already be close to completion. Proactively planning verification and conducting preliminary tests early in the design development will highlight requirements and necessary improvements, so these can then be considered during initial reviews when the design is still malleable.
From where are the verification tests derived?
Devices will have to conform to regulatory requirements; In particular the methods for conforming to the MDR 2017/745 General Safety and Performance Requirements. These regulatory requirements are largely met by complying to relevant standards. All medical devices must meet these regulatory requirements to gain a CE mark.
There are two main categories of requirements which feed into verification protocols; those requirements which are driven by standards and those other functional requirements which are not.
Standards
Standards are split into three categories; horizontal, semi-horizontal and product specific or vertical standards; horizontal standards apply to all medical devices, semi-horizontal standards apply to a family of devices (i.e., electronic devices), and vertical standards are product specific. Some engineers will perhaps only locate a few of the necessary standards before starting the protocol development; however, acquiring all the standards related to your device before the protocol writing begins will give you the clearest overall picture of what must be verified.
Other functional requirements
Standards will lay out the absolute primary medical device requirements, however there will almost definitely be other requirements which need to be verified to demonstrate the device’s successful function . These requirements are driven from many sources, and the most efficient ways of verifying against these requirements are discussed in more detail later in this article.
What to consider when translating standards into a protocols?
Translating standards into a protocol/s is not as easy as it may sound, due to many factors. However, below I have gathered from experience, the most helpful tips which will increase your effectiveness when dissecting the standards.
Understating the test rationale
You might ask yourself why reading a standard is so difficult? My advice is that you need to treat the standard as something you must study rather than read. Standards are generally going to be more condensed versions of the protocol you will write, and so when the standard is ‘unravelled’ you have to be prepared to fill in the gaps; with a thorough understanding of the test rationale and the standard, this will come much easier. This thorough understanding will also significantly impact your ability to outline your devices other functional requirements and accompanying tests which are separate to the governing standards.
Dealing with standard ambiguity
Sometimes the standard may not seem ‘black and white’. From experience, if a standard is ambiguous it is best to first reach out to a certified test house or regulatory body (i.e., BSI) before defining the test/protocol. Test houses are certified to test devices to a specific standard/s, so they will have almost definitely have had experience conducting the tests (they may have even had the same question). If after reaching out for advice you are still unsure on the correct route to take – it is best to err on the side of caution, and test ‘worst case’ if you are confident the device will still pass, taking the easy route is common, but will likely result in repeated testing.
Follow the chain
Standards may also reference other standards, and the chain continues. A reference from the core standard to another, might be for something small e.g., a piece of apparatus, but that does not make it any less important, testing with the wrong apparatus could have a knock-on effect on the test and could void the results. Give yourself enough time to navigate through all the standards which are relevant. Missing something as trivial as piece of apparatus, will start to add costs and testing time, as in this example, repeating the test with the correct apparatus would likely be required.
Do not be blind-sided by initial success
Although not related to the development of verification protocol/s, this tip is vital to help you when beginning the verification stage. Do not be blind-sided by one device pass result; just because one device passes the array of tests, it does not mean the hard work is done. Some standards will require for example, 200 devices to be tested, and the results to be evaluated through a statistical analysis to determine whether the device passes as a whole. Even if all 200 devices individually passed, the statistical analysis can essentially rescind the pass. In pre-verification stages, testing multiple devices and running results through the required statistical analysis will tell you whether your device is on-track.
What to consider when translating other functional requirements into protocols?
Sometimes, standards will only provide the core elements of the protocol. There are likely to be many other requirements that require verification, for example, a force activation test of a button; this test may not be critical to be in-line with the relevant standards, but it certainly could have an impact on performance if a user is unable to apply enough force to an activation button. The list of tips below will help guide you around the pitfalls of putting together this section of the verification protocol:
Gathering your tests
The first port of call for figuring out all the necessary tests is writing or reviewing the device’s Product Requirement Specification (PRS). The PRS is home to all of your design inputs, verification of some of these inputs will be covered by the relevant standards, and some by other means (i.e., QA at manufacture). All the design inputs which remain, will in some way require verification; these are the other functional requirements.
Consider what your device must NOT do
Depending on your DFMEA and risk analysis, there might be examples where your device should be able to take a certain amount of miss-use before it functions incorrectly, these will ultimately feedback into the PRS. The amount of acceptable misuse is likely something that will need to be considered at the verification stage.
Be precise in the requirements of the test itself
Each written test protocol should include not only the test procedure but also extra details, such as: the apparatus, required units and precision, acceptance criteria, number of devices, inspections required, statistical analysis of results, pre-conditioning, environment conditions etc. (the list can go on). Understanding what should be laid out for the technician is situational i.e., it is likely unimportant if the humidity of the room is 90%RH for a button activation force test for example. Usually, what is included is left up to the discretion of the protocol author, using the device’s governing standard as a guideline to what should be specified will help you down the right path.
Justification is key
Probably the most poignant point of all when writing this section of a protocol is your justification of each and every test and their allowable pass limits (e.g., a pass criterion of 5N to 10N of force for a button activation test). The limits for these tests are not dictated to you by a standard, so you will need to define them yourself. The most important message here is that the justification of limits requires a strong foundation.
An easy mistake to make here which ultimately leads to delayed timescales, is not thoroughly justifying your limits. If down the line your device fails a test, purely because the limits were too tight; or alternatively your device passes, but in reality, the pass criteria were way too easy, pass criteria limits will need to be adjusted.
If we look at another example of a functional requirement, a cap attachment test; if your device has a cap (or case) then the user will have to some point remove it/replace it, the test associated will likely be a simple force/torque test for attachment/detachment (depending on how simple the interaction is). There will certainly be anthropometrics and ergonomic data surrounding this seemingly trivial interaction. When reviewing such data to outline your pass criteria limits, it is crucial to consider the target population, this means understanding the full range of users which will likely use the device, there are two extremes of this being done incorrectly which you should avoid:
• Considering an unrealistically large range of users where not required, for example in terms of age, as the youngest and oldest members of the population will likely skew your ranges of acceptable limits.
• Considering only a small range of users will give easier pass criteria, however if in this example, the cap was too tight, an arthritic user may have trouble pulling it off.
Making changes to pass criteria and limits will likely require documentation from many parties to approve, things will get messy if numerous tests have unjustified limits which require changing. Clients can lose confidence in engineers quickly when these mistakes occur.
Using the rationale which is used to formulate the governing standard will make it easier to rationalise your own limits.
The above guidelines may at first seem overwhelming, but with good strategy should help you understand the level of depth and preparation that must be dedicated when developing your medical device verification protocol. You must build from the bottom and start strong if you wish to successfully verify a device without hurdles around every corner.
If this topic is something you'd like to discuss further with IDC or have any questions regarding this article please feel free to reach out any time.
About the author
Tom Milnes is a Design Engineer at IDC. Since graduating from the University of Sheffield with a Master's Degree in Mechanical Engineering, he has gained solid experience developing many products and devices across a wide range of industries and sectors including drug delivery.
Listen to our podcast episode discussing this article:
Sign up for our top resources and articles on product development.
