Ensuring Compliance for your IVD’s Performance Evaluation – Webinar Transcript – Part 2

Joseph has 20 years of experience in medical device life cycle management and quality management systems (certified to ISO 13485, ISO 9001 & ISO/IEC17025). His key expertise includes the preparation and maintenance of technical files, product safety and vigilance reporting, clinical evaluations, risk assessment, regulatory audits, CE and notified body opinion submissions. Medical devices industry – Joseph has experience in diabetes management devices, nicotine replacement therapy (NRT) devices and pressurized metered dose inhalers (pMDI). Joseph has worked for small, medium, and large Biotech companies such as Roche, OBG Pharmaceuticals, and Kind consumer Ltd.

Webinar Background:

The new in vitro diagnostics regulation of the European Union, the IVDR 746/2017, will soon be replacing the existing IVD Directive on May 26th, 2022. The regulation is set to bring in many drastic changes to the way the IVD medical devices are regulated by strengthening the old requirements and adding many new ones.

The IVDR changes have strengthened the requirements for Performance Evaluations, which are conducted in the premarket phase. Further, it mandates that the performance of devices be continuously monitored throughout the life cycle of the device. The IVDR requires the continuous data collection, analysis and drafting of a Performance Evaluation Report (PER).

During our webinar titled “Ensuring Compliance for your IVD’s Performance Evaluation” Joseph Richardson Larbi, spoke specifically about his perspective on the essential components of a PER and how to prepare key technical documentation under the EU IVDR and what manufacturers must do to stay compliant.

Why should you read the transcript/watch this video?

• To best understand the performance data requirements under the EU IVDR for IVDs
• To Learn how the requirements could affect your current product lifecycle approach/QMS
• To examine how to apply the industry best practices for systematic literature reviews
• To find out the common pitfalls to avoid when drafting a PER for a smooth notified body or competent authority submission

A full transcript of Joseph’s presentation is available to download (and to read below) and just press play to watch the clip now. 

Clinical Evidence is the data that you get from analyzing the three pillars under a performance evaluation report. So your scientific validity, your analytical performance, and your clinical performance. The third element is not always expected, or it’s not always needed especially for Class A IVDs. The IVDR makes it clear that clinical performance is not expected for those. So if your clinical performance elements are missing from your PER for your class A IVD, that will be fine. That also depends on three other parameters. In order to determine whether you need your clinical performance or not, you have to determine whether your technology or device is established and standardized. Is it an established and standardized test, or is it an established and non-standard test, or is it a novel test? Obviously, if it’s a new technology coming up, then, you have to demonstrate clinical evidence, video, scientific validity, analytical performance, and clinical performance.

The established and standardized test means tests that have clinical guidelines or consensus for the use of the test. So that means it’s established, and this is a standardized test. There is more than one commercial test available. So the technology and the test are out there which is probably approved, already approved that’s out there that people are using. So that means it’s established it’s out there and all international standards or reference materials exist. So there’s an international standard for that particular test. If you take the device which is for testing of diabetes or blood glucose meters, there is the ISO 15:1997 which is an international standard that’s already out there. So if you are coming out with technology, that’s going to be testing for blood glucose and offering a particular technology, and you’re going to be using ISO 15:1997, then immediately it’s established, it’s a standardized test. In that case, your scientific validity, your analytical performance, and clinical data will equal your clinical evidence. So in that case, as long as you can demonstrate competitors and state of the art. Demonstrate your benefit-risk ratios, then in that case, you don’t need to demonstrate clinical performance and clinical performance, just to sort of recap is to demonstrate that your test results correlate to the actual clinical condition. So that is what your clinical performance is doing.

Established and non-standardized tests mean you can have clinical guidelines. There could be more than one commercial test available, but while international standard reference materials may exist, performance results obtained from different IVD medical devices might not be used interchangeably. So there might be something different between one manufacturer and the other. This type of test and technology that they got. You cannot interchange these things in that case, it’s a non-standardized test. Examples of this category include tests for infectious diseases (e.g. Rubella, Hepatitis C), hormones (e.g. estradiol), cardiac markers (e.g. troponin).

Novel Tests – it means new technology, new target population, new application of an established technology or a new intended use. Because you’re going into a different therapeutic area or different intended use, then it will be classed as a novel test. In that case, you have to demonstrate that your clinical performance is up to standard. So in that case, you have to demonstrate that your clinical evidence includes your clinical performance.

Scientific validity is the association of an analyte to a clinical condition or physiological state.

Things that you need to consider under your scientific validity report:

You can break your performance evaluation report intersections, and then you help your section for scientific validity, which will be part of it. When you read your guidance documents through the GHTF, it is your scientific validity.

Data must be a scientific validity report. The same thing for analytical performance and the same thing for clinical performance. It doesn’t mean that there must be three separate documents. You can have them in sections and all in one particular performance evaluation report.

• What disease or condition is the IVD testing for? make sure you clearly state what you are intending to test for, whether it’s COVID, whether it’s diabetes, make sure you state that clearly.
• What analyte is a device trying to detect or measure?
• What is the proof of concept?
• Is scientific validity well established? If it is, how is it well established? make sure you state all of these things.
• What method are you going to use to demonstrate scientific validity of the analyte? Are you going to review competitor’s products and predicated IFU of literature that’s out there to draw similarity and comparison? Are you going to carry out a scientific literature review or are you going to conduct clinical performance studies to get your scientific results, to demonstrate that the particular analyte is rightfully associated with the clinical condition that you’re trying to measure?

Analytical performance is the ability and accuracy of the IVD medical device in detecting the analyte. So now that you’ve drawn your association of the analyte to the clinical condition. How accurate is the medical device in picking up a particular analyte? So you have to demonstrate this.

There might be standards that you have to test to. For example, blood glucose meter, then you’ve got the article 15197 that tells you to test the parameters that you have to test for, and you have to go through that article to demonstrate that will be your analytical performance.

What is the main analytical performance characteristic? E.g. analytical sensitivity, trueness, precision etc.  – Make sure you state those, and you state it clearly.

How and by what method is analytical performance proven. Are you going to use comparison with reference materials, are you testing or verification to harmonized standards? Comparison with predicate devices, or you’re going to conduct a clinical performance study. If you are going to conduct a clinical performance study, then ISO 14155 for clinical trials and practices and notifications to competent authorities or the FDA, whoever you have to follow all those right processes.

Does the use of the IVD depend on calibrators and or control materials? If it does you state what methodological standard is the calibrator or control material calibrated or traceable to. So it must be traceable to national standards as well, if you’re going to use all of these things, they must have uncertainty measurements of uncertainty as well. Make sure all of these things are stated in there to demonstrate your analytical performance.

Clinical performance of the IVD medical device is the ability of the device to test and correct its results to a clinical condition in a target population as intended. Clinical performance is to analyze how the IVD MD will perform in actual clinical conditions in terms of diagnostic specificity, sensitivity, predative value, likelihood ratio or expected values in normal and affected populations.

Make sure whether the clinical performance data is required for the technology or device or the analyte that you are evaluating, make sure you state those. How, and by what method will clinical performance be demonstrated. So are you going to carry out a full blown out clinical performance study? You’re going to review scientific literature, scientific opinions, or routine diagnostic testing, all of these parameters, make sure that whatever route that you use, you demonstrate the method and the clinical data that comes with that particular method, make sure you detail it extensively in your performance evaluation report.

State of the art is not clearly defined in IVDR, or it doesn’t really tell you exactly how State of the art is measured. I take a geographical area that I’m working in. I have a look at the devices that are there. What is the standard practice in that geographical area? Then I benchmark myself or the device to what is done in that particular geographical area. That is a way to demonstrate state of the art because that’s what is currently happening within that geographical area.

So if it’s within Europe and how do you test for black glucose within Europe? So you have a look at the market and what’s going on, key competitors within a particular therapeutic area or clinical condition. And you draw your similarities to it. You consider the benefits, you consider the technology, you consider the risks of competitive reports, competitor devices and you draw conclusions or parallels to what’s happening on the market. This is how I prove state of the art.

State of the art in medicine is to determine what the clinical practice or standard of care is in a particular geographical area with regards to a particular IVD diagnostic assay, and to determine how the IVD under evaluation performs against such SOTA standards in that particular clinical practice or standard of care.

How or by which method will the state of the art in medicine be determined? Are you going to review the IFUs of competitors’ products, scientific literature searches, or are you going to go through the health authority database for your market surveillance reports? In the US, you’ve got the TPLC, you’ve got the FDA MAUDE or in the UK you’ve got the MHRA.

You can go to websites of these health authorities and pull-out reports. You can have a look at post market surveillance reports that have been submitted to competent authorities, and then you’d be able to then draw comparisons to these devices.

For your scientific literature search, you can conduct scientific literature searches or reviews. If you are going to do those, then you need a good tool that will assist you in carrying out your scientific literature search. Good reporting, you must be able to report and report it effectively.

For a good tool to demonstrate that you’ve carried out your scientific literature search, at Celegence we’ve got a groundbreaking technology that we use at the moment called CAPTIS and that saves us up about 20% of our writer’s time. So you can get in touch with Celegence agents, for example, to demonstrate this new platform that we’ve got, and there’s a 30-day free trial. Go on to Celegence website, get in touch with us to arrange for a free demonstration. We will demonstrate how this new technology can save you time in carrying out a scientific literature review of your medical devices, irrespective of the therapeutic area.

For your reports, you have to make sure that you detail things properly under your scientific literature search review. The name of the device, the model, scope of the search that you are conducting, the period that you are covering, make it clear, the period that you’re covering, whether it’s 5 years, 6 years, 10 years, make sure you state those periods properly and clearly. The literature sources or the databases that you are using Google Scholar and the rest, make sure you state it clearly. The database search details, the search terms, the keywords that you are using, the selection criteria used to select and choose your articles, make sure you state those clearly. The output of your report, conclusions, suitability or how did you include certain articles? How did you exclude certain articles? the criteria and your search terms and everything, make sure everything is detailed properly in your report.

The notified body, when they’re looking at your scientific literature search report, they can say, yes, you’ve considered all the key words. The key terms you’ve looked into, you’ve used the right databases, your inclusion, exclusion criteria. Our platform CAPTIS will give you ways that you’ve never worked before. It can finalize reports that you can easily drop into your final word document or PDF straight away. It will format everything in a table for you. So it makes it easier when you’re using this.

You have to state clearly what the intended clinical benefit is. So what is the benefit to the user stated clearly? Identify the relevant GSPRs, at least GSPR 1- 9.4 if they are applicable for your particular device, make sure you state those and you state it clearly with clinical evidence. Clinical data that is relevant, and you have to make sure that it’s objectively right. Make sure there are no two ways about it. You have to make sure that you are demonstrating compliance to the GSPRs. So go through those and make sure that you’ve got the clinical data to demonstrate compliance.

Analyze ALL risks associated with the use of the device. So you have to make sure that all risk associated with the use of the device has been evaluated. Evaluate all your risk measures, make sure that you follow the ISO 14971 standard, the latest of it 2019. You make sure that all risks, harm parameters severity is recorded, controls before and after and analyze that risk. The final point to draw your benefit-risk ratio, use either qualitative or quantitative methods to demonstrate the benefit-risk ratio. If you’re using the scoring system, make it clear what your criteria is, how you are scoring, and then come up with scoring to prove that the benefit outweighs the risk. If you go in qualitatively, make sure that you say that clearly as well for the user.

• State the intended use of the IVD medical device
• Justifying the approach used to demonstrate clinical evidence and for the clinical evidence, we are talking about 3 parameters, mainly – scientific validity of the analyte, analytical performance, and clinical performance. Make sure for each of those, you state clearly what your approach is, how you are demonstrating your clinical evidence.
• How the data evaluated is sufficient and of significant quality. So if you have to score it, you have to wait for it, make sure you state all of these things properly and the principles of MEDDEV 271 do apply here. So your stage is 0 to stage 4. You have to plan it properly. You have to identify, analyze, appraise, and then analyze your data and draft the report.
• State why the device is safe and effective to use and how the clinical benefits outweigh any risk posed to users of the device. State of the art, make sure that you demonstrate that the clinical benefits that a patient or healthcare professional will get from the device outweighs any risk posed. So make it clear your indications of risk elements and benefit-risk ratios and draw proper conclusions to it.
• Add any actions or observations that require further investigation via other systems such as risk management file/system, corrective action, and preventive actions (CAPA) etc. you have to state all of those and then put your quality management system to good use, demonstrate that you are using an effective quality management system where there’ll be outputs from your performance evaluation.

• Planning (MEDDEV Step 0 – 4)
• Good SLR documentation
• Good reporting of the PER
• Clear Conclusions
• Risk Management, PMS and PMPF or justification

Q: What are the similarities or how does a performance evaluation differ from a clinical evaluation of a medical device?

A: Clinical evaluations are for medical devices under MDR. PER is for IVD’s under the IVDR. Both are trying to demonstrate that they are clinical data as sufficient and compliant, using the data to really demonstrate that they do meet the GSPRs. They’ve got good benefit-risk ratios and the clinical benefits are there when used as intended by the manufacturer.

When it comes to differences, the PER has got more specific elements of its clinical evidence. So you’re talking about scientific validity, the analytical performance and the clinical performance. These are quite specific to IVDs, and you don’t usually get this kind of detail with clinical evaluation reports or with medical devices. It’s your intended user, the benefits demonstrate the clinical data or clinical performance of your medical device and then prove that you are meeting GSPRs, and you’ve got a good benefit-risk ratio.

Q: what is the frequency to update or produce a performance evaluation report?

A: It is risk classification based. For class Cs and Ds of an IVD that is annually and for class A & Bs, you can do that every two years or as in, when you are notified by the request for one or the competent authority request for one. In general, it’s two years class A and B, and every year for class C & D.

Q: Is a PER required for a scientific opinion by the expert panel of a high risk IVD device?

A: In the IVDR it is stated for your high risk IVDs class Ds, and I think the equivalent for a medical device you need to seek scientific opinion. So this will be the manufacturer submitting the documentation to the notified body and the notified body is the one that will seek that scientific opinion from the expert panel on the PER. This is stated in article 48 of the IVDR, which then refers you to article 106 of the MDR. Yes, basically for your scientific opinion you will need your PER. So your PER amongst your total documentation is what the notified body will submit to the expert panel to get the scientific opinion.

Celegence’s comprehensive support for creating and maintaining required PMPF documentation can help ensure your organization’s IVDR compliance. Reach out to us at info@celegence.com to learn more, contact us online or read more about Celegence’s medical device services.