Implementing and Maintaining PMS & Performance Evaluation under the IVDR – Webinar Replay

James Shearn is Celegence’s Regulatory Affairs Subject Matter Expert for Medical Devices and IVDs. He has worked in the Medical Devices and In Vitro Diagnostic Devices (IVD) sector for 27 years.

Webinar Background:
The EU In Vitro Diagnostic Devices Regulation (IVDR) brings together previously disparate components of the quality system and requires them to interoperate in a highly synchronous manner with Post Market Surveillance (PMS) driving the documentation cycle for each product in a manufacturer’s portfolio.

During our webinar titled “Implementing and Maintaining PMS and Performance Evaluation under the IVDR”, James Shearn, spoke specifically about how the PMS process interoperates with other systems including Performance Evaluation, Risk Management and Post Market Performance Follow-Up to continuously update key elements of the Technical Documentation.

The transcript blog will focus on: 

• Post Market Surveillance & Performance Evaluation under IVDR
• Practical Steps for Success
• Managing Replicated Content in IVDR Deliverables

A full transcript of James Shearn’s presentation is available to download (and to read below) and just press play to watch the clip now. 

Following the IVDR article 78(2), throughout the product lifetime, we’re required to systematically gather, record, and analyze relevant data on the quality, performance, and safety of a device to draw necessary conclusions and to determine and implement and monitor any preventative and corrective actions.

Under article 78, it tells us that the PMS data is used to:

• Update Benefit Risk Determination & Improve Risk Management
• Update Design & Manufacturing Information, IFU & Regulatory Labelling
• Update Performance Evaluation
• Update Summary of Safety & Performance
• Identify need for Preventative, Corrective or Field Safety Corrective Action
• Identify Improvements to Usability, Performance & Safety
• Contribute to PMS of Other Devices
• Detect & Report Trends

Now, for many of you, a lot of these things in here will be very familiar. There are things here that we as an industry have done for a long time. Some of the items in here are also new. For example, updating the benefit risk determination and risk management is something that would normally happen under the ISO 14971 risk management.

Updating design and manufacturing information, labeling, IFUs, etc and identifying improvements to usability, performance, and safety are things that would normally be expected to happen under a current post market surveillance process.

Updating the need for corrective and preventative action and field safety corrective action would be functions of a CAPA process and a vigilance process. Detecting reports and trends are something that should be familiar to people under MDDEV 2.12-1 for many years.

But we also have some new items in here. For example, the performance evaluation is a new deliverable under IVDR, as is the summary of safety and performance. Then we have this specific requirement to contribute to the post market surveillance of the devices. Many companies will have already been doing that within a similar portfolio already, but it is a specific requirement under the regulation.

Moving on, I’d like to talk about PMS in the context of a transition from IVDD to IVDR. As we go through the transition of the technical file, a product from IVDD to IVDR, we will go through several sections.

Firstly, we’ll look at having an essential requirements checklist under Annex 1 of the IVD. That will require a detailed gap analysis to convert to the general safety and performance requirements and a checklist for that under IVDR.

Similarly, the structure of the technical file under IVDD is somewhat different from the structure of the technical documentation under IVDR. Therefore, there will need to be a rest of that to meet the IVDR requirements. In that review of the essential comments in terms of general safety performance requirements, additional technical and scientific data likely needs to be identified as needed to address gaps between the IVDD and the IVDR requirements.

That data is then going to end up in the performance evaluation report, which will become an input into both ANNEX I and ANNEX II under the technical documentation.

Looking into the post-market surveillance requirements. We had post-market surveillance under IVDD. Under IVDR, the post market surveillance process is prescribed in detail. A new process will be required to be developed and put in place in order to meet the requirements under IVDR for ANNEX III. Then in running that process, that will then have an impact upon the performance evaluation report. It will update the performance evaluation report, which we’ve talked about, is required to demonstrate compliance under the general safety performance requirements.

One of the questions we need to ask ourselves as we go through this process is, how good is our existing post-market surveillance and if it’s not at a sufficient level, do you need to run that process as an input to your performance evaluation for the first time? And if that’s the case, that means that all that process has to be in place and has to run in order to provide that input, which is important for scheduling a project.

We have new deliverables under IVDR for post-market surveillance requiring new procedures or updated procedures to create them. So, we need to create the procedures first, run the procedures, and produce those new deliverables.

Post Market Surveillance Procedures

• Post Market Surveillance Plan
• Post Market Performance Follow Up Plan
• Post Market Performance Follow Up Report
• Periodic Safety Update Report
• Summary of Safety & Performance

New deliverables under performance evaluation procedures, which is a new structure of work under the IVDR.

Performance Evaluation Procedures

• Performance Evaluation Plan
• Scientific Validity, Analytical Performance and Clinical Performance Reports
• Performance Evaluation Report

All of these reports will then be fed into a clinical evaluation report.

Things are very different under IVDR, with respect to PMS from what we’ve had previously under IVDD. So many of the processes we have in post-market surveillance under pre-existing IVDR, but they may operate independently on different schedules, and they may have been infrequently performed, and often they’re unlikely to conform to the prescriptive format that’s defined in IVDR.

The example that I like to use of this is, it’s a bit like a silent disco. So, for those of you not familiar, a silent disco is a disco where everybody wears their own set of headphones, everybody dances to their own favorite music. You have a lot of people in the same room, processing in the same room, you know, dancing along, different tunes, different moves, different speed, and their interactions. They will interact occasionally or bump into each other and spill each other’s drinks, but they’re not really working together.

Effective sequencing and coordination are critical to the success of PMS under IVDR:

• All sub-processes must be executed on time and in the correct order
• The quality of output at each stage is critical to the success of the next
• The format of many deliverables is highly prescriptive
• Sections of text are frequently reused between deliverables and must remain consistent

Previously we’ve had our silent disco where everybody’s dancing at different speeds, different tunes. So, we’re going to go from that to something more highly choreographed, where it’s not just the same tune, but everybody has to dance in step, a bit like a ballet.

Let’s look at how these things fit together, and the order of the various deliverables. There are differences depending on the classification of your device. Let’s start with class A and class B devices. We’re going to start with a post-market event procedure, and under that procedure, we’re going to create a post-market surveillance plan. In conjunction with that, there will be a post-market performance follow-up plan, unless it can be deemed appropriate that there is no post-market performance follow-up, in which case a justification must be documented within the performance evaluation report. Under the post-surveillance plan, we will gather a large and diverse set of data. We will have data on serious incidents, field safety corrective actions, non-serious incidents and data on undesirable side effects, trend reporting, feedback and complaints, relevant literature review and searches of databases, etc.

We’re also required to have publicly available information about similar medical devices. Then alongside that coming out the post-market performance follow-up plan, which is the plan that defines the active things that you’ll be doing in the PMS cycle to understand your product better. For example, additional studies and so forth, you have to have a post-market performance follow-up report detailing the output of the last cycle of post-market performance follow-up.

All that information needs to be collated, reviewed, summarized, and then used to update the benefit-risk ratio and risk management. Update the design information, the manufacturing information, and the instructions for use if needed. Notably, this is where we get into the link between the performance evaluation and PMS. It’s used to update the performance evaluation report.

For class A or a Class B product in the cycle, we’re then required to produce a post-market surveillance report containing results and conclusions of the PMS analysis and a rational description of any preventative and corrective actions taken under that cycle. That’s made available to the competent authority and, if applicable the notified body on request.

We have a PMS procedure, PMS plan, and PMPF plan, and then we have the same inputs and outputs as we discussed. And similarly, that information is also used to update the risk-benefit, risk management, design information, IFUs, and update the performance evaluation report when necessary, but at least annually which means that for Class C and D devices, we must do this every year minimal.

For C and D devices, we have a different output. We have a periodic safety update report, which contains the conclusions of the benefit-risk determination, and the main findings of the post-market performance follow-up. It also contains information about the sales volume for the product, the population and usage frequency that is also updated at least annually. It’s also made available to the notified body and the competent authority on request.

There’s an additional document, which is called the Summary of Safety and Performance. This document is sent to the notified body via EUDAMED when available, and it will be published externally. Also, if you are making Class D devices, the periodic safety update report is sent to the notified body via EUDAMED and becomes, that way becomes available to the competent authority as well. So that’s got a lot of details regarding the relationships of these various deliverables.

Now let’s talk more about the practicalities of pulling this together and how you might structure your procedures such that this concept can be made to work as a practical component of your quality system.

Thinking about where these deliverables come from, the PMS and PMPF plans start off as deliverables out of design control in the latter stages of the pre-market product design. Within your PMS plan, you can define the sources and the analysis methodologies that you will use to pull together all this information that’s required to come into your PMS. Your PMPF report will be defined by what’s in your PMS plan.

Gathering this information together can be quite challenging because it often comes from a diverse range of sources and will be owned by different people within your organization. Something that can be helpful to make that manageable is to design a set of reports by function or several functions where the specified information is pulled together into a template. There are guides as to what should be coming into it, allowing that information to be summarized consistently year on year, by the various functions.

In terms of reviewing that information, it is often very helpful to do that via cross-functional review. You can take these reports, they come into that cross-functional review. Potentially, you can summarize relevant information into a set of slides to go along with that. The review will then come together and have a series of actions having reviewed that data in the round. While you’re there because you’ve got the right people in the room, because you’ve got the right data available, you can potentially use it to confirm the benefit-risk for the product. Obviously, that meeting is going to have a set of formal minutes and will be compiled with all the supporting evidence behind it. If you’ve got a class A and B product, that can be your PMS report. Class C and D will have other deliverables. Based on the actions from that report, we can then use that to drive the updates to benefit-risks and information for the performance evaluation report. We can utilize the normal mechanisms that we have for design control, change control and CAPA if needed to do that.

For the class C and D devices, by taking the PMS output, we can combine that with data on sales volume with usage population and usage frequency. Put that into a PSUR template, and the requirements for the PSUR are very well prescribed within the regulation. After reviewing and getting approval, that generates our periodic safety update report.

Moving on from there, we can take that information and the PMS output to put into the summary of the safety and performance template. It is a very prescribed document and takes information from the declaration of conformity, technical documentation, design history file, and quite heavily from the instructions for use. Upon review and approval, that turns into our summary of safety and performance. Then it will be uploaded onto EUDAMED and reviewed by the notified body.

There will be a defined review language for that. Having been accepted by the notified body that then gets translated into all the languages of the member states where you sell the product. Just as an additional item to look at offline, there is an MDCG guidance document, which is 2022-9, which gives a template for the summary of safety and performance. I’d strongly encourage everybody to review that because, in my experience, the notified bodies are quite hot on requiring the summary of safety and performance to be in accordance with that guidance.

All the deliverables need to be made underneath a post-market event schedule to coordinate input and ensure timely outputs.

If you have multiple products, you’re probably going to want to phase that over time over the course of the year, because you don’t want to be doing some of these high-intensity activities like, for example, literature reviews for all your products all at the same time. Among that, you should consider linkages to management reviews. For example, KPIs around the performance of your complaints or PMS process in there. Also, you need to consider linkages to other processes such as change control.

Let’s now look at the details around the performance evaluation component of this. This falls in the middle of that cycle. It’s very much linked to PMS Article 56 for details.

Under performance evaluation, we will have a performance evaluation plan, and then we will have the three pillars. We have scientific validity, which is the association of the analyte, but the clinical condition or physiological state. We have analytical performance, which means the ability of the device to correctly detect or measure a particular analyte. We have clinical to detect or measure a particular analyte correctly l performance, which means the ability of the device to yield results that correlate with the clinical condition or physiological or pathological state. All the 3 report deliverables are there and then go into the performance evaluation report.

On an annual basis, we need to update all those reports and the output will then inform the post-market performance follow-up plan for the following year. Every year we go back looking at the output from this cycle and then deciding what to do in terms of updating the performance evaluation plan and the post-market performance follow-up plan.

There is some additional guidance on this. MDCG 2022-2 is one example. There is also a standard for performing clinical evaluations for ISO 2916:2019 for In Vitro diagnostic devices – clinical performance studies using specimens on human subjects, which is helpful to see.

Let’s look at practical steps for success when implementing PMS and performance evaluation for IVDR.

Planning for Assessment

• Evidence of compliance with the PMS and Performance Evaluation requirements of IVDR is required as part of Stage 1 assessment by a Notified Body (NB)
• For at least 1 major NB it gates application for assessment
• Except for PSUR, a complete set of PMS and Performance Evaluation prescribed outputs are required for Technical Documentation assessment. Instead of PSUR, you can provide a summary of the PMS history for the product, but everything else needs to be in place

Key Takeaways:

• PMS and Performance Evaluation procedures must be in place in order to schedule assessment of a QMS by an NB to IVDR requirements
• Those processes must have run a cycle in order to produce the documentation for Technical Documentation Review
• The components of the PMS must be created, implemented, and run to be able to apply for assessment by the notified body. That’s very important when it comes to scheduling a transition from IVDD to IVDR
• Plan and schedule accordingly

IVDR is (particularly with respect to many of the deliverables we talked about today) very prescriptive in terms of some of these deliverables.

1. Well designed, robust templates are an important tool in IVDR compliance
   • Template titles that match prescribed deliverables (PER, PEP, PSUR, SSP etc.)
   • Section headings that match the content required by the Regulation
   • Laid out in the same order and using the same terminology as the Regulation for ease of audit
   • Content prompts that align with the Regulation and Guidance (e.g., MDCG 2022-9 – SSP)
2. Templates help guide the creation of consistent, complete deliverables that are easy to audit and update

Literature search is a critical component of both post-market surveillance and performance evaluation.  Within this, the updating of the performance evaluation is required annually for Class C or D devices, and you’re required to update your PMS as well.

Can your organization use one set of searches for both synchronizing the cycles in order to avoid repetition? Good definition and documentation of literature search protocols are essential. For example, you might require different search terms to support your scientific validity and the analytical performance of clinical performance. Also, it’s important to note that PMS requires you to review   publicly available information about similar medical devices.

You need to include competitor products appropriately within the search terms, and you can potentially consider using specialist software and external expert resources to conduct literature searches.

Celegence has a technology solution called CAPTIS™ that helps automate the literature review process and reduces the manual effort associated with it.

Under the clinical performance report, you should clearly describe the methods used to demonstrate clinical performance through either clinical performance studies, scientific peer-reviewed literature published experience gained by routine diagnostic testing, or through other sources of clinical performance data.

If we look at the guidance MDCG 2022-2, clinical performance studies conducted under the IVDD should be considered and identified as other sources of clinical performance data per ANNEX XIII 1.2.3

Because they were conducted under the IVDD, they will not conform to the requirements for conducting similar studies under IVDR. In the performance evaluation plan, the clinical performance report, and the performance evaluation report, we should plan and document an assessment of the quality and completeness of the data, identifying any potential gaps. That data should be supported by either literature or data from published experience gained by routine diagnostic testing.

There are many instances where IVDR requires the same content to be within multiple documents.

Examples of Content Replication throughout the PMS & Performance Evaluation Documentation:

You are required to document an intended purpose. It’s not mandated that you have an intended purpose document, but I can highly recommend that you do because it is used everywhere through the IVDR. Within that, you will have a description of the intended purpose that will turn up in the performance evaluation plan, the clinical performance report, and the performance evaluation report. It will then turn up in your instructions for use and your summary of safety and performance.

Examples of Content Reuse throughout the PMS & Performance Evaluation Documentation:

Looking at other examples, we have a summary of scientific validity that’s in the scientific literature report. We have a description of the analytical performance that’s in the analytical performance report, and we have a description of the clinical performance that’s in the clinical performance report. All of those end up in the performance evaluation report, the instructions for use, and the summary of safety and performance.

IVDR requires the same sections of content to appear in multiple deliverables. This presents several challenges for us as manufacturers.

• Version control is usually at document level. So how do we manage it in a ‘section’ of a document where we’re trying to control a paragraph or a page rather than the document.
• Documents are frequently updated even within one PMS cycle. How do we map the reuse of content and ensure that it’s updated in all impacted documents consistently?
• Furthermore, how do we avoid inconsistencies resulting from copy-and-paste errors and other edits when documents are being reviewed by various individuals across the business?

It is possible to be done manually, but it’s very labour intensive and particularly if you’ve got many products that you are dealing with under PMS.

CAPTIS™ facilitates the tracking and automatic update of replicated content across multiple deliverables for IVDR, including PMS and Performance Evaluation. A tool like this can potentially help with this challenge.

• The processes and deliverables for PMS and Performance Evaluation are intimately linked under IVDR
• Quality Management Systems need to be updated and reorganized to produce the required deliverables in a synchronized manner to follow regulations.
• Updated procedures and their outputs are a prerequisite for Notified Body assessment to IVDR. This is a critical factor in project planning for doing IVDD to IVDR transition
• Good, comprehensive templates are a useful tool in IVDR compliance to make very structured deliverables under the regulation
• Control of replicated content across IVDR deliverables is a significant challenge for the creation and maintenance of IVDR Technical Documentation

Celegence can provide you with:

• Experience and resources to help manufacturers transition their QMS
• Templates for key PMS and Performance Evaluation deliverables
• The CAPTIS™ technology solution to aid in PMS Documentation Maintenance, including Systematic Literature Review

Celegence’s comprehensive support for creating and maintaining required documentation can help ensure your organization’s IVDR compliance. Contact us at info@celegence.com, contact us online or read more about Celegence’s IVDR services.