Helpful Guidelines for Producing Your Systematic Reviews

A Systematic Review (SR) is a synthesis of evidence that is identified and critically appraised to understand a specific topic. While SR covers a wide range of issues, its primary application lies in studying the safety/performance and effectiveness of drug treatments, medical device, or alternative treatments.

This process is paramount for establishing conformity with the General Safety and Performance Requirements (GSPRs) of the EU Medical Device Regulation (MDR) 2017/745. New requirements of the EU MDR mandate a stricter requirement for clinical data and a continuous process of clinical evaluation. It is necessary to restructure the manufacturer’s post-market surveillance (PMS) strategy to align with the new regulations, which include post market surveillance reports (PMSRs) for class I devices and periodic safety update report (PSURs) for Class IIa, IIb and Class III devices.

Systematic reviews, as outlined in EU MDR 2017/745, play a crucial role in the following five sections:

1. Demonstration of Equivalence: Establishing that changes in components or materials do not impact safety profiles.
2. Post-Market Surveillance (PMS) and Post-Marketing Clinical Follow-up (PMCF): Adhering to the new regulations with PMS reports for Class I devices and PSURs for Class IIa, IIb, and Class III devices.
3. Investigator’s Brochure and Clinical Investigation Plan: Incorporating a background literature review to identify baseline clinical studies justifying study type and objectives.
4. Current State of the Art (SOTA): Utilizing systematic reviews to understand the current landscape in medical device technologies.
5. Establishing Acceptance Criteria: Deriving criteria from SOTA and industry standards to determine the benefit-risk profile’s acceptability.

For legacy devices lacking pre-market clinical investigations, reliance on PMS data, complaints (manufacturer held) and published scientific literature (external PMS data) to establish conformity to the regulations is common. However, under the EU MDR, the biggest challenge lies in determining the amount of data needed to generate sufficient clinical evidence. When the amount of data obtained from the above sources have been deemed insufficient, the next step is to produce a robust PMCF plan to address any gaps in clinical evidence.

To ensure compliance to GSPRs, it is essential to verify that all intended purposes of the device under evaluation have sufficient support. For legacy devices, this data usually comes from published literature. Meticulous sorting and categorization of data from published literature (during literature review), noting the amount of data available for each indication along with the quality of identified data is recommended to prevent last minute scrambling for data or updates to the IFUs.

For instance, the presence of several Randomized Controlled Trials (RCTs) or meta-analyses indicates (possibly) sufficient data, but challenges arise when your search results in only a large number of case reports and a lack of high-quality clinical data for a specific intended purpose.

To navigate this challenge effectively, a strategic plan must be formulated early in the process, rather than as a last-minute effort. This plan could include:

1. Refining the Search:
   • Focusing on specific indications for which data is lacking.
   • Conducting a targeted search to fill gaps in data related to those specific indications.
2. Planning a PMCF:
   • Developing a comprehensive PMCF plan to gather additional data over time.
   • Ensuring the plan aligns with the regulatory requirements and addresses the identified data gaps.
3. Early Strategic Decision-Making:
   • Making informed decisions on addressing data gaps at the early stages of the systematic review.
   • Avoiding last-minute challenges by incorporating data gap management into the overall review strategy.

The success of this proactive approach hinges on the meticulous sorting and categorization of data during the literature review phase. By categorizing the available clinical data based on the type of studies, such as RCTs, meta-analyses, or case reports, manufacturers can estimate the adequacy of data for each intended purpose.

Here are key considerations and practices to adhere to regulatory requirements for Systematic Reviews:

1. Documentation of Methodology: The methodology employed in the literature review, including details such as search engine used, results obtained, filters applied, and the specified date range, must be clearly documented. This documentation serves as a blueprint for reproducibility and validation.
2. Trained and Qualified Individuals: Literature searches should be conducted by individuals who are adequately trained and possess the necessary qualifications. This ensures a high level of competence in executing the search strategy and evaluating the gathered data.
3. Sound Methodology Aligned with Research Questions: The literature review methodology should be underpinned by a sound approach based on well-defined research questions. This ensures that the search is purposeful and directly aligned with the objectives of the clinical evaluation.
4. Utilization of Established Methodologies: Search terms and strings should be crafted using established methodologies, such as the Patient/Population, Intervention, Comparison, Outcome (PICO) framework. Alternatively, other validated methods can be employed, ensuring a systematic and structured approach to information retrieval.
5. Documentation in Search Plan or Protocol: The methodology employed in the literature review should be formally documented in the search plan or protocol. This comprehensive document serves as a reference point for reviewers, auditors, and regulatory authorities, allowing for transparency and traceability.
6. Databases for Systematic Searches: Leveraging reputable databases is crucial for comprehensive and thorough literature searches. Some databases commonly used for systematic searches in clinical evaluations include:
   • Cochrane
   • PubMed/Medline
   • EMBASE
   • Web of Science
   • Science Direct
   • Google Scholar

Conducting systematic literature reviews for medical device clinical evaluations presents various challenges, including inability to document a reproducible search strategy, defining and applying crisp inclusion and exclusion criteria, developing in-article appraisal strategies, ensuring all outcomes/ intended uses are covered in the search, ensuring presence of sufficient high-quality data for intended use – including correct population, correct anatomical location, correct methodology.

Handling large datasets often leads to errors too. To avoid this, replacing manual work with smart automation is the answer. A system that can categorically house data in a certain way is a lifesaver. Additionally, as the video below shows, clinical evaluations involve several experts from different departments in the authoring and reviewing process. To mitigate collaboration errors, there needs to be an effective and easy way to facilitate data transfer of large volumes of screened data and full-text articles to different team members.

As mentioned earlier, meticulous documentation of the search strategy within the clinical evaluation plan is a critical aspect of regulatory compliance. Any minor error in this documentation, such as a missed quotation, incorrectly captured filter, or a spelling mistake, can render the entire search invalid. Therefore, it is imperative that the search be documented verbatim to ensure accuracy and reproducibility.

The issue of duplicated data in Clinical Evaluation Reports (CERs) is a common challenge that can compromise the accuracy and reliability of the evaluation. The complexity of handling a large number of articles, coupled with the possibility of multiple team members being involved in screening and summarization, increases the risk of duplicate data inclusion. Handling the presentation of the same data in different articles, particularly when one article describes a prospective study and another conducts a meta-analysis using that study as a subset, adds another layer of complexity to the management of duplicate data. Moreover, when screening, review, and summarization tasks are distributed among several team members, the potential for errors is further heightened.

Thus, it is recommended to pre-screen your data to ensure all duplicates must are filtered out, and conduct a sanity check to ensure data has not been duplicated, post collation of the summaries.

1. Appendices and Flowcharts: Notified Bodies mandate the inclusion of an appendix that comprehensively documents the entire screening process. This must detail the number of articles included at each level, the quantity excluded, and the reason for exclusion. It is also ideal to present this data in a flowchart form for ease of understanding and review for both internal and external stakeholders. Again, doing this manually is not only time-consuming and cumbersome, but also a major source of errors. Working with suitable software that can automate the creation of appendices based on the screening done and generate prisma flowcharts to depict the screening process is the way to go to avoid unnecessary loss of time and labour.
2. Validation of Search Strategy: Notified bodies emphasize the importance of validating the search strategy to ensure its accuracy and reliability. It is essential to ensure the search strategy is prepared and executed by suitable resources to ensure compliance with EU MDR. Additionally, application of formulas to assess the precision and accuracy of the searches can help validate the search strategy.
3. Adherence to Established Methods: The search strategy must be developed based on well-established methodologies such as PICO, PEO (Population, Exposure and Outcome), SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) etc.
4. Bias Reduction Methods: Implement methods to reduce bias in the search strategy, and thoroughly document these methods within the plan. This includes addressing potential biases related to study selection, data extraction, and analysis. Clearly outline steps taken to minimize bias, enhancing the credibility of the overall clinical evaluation.
5. Documentation of Deviations: In the event of any deviations from the initially planned search strategy, document these deviations in the final report. Include a well-justified explanation for the deviations, providing transparency and accountability.

Keeping the above guidelines in mind, Celegence developed CAPTIS to help alleviate the manual burden of conducting a systematic search for our medical writing teams. This innovative software platform helps our teams in countless ways – making the searches more effective, easy to execute, easy to screen. It also makes it significantly easier to collaborate and handle large volumes of data.

1. Customizable Tagging System:
   • CAPTIS offers a customizable tagging system that allows Celegence’s reviewers to create specific tags related to article quality or content.
   • Tags can be filtered to assess the amount of data available for each indication, intended purpose, population, or anatomical location.
2. Flexible Search Modification:
   • Reviewers can modify searches within CAPTIS by extending date ranges, adjusting filters, and modifying search terms.
   • The system retains the actions taken on previous articles, eliminating the need to redo screening for overlapping results with new search strings.
3. Automated Duplicate Detection:
   • CAPTIS includes an automated duplicate detection feature, significantly reducing the manual effort required to identify and manage duplicate data.
   • This feature enhances data accuracy and minimizes the risk of including redundant information in the final clinical evaluation.
4. Comprehensive Data Storage:
   • CAPTIS downloads and stores all freely available full-text articles, eliminating the need for manual searches, downloads, and renaming.
   • Stored articles are accessible within the system, eliminating the hassle of transferring content among team members via external platforms like SharePoint or OneDrive.
5. Efficient Collaboration Tools:
   • CAPTIS facilitates collaboration among our team members by providing efficient tools for data transfer, including large volumes of screened data and full-text articles.
   • Team members can seamlessly contribute to authoring and reviewing documents, minimizing the potential for errors in the collaborative process.

1. Time Efficiency:
   • CAPTIS significantly reduces the time required for systematic searches by automating processes such as duplicate detection and article storage.
   • The system’s efficiency ensures that our teams can focus on analysis and evaluation rather than time-consuming manual tasks.
2. Data Accuracy and Consistency:
   • The customizable tagging system and automated duplicate detection contribute to enhanced data accuracy and consistency throughout the systematic review process.
   • CAPTIS minimizes the risk of errors associated with manual data handling.
3. Regulatory Compliance:
   • CAPTIS is designed with the specific requirements of EU MDR in mind, ensuring that the systematic searches conducted within the system align with regulatory expectations.
   • The system facilitates the documentation and reporting necessary for compliance with Notified Body requirements.
4. Centralized Access to Source Documents:
   • CAPTIS provides a centralized repository for source documents, making it easy for team members to access relevant information with a simple click.
   • This centralized access streamlines collaboration and eliminates the need for external document transfer.

Led by CEO Sonia Veluchamy, Celegence aims to provide reliable, fast, and cost-efficient regulatory services by harnessing the power of innovative technology like CAPTIS. Our tech-enabled services, combined with our global regulatory expertise, offer our clients a smarter way to stay ahead of regulatory changes, significantly reducing the cost and time associated with regulatory processes. Partnering with Celegence means not just overcoming challenges but redefining your approach to medical device regulatory affairs through tech-driven solutions — a smarter way forward. Reach out to us at info@celegence.com or contact us online for more information.