11.2. Data quality frameworks
Quality in research impacts on regulatory practice, medicines development and public health. Quality is a measure of excellence and quality management includes all the activities that organisations use to direct, control and coordinate quality (International Standards Organization, ISO 9000). Quality management principles as described in ISO Quality management principles are also applicable to pharmacoepidemiological research. The book Total Quality Management-Key Concepts and Case Studies (D.R. Kiran, BSP Books, Elsevier, 2016) deals with the management principles and practices that govern the quality function and presents all the aspects of quality control and management in practice.
The Commission Implementing Regulation (EU) No 520/2012 and the Good pharmacovigilance practices (GVP) Module I provide a framework for the quality management of pharmacovigilance and safety studies of authorised medicinal products. Measurable quality requirements can be achieved by:
Quality planning: establishing structures (including validated computerised systems) and planning integrated and consistent processes;
Quality assurance and control: monitoring and evaluating how effectively the structures and processes have been established and how effectively the processes are being carried out;
Quality improvement: correcting and improving the structures and processes where necessary.
Pharmacoepidemiological research is becoming more complex and may use a large amount of data. In such situation, managing quality implies a risk-based approach. Risk-based quality management is incorporated as Good Clinical Practice expectation in ICH E8 (R1) and addressed in the European Commission’s Risk proportionate approaches in clinical trials, EMA’s Reflection paper on risk-based quality management in clinical trials and the GVP Module III on Pharmacovigilance inspections.
The considerations and recommendations of Chapter 5.1 on the definition and validation of drug exposure, outcomes and covariates are essential aspects that need to be addressed for quality management.
Large electronic data sources such as electronic health care records, insurance claims data and administrative data have opened up new opportunities for investigators to rapidly conduct pharmacoepidemiological studies and clinical trials in real-world health care settings and with a large number of patients. A concern is that these data have not been collected systematically for research on the utilisation, safety and effectiveness of medicinal products, which could affect the validity, reliability and reproducibility of the investigation. Attempts have therefore been made to create a systematic methodology for data quality assessment in order to understand the strengths and limitations of the data to answer a research question, the impact they may have on the study results and the measures to be taken to improve or complement the available data. Several data quality frameworks, which are generally concordant as regards their main quality components, have been published.
A Harmonized Data Quality Assessment Terminology and Framework for the Secondary Use of Electronic Health Record Data (eGEMs. 2016;4(1):1244) describes a framework with three data quality categories: Conformance (with sub-categories of Value, Relational and Computational Conformance), Completeness and Plausibility (with sub-categories of Uniqueness, Atemporal and Temporal Plausibility). These categories are applied in two contexts: Verification and Validation. This framework is used by the US National Patient-Centered Clinical Research Network (PCORnet), with an additional component, persistence, and the Observational Health Data Science and Informatics (OHDSI) network. Based on the same framework, the Data Analytics chapter of the Book of OHDSI (2020) provides an automated tool testing the data quality checks in databases conforming to the OMOP common data model. Increasing Trust in Real-World Evidence Through Evaluation of Observational Data Quality (medRxiv. 2021) describes an open source R package that executes and summarises over 3,300 data quality checks in databases available in OMOP.
Duke-Margolis Center’s Characterizing RWD Quality and Relevancy for Regulatory Purposes (2018) specifies that determining if a real-world dataset is fit-for-regulatory-purpose is a contextual exercise, as a data source that is appropriate for one purpose may not be suitable for other evaluations. A RWD set should be evaluated as Fit-for-purpose if, within the given clinical and regulatory context, it fulfils two dimensions: Data Relevancy (including Availability of key data elements, Representativeness, Sufficient subjects and Longitudinality) and Data Quality (Accuracy, Completeness, Provenance and Transparency of data processing).
Data quality frameworks have been described for specific data sources. For example, the EMA’s Draft Guideline on Registry-based studies describes four quality components for use of patient registries (mainly disease registries) for regulatory purposes: Consistency, Completeness, Accuracy and Timeliness.
Rules, procedures, roles and responsibilities of quality assurance and quality control for clinical trials and biomedical research are well defined and described in many documents, such as the ICH E6 (R2) Good clinical practice, the European Forum for Good Clinical Practice (EFCGP) Guidelines, the Imperial College Academic Health Science Centre (AHSC)’s Quality Control and Quality Assurance SOP or the article Quality by Design in Clinical Trials: A Collaborative Pilot With FDA (Ther Innov Regul Sci. 2013; 47(2):161-6).
Quality management principles applicable to observational studies with primary data collection or secondary use of data are described in the Commission Implementing Regulation (EU) No 520/2012, GVP Module I, FDA’s Best Practices for Conducting and Reporting Pharmacoepidemiologic Safety Studies Using Electronic Health Care Data Sets, the ISPE GPP or the Guidelines and recommendations for ensuring Good Epidemiological Practice (GEP): a guideline developed by the German Society for Epidemiology (Eur J Epidemiol. 2019;34(3):301-17).
Real-World Data for Regulatory Decision Making: Challenges and Possible Solutions for Europe (Clin Pharmacol Ther. 2019; 106(1):36-9) describes four criteria for acceptability of RWE for regulatory purposes: Derived from data source of demonstrated good quality, Valid (internal and external), Consistent and Adequate. Challenges for this acceptability and possible solutions in the EU context are presented.
The following articles are practical examples of quality aspects implementation in different settings: