Print page Resize text Change font-size Change font-size Change font-size High contrast


methodologicalGuide9_2.shtml
Home > Standards & Guidances > Methodological Guide

ENCePP Guide on Methodological Standards in Pharmacoepidemiology

 

9.2. Methods of statistical signal detection

Quantitative analysis of spontaneous adverse drug reaction reports is routinely used in drug safety research. Quantitative signal detection using spontaneous ADR reporting (Pharmacoepidemiol Drug Saf. 2009;18(6):427-36) describes the core concepts behind the most common methods, the proportional reporting ratio (PRR), reporting odds ratio (ROR), information component (IC) and empirical Bayes geometric mean (EBGM). The authors also discuss the role of Bayesian shrinkage in screening spontaneous reports and the importance of changes over time in screening the properties of the measures. Additionally, they discuss major areas of controversy (such as stratification and evaluation and implementation of methods) and give some suggestions as to where emerging research is likely to lead. Data mining for signals in spontaneous reporting databases: proceed with caution (Pharmacoepidemiol Drug Saf. 2007;16(4):359–65) reviews data mining methodologies and their limitations and provides useful points to consider before incorporating data mining as a routine component of any pharmacovigilance program. Disproportionality Analysis for Pharmacovigilance Signal Detection in Small Databases or Subsets: Recommendations for Limiting False-Positive Associations (Drug Saf. 2020;43(5):479-87) evaluates the impact of database size on the performance of disproportionality analysis, with regards to limiting spurious associations.

 

The revised guidance on Screening for adverse reactions in EudraVigilance describes methods for screening adverse drug reactions used by the European Medicines Agency and national competent authorities. The proposed methods complement the classical disproportionality analysis with additional data summaries, based on both statistical and clinical considerations. This approach is based on the fact that, although disproportionality methods have demonstrated to detect many adverse reactions before other currently used methods of signal detection, this is not true for all types of adverse reactions. Hence a comprehensive and efficient routine signal detection system will seek to integrate a number of different methods to prioritise the drug event combinations for further evaluation. For the methods recommended, the guidance addresses elements of their interpretation, their potential advantages and limitations and the evidence behind. Areas of uncertainty that require resolution before firm recommendations can be made are also mentioned.

 

Methods such as multiple logistic regression (that may use propensity score-adjustment) have the theoretical capability to reduce masking and confounding by co-medication and underlying disease. The letter Logistic regression in signal detection: another piece added to the puzzle (Clin Pharmacol Ther. 2013;94(3):312) highlights the variability of results obtained in different studies based on this method and the daunting computational task it requires. More work is needed on its value for pharmacovigilance in the real-world setting.

 

A more recent proposal involves a broadening of the basis for computational screening of individual case safety reports, by considering multiple aspects of the strength of evidence in a predictive model. This approach combines disproportionality analysis with features such as the number of well-documented reports, the number of recent reports and geographical spread of the case series (Improved statistical signal detection in pharmacovigilance by combining multiple strength-of-evidence aspects in vigiRank, Drug Saf. 2014;37(8):617–28). In a similar spirit, logistic regression has been proposed to combine a disproportionality measure with a measure of unexpectedness for the time-to-onset distribution (Use of logistic regression to combine two causality criteria for signal detection in vaccine spontaneous report data, Drug Saf. 2014;37(12):1047-57). In A prediction model‐based algorithm for computer‐assisted database screening of adverse drug reactions in the Netherlands (Pharmacoepidemiol Drug Saf. 2018;27(2):199-205), five relevant characteristics (number of reports, disproportionality, Naranjo score, proportion of MAH reports, proportion of HCP reports) were chosen as potential predictors in the model and tested against the presence in the Summary of Product Characteristics (SmPC) of each unique drug‐ADR association at the time of the analysis. All candidate predictors were included into the final model with an increased screening efficiency. The authors comment that the choice of candidate predictors may depend on each spontaneous report databases but that the method of generating a prediction model‐based priority list of signals could be useful in other databases.

 

Disproportionality methods are usually calculated on the cumulative data and therefore do not provide a direct insight into temporal changes in frequency of reports. Methodologies to monitor changes in the frequency of reporting over time have been developed with the focus to enhance pharmacovigilance when databases are small, when drugs have established safety profiles and/or when product quality defects, medication errors and cases of abuse or misuse are of concern. Automated method for detecting increases in frequency of spontaneous adverse event reports over time (J Biopharm Stat. 2013; 23(1):161-77) presents a regression method with both smooth trend and seasonal components, while An algorithm to detect unexpected increases in frequency of reports of adverse events in EudraVigilance (Pharmacoepidemiol Drug Saf. 2018;27(1):38-45) presents the testing of a model based on a negative binomial time-series regression model on thirteen historical concerns. Additionally, a modification of the Information Component to screen for spatial-temporal disproportionality is described in Using VigiBase to Identify Substandard Medicines: Detection Capacity and Key Prerequisites (Drug Saf. 2015; 38(4): 373–82). Despite the promising results of these methods, and even if theoretically they seem appealing, limited work has been performed to assess their effectiveness. Thus, these methods should be implemented with quality control measures to ensure acceptable performance.

 

As understanding increases regarding the mechanisms at a molecular level that are involved in adverse effects of drugs it would be expected that this information will inform efforts to predict and detect drug safety problems. Such modelling is presented in Data-driven prediction of drug effects and interactions (Sci Transl Med. 2012 14;4(125):125ra31) and should be a major focus of drug safety research activities. An example of an application of this concept is illustrated in the paper Cheminformatics-aided pharmacovigilance: application to Stevens-Johnson Syndrome (J Am Med Inform Assoc. 2016; 23(5): 968–78) where the authors apply a Quantitative Structure-Activity Relationship (QSAR) model to predict the drugs associated with Stevens Johnson syndrome in a pharmacovigilance database. In Target Adverse Event Profiles for Predictive Safety in the Postmarket Setting (Clin Pharmacol Ther. 2021;109(5):1232-43), the authors identify drugs that share pharmacological targets with the drug of interest and use information from these drugs to predict post-marketing adverse drug reactions of the drug of interest. Machine learning on data from the FDA Adverse Event Reporting System, peer-reviewed literature and FDA drug labels is used for the prediction.

 

« Back