Evolution of assay interference concepts in drug discovery

KEYWORDS:

• High-throughput screening
• assay artifacts
• interference compounds
• substructures
• screening data analysis
• X-ray structures
• lead optimization
• drug discovery

1. Introduction

Biological assays for small molecules are vulnerable to experimental limitations and artifacts. False-positive activity readouts represent a recurrent and well-appreciated problem for high-throughput screening. Accordingly, hits from a primary high-throughput screen are usually subjected to secondary assays to reevaluate activity signals and establish dose–response behavior of active compounds. Typically, such confirmatory assays eliminate many but not all false-positive hits. While assay artifacts may frequently be attributable to technical issues, they are also caused by compounds that are not specifically active against a target of interest, but fake positive assay signals by a variety of undesired mechanisms. Such chemical entities are often termed assay interference compounds. For drug discovery, major problems arise when such interference compounds are difficult to identify and camouflaged false-positives enter hit-to-lead and lead optimization stages of the discovery pipeline. Significant time and resources might be wasted until their artificial nature is ultimately uncovered, especially if derivatives of pseudo-hits approach the realm of potential candidate compounds. Furthermore, undetected false-positives plague science and hinder progress. For example, in academic settings, new compounds – including those originating from computational screening – are often only investigated in a single assay system. If subtle artifacts occur under such conditions, they are notoriously difficult to detect. Once published, pseudo-hits tend to propagate through the literature and frequently give rise to follow-up studies that are doomed to fail and misleading at best. Over the past decade, it has increasingly been recognized that assay interference effects often are of unexpectedly large magnitude, continue to compromise drug discovery efforts, and pollute the scientific literature [1]. Most of these problems are caused by assay interference compounds, ‘chemical con artists’ [2], which are often rather difficult to uncover [1,2]. Different concepts have been introduced for recognizing and combating assay interference, which are of critical importance for the practice of biological screening and medicinal chemistry. However, these concepts are also controversially viewed for reasons discussed below. To assess their value, different scientific approaches and viewpoints should be taken into consideration.

2. Assay interference concepts

Assay interference might result from a variety of molecular effects. Beginning in 2002, work of Shoichet and colleagues firmly established colloidal aggregation of compounds and resulting nonspecific interactions between aggregates and proteins compromising their function as a major source of assay interference [3]. In subsequent studies over more than a decade, compound aggregation effects have been explored in many experimental settings, providing a refined and detailed picture of aggregation-based assay interference. At the computational level, these developments also included molecular similarity calculations and the introduction of similarity-based rules for the detection of likely aggregators [4]. In 2010, Baell and Holloway introduced pan-assay interference compounds (PAINS) based on the analysis of screening data as another major source of assay interference [5], due to undesired reactivities. Among others, these include different types of covalent modifications disabling proteins or disrupting cell membranes, redox cycling producing hydrogen peroxides, metal ion chelation compromising the function of metal-dependent enzymes, or auto-fluorescence giving rise to false signals in fluorescence-based assays. More than 400 compound classes were categorized as PAINS, which are typically embedded as substructures in larger compounds. Similar to colloidal aggregators, the PAINS concept has also been further refined over the years [6] and extended to natural products, leading to the introduction of invalid metabolic panaceas (IMPs) [7]. IMPs represent naturally occurring molecules with frequently reported activities that cannot be rationalized and are artificial in nature. Furthermore, in 2012, a compendium of 275 rules was introduced for identifying assay interference compounds that were derived by practicing medicinal chemists in the pharmaceutical industry over the course of nearly 20 years [8]. These rules account for similar reactivities and effects as the PAINS classification and represent another extensive knowledge-based framework for detecting and rationalizing assay interference. Taken together, these exemplary studies have raised awareness of assay interference and its multi-facetted origins and paved the way for identifying interference compounds. It is fair to say, however, that the PAINS concept has become a paradigm for compounds with undesired reactivity in biological assays, due to pioneering efforts of Jonathan B. Baell, Michael A. Walters, and colleagues [2,5,9,10].

3. PAINS revisited

Substructures representing PAINS are found in many compounds from drug discovery including target class-directed privileged structures such as flavonoids [11] or rhodanines [12] that are often investigated in medicinal chemistry. For example, rhodanines were shown to undergo light-induced reactions leading to irreversible covalent modifications of proteins compromising their function. Moreover, widely considered drug candidates have also been categorized as PAINS or IMPs. An instructive example is provided by curcumin [10], a major natural ingredient of turmeric used in traditional medicine. Curcumin derivatives have been associated with a variety of biological activities and continue to be investigated in many clinical studies. However, curcumin was found to be reactive and unstable under biologically relevant conditions having only little bioavailability. Its reaction products covalently modify proteins or disrupt biological membranes. Many putative activities of curcumin derivatives were not verifiable and hence attributed to undesired reactivity [10]. Such findings call the activities of popular compounds into question and have contributed to the increasing popularity of the PAINS concept, but also triggered a further assessment of PAINS. In independent investigations, systematic analysis of publicly available screening assays revealed that compounds containing essentially the entire spectrum of classified PAINS had very different frequencies of activity [13,14]. Compounds containing PAINS often yielded unusually high hit rates, clearly indicating the presence of assay artifacts. However, other screening molecules containing even most notorious PAINS such as rhodanines, quinones, or catechols were rarely active or even consistently inactive in screening assays. Furthermore, compounds with the same PAINS substructure were found to display very different activities. Moreover, compounds with PAINS can also possess specific activity, as has been shown in various case studies using orthogonal assay systems or by X-ray crystallography. For example, structures of complexes formed between target proteins and flavonoid or rhodanine inhibitors have been determined [15], providing ultimate proof of their activity. In fact, a systematic analysis of publicly available X-ray structures identified nearly 3000 complexes containing ~1100 ligands with 70 different PAINS substructures [15]. Taken together, the results of large-scale analyses of biological screening assays or X-ray data have revealed that compounds with PAINS substructures do not necessarily cause artifacts. While some of these compounds display artificially high hit rates in screening assays and fake activities that cannot be confirmed, others exhibit greatly varying activity profiles in screening panels, may be specifically active, or consistently inactive. From these studies, the picture is emerging that undesirable PAINS reactivity and resulting assay artifacts on the one hand, or chemical integrity and specific biological activity on the other depend on how PAINS substructures are embedded in larger compounds. Thus, their structural context essentially determines their fate and it cannot be generalized that compounds with PAINS substructures elicit assay artifacts.

4. Expert opinion

Assay interference compounds represent a major caveat for basic research and drug discovery [1]. Different approaches have been introduced to identify assay artifacts and associated interference compounds. Among these, the PAINS concept plays a central role and has experienced increasing attention over the past decade. Notably, studying assay interference and deriving classifications of interference compounds is far from being trivial and requires high-level medicinal chemistry expertise [2,8].

While the notion of PAINS has raised widespread awareness of assay interference and put its assessment onto a new level, the concept has also been controversially viewed. An often articulated critique refers to the fact that the original classification of PAINS [5] was based upon limited data from a single assay format, thus calling generalization into question. Although this concern was principally valid, it does not take anything away from the fundamental relevance of this concept and its positive impact on drug discovery efforts.

Data sparseness in the assessment of assay interference can be overcome in different ways. For example, thousands of screening compounds were identified that produced high hit rates in hundreds of different assays they were tested in [16]. From these compounds, more than 6000 pairs, triplets, or larger series of structural analogs were extracted and more than 5000 of these series did not contain any PAINS substructures or compounds with known aggregation potential [16]. When made publicly available [17], such data sets and the associated structural and activity information provide a sound basis for an unbiased large-scale assessment of potential assay interference effects and/or the study of multi-target activity. Of note, focusing assay interference analysis on analog series instead of individual compounds provides a valuable control for the structural context dependence of potential undesired reactivities discussed above.

PAINS analysis has been further evolved and refined over time [6]. Moreover, the complexity of PAINS actions and variability of their activity profiles have been demonstrated at different levels [13–15]. These studies have revealed that PAINS-induced activity artifacts cannot be generalized, but require careful assessment on a case-by-case basis. To these ends, the notion of PAINS is indispensable.

Because PAINS are typically contained as substructures in larger compounds, they can conveniently be detected by substructure searching. Accordingly, PAINS have been promoted for use as substructure filters [5,6]. This has been another issue of substantial debate and critique in the field [13]. Detection of PAINS by substructure filters may lead to a ‘black and white’ categorization of PAINS, at least by an uninitiated user. Clearly, this would be inconsistent with the differential activity profiles of PAINS-containing compounds and their potential to specifically interact with targets. Thus, uncritical application of PAINS filters is also likely to eliminate interesting active compounds from further consideration. However, this does not render such filters obsolete and their utility is often misunderstood, even by experts. Substructure filters enable the quick identification of PAINS in test compounds, but do not imply that these compounds inevitably cause artifacts. Rather, PAINS detection should raise awareness that an active compound might be associated with potential liabilities that should be carefully considered and experimentally assessed. For this purpose, the use of PAINS filters continues to be meaningful. To this date, potential assay interference compounds frequently remain unrecognized in publications and there is still room for improvement in their detection and further evaluation. Importantly, the study of interference compounds in different types of assays continues to be an active area of research [18] and, while already far advanced through the concepts discussed herein, still provides many opportunities for future scientifc work.

Declaration of interest

J Bajorath has no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Additional information

Funding

J Bajorath is funded by the University of Bonn.