Challenges and Solutions in the Development and Validation of Molecular-Based Assays
Learn more about molecular standards
Download printer-friendly version
Molecular-based assays are powerful and precise diagnostic tools for the detection of clinically relevant infectious agents. These novel technologies have demonstrated a number of advantages over traditional culture-based approaches, including increased sensitivity and specificity, rapid turnaround time, multiplexing, reproducibility, and the ability to detect fastidious and unculturable organisms. While molecular-based assays have exhibited a number of benefits, great care must be taken to ensure that assays are properly validated to guarantee performance and uncompromised data. The challenges associated with evaluating the analytical performance of a molecular-based assay as well as guidance on selecting the ideal reference materials that support this endeavor are discussed below.
One of the primary considerations in the development and validation of a novel molecular-based assay is the evaluation of analytical specificity and sensitivity.1 Here, analytical specificity is defined as the ability of an assay to distinguish between target and non-target sequences. Assays designed to be highly selective with specificity toward a particular genetic target that is not present in non-target organisms are described as exhibiting exclusivity. In contrast, an assay may target a genetic sequence that is found in several strains or serovars of a species, or several species of a genus, while excluding other related organisms that may cause cross-reactivity. This latter assay type can be described as exhibiting inclusivity.1 Establishing the ideal inclusivity/exclusivity parameters is an essential part of assay development and validation, particularly when evaluating diagnostic assays that have results that can affect public health. In many cases, the rapid and accurate identification of an infectious agent is critical to the timely delivery of therapeutic medication. Thus, choosing suitable reference materials is imperative in ensuring the precision of a new assay.
Determining which reference strains are ideal for establishing analytical specificity can be a daunting task and will depend on the application of the assay and critical information about the target. For example, a screening assay used to diagnose cases of urogenital chlamydia in humans could require testing endocervical, urethral, vaginal, rectal, or urine specimens for the detection and identification of Chlamydia trachomatis.2 For inclusivity testing, the use of nucleic acids representing each of the C. trachomatis serovars predominantly isolated from the urogenital tract (D, Da, E, F, G, Ga, H, I, J, K) may be recommended.3 In contrast, exclusivity could be established and evaluated through the use of reference materials representing additional serovars typically isolated from the conjunctiva and inguinal lymph nodes (A, B, Ba, C, and L1, L2, L2’, L2a, L2b, and L3, respectively), as well as other Chlamydia species, strains within the related genus Chlamydophila, and non-related genera that share the same clinical niche with the target organism.3,4 These latter strains should be carefully selected to evaluate and confirm that the assay does not exhibit cross-reactivity.
In addition to choosing the appropriate strains, having an expansive sample set is imperative in determining the significance of your experimental results. Using the example above, if the assay was able to accurately detect 58 strains within an inclusivity panel comprising 60 C. trachomatis strains representing common serovars isolated from the urogenital tract, it would indicate that the test has 96.7% specificity for accurately identifying the sample set analyzed. Regarding exclusivity, if the assay was unable to detect 78 strains within an exclusivity panel comprising 80 strains of related and non-related non-target serovars and species, it would indicate that the test has 97.5% specificity for correctly providing a negative result for the sample set analyzed. Taking this data into account, along with sample size and the statistical likelihood of false positives or false negatives, you could infer that there is a high probability that the test would accurately diagnose urogenital cases of chlamydia.
It is also important to assess analytical sensitivity, which is known as the limit of detection. This parameter is the minimum amount of the target sequence that can be accurately distinguished from the absence of a sample within a given level of confidence. The method used to establish the limit of detection can vary depending on assay type and use, though it is frequently reviewed through creating a serial dilution of the target, spiking the preparations into the appropriate sample matrix, and performing the assay as described. Using the example above, samples representing inclusive C. trachomatis strains could be quantified and diluted, spiked into a urogenital specimen matrix, and then analyzed in replicate using the assay. Here, the lower limit of detection would be considered as the last dilution exhibiting an accurate and reliable positive result.
When analyzing analytical sensitivity, the significance of your results can be dependent on the dilution range used and the number of replicates. Prior to the analysis, samples should be quantified then serially diluted to an appropriate range of concentrations that were previously determined through a range finding study. Depending on the assay being analyzed, the dilution series may vary in the number of dilutions used as well as the dilution factor. Generally, the closer a dilution series is to the target concentration, the more accurate the limit of detection will be.
When you are ready to obtain reference materials for evaluating analytical specificity and sensitivity, it is important to go to a reliable source that provides authenticated standards. For analytical specificity, preparations that are identified down to the species or strain level, as well as functional characteristics for any important traits such as serovar, toxinotype, or drug resistance are ideal for creating inclusivity and exclusivity panels. Regarding analytical sensitivity, reference standards that are quantified for concentration or genome copy number and are authenticated to ensure integrity, purity, functional activity, and identity are ideal for determining the detection limit of an assay. Currently, biological and molecular reference standards are produced by a number of entities, including government agencies, commercial companies, and non-profit institutions. ATCC, for example, offers an expansive array of cultures and nucleic acids for use as reference materials in evaluating analytical specificity and sensitivity. These products are prepared as high-quality, authenticated materials backed by meticulous quality control procedures, making them ideal for use in assay development and validation. Further, ATCC offers custom products and servicesthat are designed to solve complex biological challenges, including nucleic acid purification, quantitation and titering services, inactivated materials, as well as master and working cell bank expansion.
Overall, there are a number of challenges associated with the development and validation of a molecular-based assay. Choosing the ideal reference materials is critical in the evaluation of analytical performance. Through the use of a diverse array of representative authenticated and quantified materials from a reputable source such as ATCC, analytical specificity and sensitivity can be established.
Download printer-friendly version
- OIE. Validation Guideline 3.6.3. Development and Optimisation of Nucleic Acid Detection Assays. 2014.
- Mishori R, McClaskey EL, and WinklerPrins VJ. Chlamydia trachomatis infections: screening, diagnosis, and management. Am Fam Physician 86: 1127-1132, 2012.
- Morre SA, et al. Urogenital Chlamydia trachomatis serovars in men and women with a symptomatic or asymptomatic infection: an
association with clinical manifestations? J Clin Microbiol 38(6): 2292-2296, 2000.
- Somboonna N, Mead S, Liu J, and Dean D. Discovering and differentiating new and emerging clonal populations of Chlamydia trachomatis
with a novel shotgun cell culture harvest assay. Emerg Infect Dis 14(3): 445-453, 2008.