Challenges and Solutions in the Development and Validation of Molecular Assays

Author: Cara N. Wilder, PhD

Molecular assays are powerful and precise diagnostic tools for the detection of clinically relevant infectious agents. These technologies have demonstrated a number of advantages over traditional culture-based approaches, including increased sensitivity and specificity, rapid turnaround time, multiplexing, reproducibility, and the ability to detect fastidious and unculturable organisms. While molecular assays have exhibited a number of benefits, great care must be taken to ensure that assays are properly validated to guarantee performance and uncompromised data. The challenges associated with evaluating the analytical performance of a molecular assay as well as guidance on selecting the ideal reference materials that support this endeavor are discussed below.

One of the primary considerations when developing a new molecular assay is performing validation studies on the inclusivity, cross-reactivity, and limit of detection of the assay. Inclusivity and cross-reactivity testing are performed to confirm that the assay detects the desired target while excluding undesired targets that may be present in the sample. The limit of detection is the minimum amount of the desired target that can be accurately distinguished from the absence of a sample within a given level of confidence. Establishing these parameters is an essential part of assay development and validation, particularly when evaluating diagnostic assays whose results can affect public health. In many cases, the rapid and accurate identification of an infectious agent is critical to the timely delivery of therapeutic medication. Thus, choosing suitable reference materials is imperative in ensuring the precision of a new assay.

Determining which reference strains are ideal for evaluating inclusivity and cross-reactivity can be a daunting task and will depend on the application of the assay and critical information about the target. For example, a screening assay used to diagnose cases of urogenital chlamydia in humans could require testing endocervical, urethral, vaginal, rectal, or urine specimens for the detection and identification of Chlamydia trachomatis.² For inclusivity testing, the use of nucleic acids representing each of the C. trachomatis serovars predominantly isolated from the urogenital tract (D, Da, E, F, G, Ga, H, I, J, K) may be recommended.³ In contrast, exclusivity could be established and evaluated through the use of reference materials representing additional serovars typically isolated from the conjunctiva and inguinal lymph nodes (A, B, Ba, C, and L1, L2, L2’, L2a, L2b, and L3, respectively), as well as other Chlamydia species, strains within the related genus Chlamydophila, and non-related genera that share the same clinical niche with the target organism.^3,4 These latter strains should be carefully selected to evaluate and confirm that the assay does not exhibit cross-reactivity.

In addition to choosing the appropriate strains, having an expansive sample set is imperative in determining the significance of your experimental results. Using the example above, if the assay was able to accurately detect 58 strains within an inclusivity panel comprising 60 C. trachomatis strains representing common serovars isolated from the urogenital tract, it would indicate that the test has 96.7% specificity for accurately identifying the sample set analyzed. Regarding exclusivity, if the assay was unable to detect 78 strains within an exclusivity panel comprising 80 strains of related and non-related non-target serovars and species, it would indicate that the test has 97.5% specificity for correctly providing a negative result for the sample set analyzed. Taking this data into account, along with sample size and the statistical likelihood of false positives or false negatives, you could infer that there is a high probability that the test would accurately diagnose urogenital cases of chlamydia.

It is also important to assess the limit of detection. The method used to establish this parameter can vary depending on assay type and use, though it is frequently reviewed through creating a serial dilution of the target, spiking the preparations into the appropriate sample matrix, and performing the assay as described. Using the example above, samples representing inclusive C. trachomatis strains could be quantified and diluted, spiked into a urogenital specimen matrix, and then analyzed in replicate using the assay. Here, the lower limit of detection would be considered as the last dilution exhibiting an accurate and reliable positive result.

When analyzing analytical sensitivity, the significance of your results can be dependent on the dilution range used and the number of replicates. Prior to the analysis, samples should be quantified then serially diluted to an appropriate range of concentrations that were previously determined through a range finding study. Depending on the assay being analyzed, the dilution series may vary in the number of dilutions used as well as the dilution factor. Generally, the closer a dilution series is to the target concentration, the more accurate the limit of detection will be.

When you are ready to obtain reference materials for evaluating analytical specificity and sensitivity, it is important to go to a reliable source that provides authenticated standards. For inclusivity and cross-reactivity testing, preparations that are identified down to the species or strain level, as well as functional characteristics for any important traits such as serovar, toxinotype, or drug resistance are ideal for creating inclusivity and exclusivity panels. Regarding the limit of detection, reference standards that are quantified for concentration or genome copy number and are authenticated to ensure integrity, purity, functional activity, and identity are ideal for determining the detection limit of an assay. Currently, biological and molecular reference standards are produced by a number of entities, including government agencies, commercial companies, and non-profit institutions. ATCC, for example, offers an expansive array of cultures and nucleic acids for use as reference materials in evaluating analytical specificity and sensitivity. These products are prepared as high-quality, authenticated materials backed by meticulous quality control procedures, making them ideal for use in assay development and validation. Further, ATCC offers custom products and services that are designed to solve complex biological challenges, including nucleic acid purification, quantitation and titering services, inactivated materials, as well as master and working cell bank expansion.

Overall, there are a number of challenges associated with the development and validation of a molecular assay. Choosing the ideal reference materials is critical in the evaluation of analytical performance. Through the use of a diverse array of representative authenticated and quantified materials from a reputable source such as ATCC, analytical specificity and sensitivity can be established.

References

1. OIE. Validation Guideline 3.6.3. Development and Optimisation of Nucleic Acid Detection Assays. (2014).
2. Mishori, R., McClaskey, E. L. & WinklerPrins, V. J. Chlamydia trachomatis infections: screening, diagnosis, and management. American family physician 86, 1127-1132 (2012).
3. Morre, S. A. et al. Urogenital Chlamydia trachomatis serovars in men and women with a symptomatic or asymptomatic infection: an association with clinical manifestations? Journal of clinical microbiology 38, 2292-2296 (2000).
4. Somboonna, N., Mead, S., Liu, J. & Dean, D. Discovering and differentiating new and emerging clonal populations of Chlamydia trachomatis with a novel shotgun cell culture harvest assay. Emerging infectious diseases 14, 445-453, doi:10.3201/eid1403.071071 (2008).