The ThinPrep Pap Test, a liquid-based cytology test (LBC) approved by the FDA the year before, made it possible, and convenient, to perform both a Pap test and HPV testing on cervical cells collected and stored in just one solution-filled vial.
Over the years, four more HPV tests were approved by the FDA. One of them—the cobas HPV Test—was approved in 2014 as the first HPV test for primary cervical cancer screening.
The basic goal of screening is to detect pre-cancer that’s going to become cancer if it’s not treated—and to treat it to avoid cancer. It’s essentially a preventive, not a diagnostic test. In the United States, where invasive cancer itself is quite rare, according to cancer experts, pre-cancers in the tens and hundreds of thousands are detected, and treated in order to prevent cancer.
For that primary goal of screening, HPV alone is overwhelmingly more sensitive than Pap alone. When both are used together, it is called co-testing, which is used widely only in the United States. However, in a paper published in 2014 in JNCI,1 authors from the National Cancer Institute and Kaiser Permanente Northern California concluded from a study of hundreds of thousands of women from Kaiser that there is only miniscule additional reassurance of adding a Pap test when HPV is negative.
When the SurePath LBC test became widely used for Pap tests, many clinicians and clinical laboratories assumed the liquid collection media could be used interchangeably with ThinPrep for HPV testing—that the two liquid media would perform the same way.
This, however, is not the case—the concentration of cells is significantly different. With ThinPrep, the clinician collects a Pap smear from a woman with the same device, and can rinse it into the ThinPrep vials filled with 20 mLs of fixative solution. With SurePath media, the volume is 10 mLs— so there are twice as many cells per mL in SurePath as there are in ThinPrep. Taking the same volume out of the vial—because the assays are all volume-dependent—yields twice as many cells in one versus the other.
“FIXING” CELLS MAY END UP BREAKING THEM, OVER TIME
Counterbalancing that potentially beneficial increase of cell inputs is that SurePath contains formalin (a solution of formaldehyde that does an excellent job of preserving cells for Pap smears). Depending on how long the cells are exposed to that little bit of formalin alters the way the DNA performs in molecular assays—especially PCR-based assays. With the addition of those counterbalancing effects, some labs began to notice a marked impact.
TESTING PROTOCOL: TAKE YOUR PICK FROM COLUMN A, B, OR C
Because there was no FDA-approved protocol for using the SurePath media for HPV testing, many labs assumed that they could devise one themselves through a particular type of laboratory-developed-test (LDT) validation. Under the circumstances, the test manufacturers were forbidden from providing guidance to the labs. Thus, labs that opted to use SurePath were left to devise a protocol for themselves.
SELF-VALIDATION IN SUREPATH CAN UNWITTINGLY LEAD TO FALSENEGATIVES
Is it possible for a lab—including commercial labs—to perform a clinical validity study for HPV testing in-house with the same degree of scientific rigor as that achieved in a prospective, multicenter US cervical cancer screening trial or FDA approvals?
“The answer is, in my opinion—and some people disagree with me—but the answer is, they cannot,” says Mark H. Stoler, MD, a Professor (Emeritus) of Pathology, Cytology, and Gynecology and associate director of Surgical Pathology and Cytopathology at the University of Virginia School of Medicine. “Because they don’t have the kind of clinical follow-up and systematic disease ascertainment that allows you to properly categorize the false-positives versus the false-negatives. And this is why so much has been written about the clinical validity of HPV testing, and why it’s so hard to get an FDA-approved HPV test.”
Stoler says the kind of data developed for this particular type of LDT is not really possible in the average clinical setting, mainly because not all of the patients who need to return for a colposcopic biopsy (and the manner in which pathologists read the biopsies) leads to erroneous perceptions of test performance. “It has to be in a clinical trial,” Stoler says, while acknowledging that clinical labs that have been offering this test for years all have their own opinions on this.
THE WORRISOME WARNING LETTER HEEDED BY SOME, BUT NOT ALL
The maker of SurePath issued a warning to labs in 2012, cautioning them that using cervical cell samples collected for a Pap test and stored in SurePath for HPV testing might generate false-negative results—which ultimately, could result in not detecting cases of cervical cancer. To wit, how would labs know a falsenegative had been generated?
Some labs stopped using SurePath for HPV testing, others continued to do so, and still others partially switched. Until July 2016, all labs that continued to use SurePath for HPV testing did so without an FDAapproved protocol.
THE SOLUTION
In July 2016, the FDA approved the Roche cobas HPV Test as the first HPV test for use with cervical cell samples collected for a Pap test and stored in SurePath. It can be used in conjunction with a Pap as a co-test to screen women who are 30 and older to establish if further follow-up and diagnostic procedures are warranted, and also, for women who are 21 and older who have had an equivocal Pap test result. And just as with ThinPrep, it can detect high-risk HPV genotypes 16 and 18—behind most cancers that arise from HPV—in the same groups of women. As part of the approval process, Roche conducted an independent clinical trial with more than 900 women, the results of which, complemented by the other data from the ATHENA trial, led to this approval.
Because the FDA required another separate study similar to ATHENA, the cobas HPV Test is not yet approved as a first-line primary screening test for samples collected in SurePath; data from that ongoing trial, when completed, will be submitted to the FDA for approval.
The formalin in SurePath aids the other chemicals in the solution in preservation of the protein and nucleic acids during the fixation process. In that process, chemical cross-linking bonds between the proteins and the DNA and the RNA are formed. Those cross-linking bonds restrict nucleic acid accessibility to detection, especially for samples that have been in solution a long time. Roche developed a preanalytic process that involves a buffer change and then the application of heat to solve this critical problem.
Through the application of a different buffer, a different pH, and/or heat, one can reverse the chemical cross-linking. By doing that, one can then conduct a study to do the kind of comparisons (like in PREDICTORs 42 or in the Roche application) that demonstrate the necessary degree of clinical validity.
LDTS VERSUS CLINICAL TRIALS
Stoler calls the process of using SurePath with a non-FDA-approved HPV test an “LDT-type of modification of an FDA-approved test, and that is very different then a completely home-brew LDT that are potentially even more problematic in the HPV space.”
Labs that were using these types of LDTs, which didn’t want to heed the data or the argument put forth by SurePath’s manufacturer (Beckton Dickinson), don’t have a leg to stand on now, Stoler says. “Except they will argue ‘Well, we’ve clinically validated it to our satisfaction.’ Yet in my opinion, I would say not,” he says. And the labs say it’s less costly to use an LDT. “I don’t care what’s cheaper, I only care about what’s right for the patient. So I would submit that using the FDA-approved protocol(s) you know the clinical performance—where you minimize the variables in the clinical performance, and you absolutely do not know this with your in-house validated LDT despite your belief that you do.”
This is not to say, in an era when LDTs could potentially be highly regulated, that labs can’t bring
What’s done in the typical LDT validation is the lab selects some obvious high-grade cervical intraepithelial neoplasia (CIN) lesions and some cancers and says, “Oh look, it’s our assay is positive,” Stoler says. The lab then selects some normal samples, but usually not enough of either category— it’s a matter of perspective and statistical power, he says. The problem is, labs need a lot of positives, especially positives from patients whose Paps are either normal or at most ASCUS, and they need to make sure they’ve done complete disease ascertainment, Stoler says. “If you just do what you usually do with an LDT—for an assay that doesn’t have a cutoff in the positive range—and indeed most LDT’s don’t have cutoffs in the positive range, they end up overestimating the sensitivity and specificity of the assay. So for many patients, they do an HPV test, and they’re going to tell you you’re HPV-negative. A clinically valid but negative HPV test like Hybrid Capture, or cobas—it doesn’t mean you don’t have HPV.”
This is a critical point. “It means you don’t have HPV at a level that predicts for CIN 3 risks, cancer risks,” Stoler says. “So the negative doesn’t mean negative. Most LDTs in the lab look at hard end points ‘I have the mutation or I don’t have the mutation. I have this infection, or I don’t have this infection.’ It’s a relatively absolute (albeit, never perfect) thing.” Such end-point assays are fine LDT candidates, he says. Because the analytic performance essentially equals the clinical performance. “But disease-predictive assays that have arbitrary cutoffs in the positive range, they’re that are trying to trade off on disease prediction versus not, as opposed to HPV infection versus not, require rigorous clinical trial data and complete disease ascertainment as best as you can do it,” Stoler says. “And that’s what labs can’t do on their own.”
The key thing, he says, is not ultimate anAlytic sensitivity, but proper clinical detection of women at risk of pre-cancer who should be treated for cancer. “While, as much as possible, not detecting irrelevant infections that are extremely common—since it’s the most common STD in the aggregate,” Dr. Schiffman adds.
“I can say it’s a factual reality that the added reassurance of cytology, added to primary HPV testing, is extremely small,” Dr. Schiffman says. “No major country but the United States is considering a lifetime program using both tests.”
THE OFFICIAL WORD FROM THE FDA
In 2015, prior to its approval of the Roche cobas test using SurePath, the FDA published case reports on 20 LDTs that “may have caused or have caused actual harm to patients.” The report notes that in some cases, “LDTs were prone to false-negative results, in which patients’ life-threatening diseases went undetected. As a result, patients failed to receive effective treatments.” The report highlights HPV tests using SurePath collection medium as being among the LDTs that are prone to false-negative results. FDA-approved HPV tests, as part of their approval process, cite specific collection media used for transporting samples to the lab. The report notes that for “women with equivocal Pap test results, all these approved assays have negative predictive values (NPVs) above 99%, so a negative test can avert colposcopy . . . .” The maker of SurePath had not publicly reported the NPV or the PPV (positive predictive value) for any HPV tests used with cervical samples in the SurePath medium, according to the report. It goes on to say that, “Under existing guidelines, a false-negative test result could lead to the absence of patient follow-up and, ultimately, to preventable cancer progression. For this reason, the professional societies that set U.S. cervical cancer screening guidelines specifically recommend against the use of LDTs for cervical cancer screening.”
HOW MIGHT THIS ALL SHAKE OUT IN THE FUTURE?
Many stakeholders are often involved in decisions pertaining to testing. Long-term budgeting for capital expenditures, costly investments in instrumentation, test kits, staff training, and more play a part. Will the FDA or the Centers for Medicare and Medicaid Services (CMS) eventually exert more control over LDTs? Could CMS stop reimbursing labs that perform tests that are not FDA-approved, such as the afore-mentioned LDTs or modifications of approved tests?
Could labs one day be required to submit their self-validation documentation to the FDA before the FDA would recommend to CMS that they get reimbursed for it? Could all of this precipitate a lab’s decision to switch to an FDA-approved platform?
“Eventually, we’re going to get to the right place,” Dr. Stoler says. “Being a patient-centric advocate, ie, doing what is best for patients, the right test at the right time with cost only being a consideration after the medical issues are assured, has to always be the winning strategy.”
REFERENCES
1. Julia C. Gage, Mark Schiffman, Hormuzd A. Katki, et al. Reassurance Against Future Risk of Precancer and Cancer Conferred by a Negative Human Papillomavirus Test. JNCI. 2014;106(8): dju153 doi:10.1093/jnci/dju153
2. Jack Cuzick, Amar S. Ahmad, Janet Austin, et al. A comparison of different human papillomavirus tests in PreservCyt versus SurePath in a referral population—PREDICTORS 4. J Clin Virol. 2016 Sep;82:145–151. doi: 10.1016/j.jcv.2016.06.015