Since last week, when the Stanford team posted a non-peer-reviewed preprint online, they’ve stirred debate with their findings: that as of early April, in contrast to the 956 cases reported in Santa Clara County, there were an estimated 48,000 to 81,000 infections that had gone undetected. And while the death rate from confirmed cases in the US is over 5%, due in part to a lack of diagnostic testing, the researchers said their results suggested the true rate, from diagnosed and undiagnosed cases, was drastically lower: between 0.12% to 0.2%.
Their study, based on tests administered to 3,330 people, is one of the biggest antibody surveys in the US so far. Scientists around the world, from New York to Germany to Italy, are counting on such tests to help answer one of the most important questions about the virus: how widespread, exactly, it is. Results from these early surveys vary widely.
Antibody tests, also known as serology tests, differ from the nasal- and throat-swab tests that can spot a current infection. Instead, these look for signs in the blood that the immune system had fought off an infection in the past. Government officials are touting them as key to reopening the economy and helping identify who is safe from reinfection and safe to go back to work.
But there are limits to what antibodies can reveal. With other infectious diseases, antibodies signal a measure of protection against getting sick again. No one knows if that is true of this virus, or, if it is, how long such immunity might last.
There are also potential problems with the antibody tests themselves. To quickly increase availability, the FDA is allowing tests to be sold without the agency independently validating their accuracy claims. In the US, only four tests have “emergency use authorization” indicating the agency has reviewed their data — and more than 130 others have unvalidated accuracy rates.
One of these unauthorized tests was used for the Santa Clara County study, as well as for a recent, similar study in Los Angeles County.
But Su’s email to parents at Egan Junior High, which is attended by more than 650 seventh- and eighth-graders, falsely described the test as an “FDA approved antibody test for Coronavirus.”
In fact, the test was distributed by Premier Biotech in Minnesota, and made by Hangzhou Biotest Biotech of China. As NBC News has reported, China recently banned the test, along with other COVID-19 tests, from being exported because it hadn’t been vetted by China’s equivalent of the FDA. (The tests used in the two studies were shipped to the US legally, before the ban, according to Wired.)
Bhattacharya did not respond to questions addressing the false claims made about the test.
Then there’s the question of how to accurately calculate a virus’ prevalence in a community. To do so, it helps to have underlying test samples from people who are representative of their community as a whole.
If, for instance, tests are disproportionately done on people who were sick in the past, their positive samples can skew the prevalence estimates higher than the true number.
The April 2 email asked for “HEALTHY volunteer[s].” But it also mentioned that “the disease likely entered the California in December (based on 2019 travel data),” due to travel between the US and Wuhan, China, at that time. And the email explained that “the serum antibody test determines whether your immune system has fought off the virus and created antibodies to protect you from future exposure.”
Those statements might make the study sound especially appealing to people who had COVID-19 symptoms in the last few months, but weren’t able to get tested.
“It’s troubling in that it seems the wording might bias people to be more likely to participate in the study if they thought they have antibodies against the virus,” Kilpatrick said.
Bhattacharya said that the issue of sample selection bias would be addressed in the forthcoming revision.
In their preprint, the researchers said that they’d recruited participants through locally targeted Facebook ads, which led people to a survey that “provided information” about the study. They didn’t share the ads’ language.
The study ended up recruiting a group that wasn’t representative of the county as a whole in some ways. Some zip codes were more represented than others — with zip codes in Los Altos making up some of the highest concentrations of participants, according to a figure in the preprint. Bhattacharya acknowledged that the email had contributed to this effect, but argued that he and his team had corrected for the imbalance in their statistical analysis by adjusting for the difference.
At the bottom of the email, Su explained that participating would “contribute to knowledge of the prevalence of virus spread in Santa Clara County.”
For further reading, she linked to a March 24 Wall Street Journal op-ed that argued the number of true infections from the coronavirus was likely much higher than reported, and its fatality rate was therefore likely to be much lower. The authors were two Stanford professors, one of them her husband.
This content was originally published here.