In a seminal 1989 article published in the New England Journal of Medicine, the American nephrologist Jerome P. Kassirer wrote, “Absolute certainty in diagnosis is unattainable, no matter how much information we gather, how many observations we make or how many tests we perform.”
A British National Health System survey in 2009 reported that 15% of its patients were misdiagnosed, and according to a study published in 2014, each year in the US, approximately 12 million adults (5% of the total adult patient population) who sought outpatient medical care were misdiagnosed in hospital settings, in outpatient clinics and doctors’ offices.
Thus, the quantum of clinical diagnostic errors may be huge, and its impact may be severe.
During the ongoing COVID-19 pandemic, there is a lot of buzz regarding the possible errors in diagnoses with both RT-PCR tests and the faster antibody-based tests, all over the world. To understand how serious these errors might be during a pandemic, we need to understand the nature of different types of errors.
When a new test is rapidly created and deployed, as in the case of the current coronavirus, its accuracy cannot be exactly predicted beforehand. A test developed under controlled lab conditions might behave in a different way when applied in the real world, and this might enhance the likelihood of errors due to many unforeseen events.
In reality, we encounter four types of scenarios:
1.True positive: A person with COVID-19 tests positive for COVID-19
2.False positive: A person without COVID-19 tests positive for COVID-19
3.False negative: A person with COVID-19 tests negative for COVID-19
4.True negative: A person without COVID-19 tests negative for COVID-19
While (1) and (4) are desirable situations, (2) and (3) types of testing errors.
Now, the ratio of (1) + (4) to (1) + (2) + (3) + (4) obviously represents the proportion of correct results from the test, and is a measure called accuracy. The ratio of (1) to (1) + (2) is the proportion of correct diagnoses out of all those who tested positive, and is a measure called precision.
Note that both (1) and (3) both represent people who actually have COVID-19. The likelihood of a test result coming back positive when the person actually has COVID-19 is known as sensitivity. It is equal to the ratio of (1) to (1) + (3). Finally, the specificity of the test is the ratio of (4) to (2) + (4). Here both (2) and (4) stand for people who do not have COVID-19, so specificity is the likelihood of a test result coming back negative when COVID-19 is absent in the person.
A quick recap:
Accuracy = true positives + true negatives / all results
Precision = true positives / true positives + false positives
Sensitivity = true positives / true positives + false negatives
Specificity = true negatives / true negatives + false positives
Different values of the sensitivity of RT-PCR tests to COVID-19 have been reported in different parts of the globe. According to one analysis with 51 people in China, up to 29% of people with the coronavirus tested negative. Studies in the US returned multiple values – sometimes it was 95%, sometimes 85%, and even 75%.
If the sensitivity of one RT-PCR test kit is 90%, for example, and two successive negative test results are used to declare someone free of disease, there is still a 1% chance that a person with the disease would be declared negative. And if that is so, about 200,000 COVID-19 patients could have been wrongly diagnosed among the more than 20 million tests worldwide! This may be sufficient ground for quarantining patients with ‘negative’ test results for the recommended period (say, 14 days) in order to restrict the virus’s spread.
Antibody tests could help to find people who can be presumed to be immune (our experience with other viruses is that after the first infection has been defeated, a body becomes immune to the causative pathogen for a certain period; we still need studies to confirm this is true of the new coronavirus as well). But at the moment, we don’t have enough information on the accuracy of antibody tests. The very limited data from other countries suggests that such tests might have fewer false negative results than RT-PCR tests but more false positives.
Usually, researchers design their tests to be as sensitive and as specific as possible – but despite their best efforts, Kassirer’s statement holds: no test can be 100% accurate. So when a positive (or negative) result is obtained, what is the probability of having (or not having) COVID-19?
An 18th century concept in statistics, known as Bayes’s theorem, can help us. This theorem tells us how to calculate the probability of an event given that another event has happened. For example, say people in a particular colony are being tested, and 20% of them actually have the disease. Next, say the sensitivity (probability of a positive result given the disease is present) of the test being used is 80% and its specificity (probability of a negative test result given the disease is not present) is 90%. A little bit of math yields the following probabilities:
True positive = 0.16
False positive = 0.08
False negative = 0.04
True negative = 0.72
According to Bayes’s theorem, the probability that the disease is present given a negative test result can be obtained by multiplying the probability of disease in the locality (0.20) and the probability of a negative result given the disease is present (0.20), then dividing this by the probability of a negative test result (0.76). This value comes out to be 5.26%. That is, a little more than 1 in 20 people who test negative may actually have the disease. Similarly, the probability of ‘no disease’ given a positive test result is 33.3%.
If the disease’s prevalence in the colony rises to 50%, these two figures become 18.2% and 11.1%, respectively. If the prevalence increases to 80%, these figures become 47.1% and 3%, respectively.
In his 2015 book The Laws of Medicine: Field Notes From an Uncertain Science, Siddhartha Mukherjee, the Pulitzer Prize-winning author and one of the world’s foremost cancer researchers, wrote that “a strong intuition is much more powerful than a weak test”. Thus, a test result for a person should not depend on the accuracy of the test alone but also on the estimated risk of disease before testing.
No comments:
Post a Comment