What is a good ppv?

The positive and negative predictive values (PPV and NPV respectively) are the proportions of positive and negative results in statistics and diagnostic tests that are true positive and true negative results, respectively.[1] The PPV and NPV describe the performance of a diagnostic test or other statistical measure. A high result can be interpreted as indicating the accuracy of such a statistic. The PPV and NPV are not intrinsic to the test (as true positive rate and true negative rate are); they depend also on the prevalence.[2] Both PPV and NPV can be derived using Bayes' theorem.

Although sometimes used synonymously, a positive predictive value generally refers to what is established by control groups, while a post-test probability refers to a probability for an individual. Still, if the individual's pre-test probability of the target condition is the same as the prevalence in the control group used to establish the positive predictive value, the two are numerically equal.

In information retrieval, the PPV statistic is often called the precision.

The positive predictive value (PPV), or precision, is defined as

where a "true positive" is the event that the test makes a positive prediction, and the subject has a positive result under the gold standard, and a "false positive" is the event that the test makes a positive prediction, and the subject has a negative result under the gold standard. The ideal value of the PPV, with a perfect test, is 1 (100%), and the worst possible value would be zero.

The PPV can also be computed from sensitivity, specificity, and the prevalence of the condition:

cf. Bayes' theorem

The complement of the PPV is the false discovery rate (FDR):

The negative predictive value is defined as:

where a "true negative" is the event that the test makes a negative prediction, and the subject has a negative result under the gold standard, and a "false negative" is the event that the test makes a negative prediction, and the subject has a positive result under the gold standard. With a perfect test, one which returns no false negatives, the value of the NPV is 1 (100%), and with a test which returns no true negatives the NPV value is zero.

The NPV can also be computed from sensitivity, specificity, and prevalence:

The complement of the NPV is the false omission rate (FOR):

Although sometimes used synonymously, a negative predictive value generally refers to what is established by control groups, while a negative post-test probability rather refers to a probability for an individual. Still, if the individual's pre-test probability of the target condition is the same as the prevalence in the control group used to establish the negative predictive value, then the two are numerically equal.

The following diagram illustrates how the positive predictive value, negative predictive value, sensitivity, and specificity are related.

Note that the positive and negative predictive values can only be estimated using data from a cross-sectional study or other population-based study in which valid prevalence estimates may be obtained. In contrast, the sensitivity and specificity can be estimated from case-control studies.

Suppose the fecal occult blood (FOB) screen test is used in 2030 people to look for bowel cancer:

The small positive predictive value (PPV = 10%) indicates that many of the positive results from this testing procedure are false positives. Thus it will be necessary to follow up any positive result with a more reliable test to obtain a more accurate assessment as to whether cancer is present. Nevertheless, such a test may be useful if it is inexpensive and convenient. The strength of the FOB screen test is instead in its negative predictive value — which, if negative for an individual, gives us a high confidence that its negative result is true.

Note that the PPV is not intrinsic to the test—it depends also on the prevalence.[2] Due to the large effect of prevalence upon predictive values, a standardized approach has been proposed, where the PPV is normalized to a prevalence of 50%.[12] PPV is directly proportional[dubious – discuss] to the prevalence of the disease or condition. In the above example, if the group of people tested had included a higher proportion of people with bowel cancer, then the PPV would probably come out higher and the NPV lower. If everybody in the group had bowel cancer, the PPV would be 100% and the NPV 0%.

To overcome this problem, NPV and PPV should only be used if the ratio of the number of patients in the disease group and the number of patients in the healthy control group used to establish the NPV and PPV is equivalent to the prevalence of the diseases in the studied population, or, in case two disease groups are compared, if the ratio of the number of patients in disease group 1 and the number of patients in disease group 2 is equivalent to the ratio of the prevalences of the two diseases studied. Otherwise, positive and negative likelihood ratios are more accurate than NPV and PPV, because likelihood ratios do not depend on prevalence.

When an individual being tested has a different pre-test probability of having a condition than the control groups used to establish the PPV and NPV, the PPV and NPV are generally distinguished from the positive and negative post-test probabilities, with the PPV and NPV referring to the ones established by the control groups, and the post-test probabilities referring to the ones for the tested individual (as estimated, for example, by likelihood ratios). Preferably, in such cases, a large group of equivalent individuals should be studied, in order to establish separate positive and negative predictive values for use of the test in such individuals.

Bayes' theorem confers inherent limitations on the accuracy of screening tests as a function of disease prevalence or pre-test probability. It has been shown that a testing system can tolerate significant drops in prevalence, up to a certain well-defined point known as the prevalence threshold, below which the reliability of a positive screening test drops precipitously. That said, Balayla et al.[13] showed that sequential testing overcomes the aforementioned Bayesian limitations and thus improves the reliability of screening tests. For a desired positive predictive value ρ {\displaystyle \rho } that approaches some constant k {\displaystyle k} , the number of positive test iterations n i {\displaystyle n_{i}} needed is:

where

Of note, the denominator of the above equation is the natural logarithm of the positive likelihood ratio (LR+).

PPV is used to indicate the probability that in case of a positive test, that the patient really has the specified disease. However, there may be more than one cause for a disease and any single potential cause may not always result in the overt disease seen in a patient. There is potential to mix up related target conditions of PPV and NPV, such as interpreting the PPV or NPV of a test as having a disease, when that PPV or NPV value actually refers only to a predisposition of having that disease.

An example is the microbiological throat swab used in patients with a sore throat. Usually publications stating PPV of a throat swab are reporting on the probability that this bacterium is present in the throat, rather than that the patient is ill from the bacteria found. If presence of this bacterium always resulted in a sore throat, then the PPV would be very useful. However the bacteria may colonise individuals in a harmless way and never result in infection or disease. Sore throats occurring in these individuals are caused by other agents such as a virus. In this situation the gold standard used in the evaluation study represents only the presence of bacteria (that might be harmless) but not a causal bacterial sore throat illness. It can be proven that this problem will affect positive predictive value far more than negative predictive value.[14] To evaluate diagnostic tests where the gold standard looks only at potential causes of disease, one may use an extension of the predictive value termed the Etiologic Predictive Value.[15][16]

The ideal value of the PPV, with a perfect test, is 1 (100%), and the worst possible value would be zero. The PPV can also be computed from sensitivity, specificity, and the prevalence of the condition: cf.

The aim of this article is to help provide an understanding of sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) in an intuitive and comprehensible format.

Sensitivity and specificity are characteristics of a test.

Positive predictive value (PPV) and negative predictive value (NPV) are best thought of as the clinical relevance of a test.

The significant difference is that PPV and NPV use the prevalence of a condition to determine the likelihood of a test diagnosing that specific disease. Whereas sensitivity and specificity are independent of prevalence.

Prevalence is the number of cases in a defined population at a single point in time and is expressed as a decimal or a percentage.

Sensitivity is the percentage of true positives (e.g. 90% sensitivity = 90% of people who have the target disease will test positive).

Specificity is the percentage of true negatives (e.g. 90% specificity = 90% of people who do not have the target disease will test negative).

These allow you to rule conditions in or out but not definitively diagnose a condition.

A classic table that allows sensitivity and specificity to be worked out quantitatively can be seen below.

The sensitivity of a test is the proportion of people who test positive among all those who actually have the disease.

A sensitive test helps rule out a disease when the test is negative (e.g. negative amylase in pancreatitis). Highly SeNsitive = SNOUT = rule out.

Sensitivity can be thought of as ‘how delicate/sensitive the test is to picking up little changes’. The test for amylase is highly sensitive because it is capable of picking up very small amounts of amylase in the blood. As a result, the chance of amylase being present that is “below the threshold for detection” is small. Therefore, a negative result would mean one of two things. Firstly, that amylase is present but in such small quantities that it is undetectable by the test (unlikely because this test picks up small changes). Secondly, that amylase is not present at all (more likely).

This example works because the disease (pancreatitis) has a trait (amylase) that is almost always present and the test looks for that trait. If the trait is not present, the disease is unlikely to be present and can be ruled out.

The specificity of a test is the proportion of people who test negative among all those who actually do not have that disease.

A specific test helps rule a disease in when positive (e.g. urine dipstick for nitrites in UTI). Highly SPecific = SPIN = rule in.

If a disease (UTI) has a trait (nitrites in urine) that is rare in other diseases, a test for that trait can be thought of as being highly specific because the trait is specific to that disease. However, a positive result would not mean they definitely have a UTI because a highly specific test does not factor in how common the disease is (prevalence).

Positive predictive value (PPV) and negative predictive value (NPV) are directly related to prevalence and allow you to clinically say how likely it is a patient has a specific disease.

The positive predictive value is the probability that following a positive test result, that individual will truly have that specific disease.

The negative predictive value is the probability that following a negative test result, that individual will truly not have that specific disease.

For any given test (i.e. sensitivity and specificity remain the same) as prevalence decreases, the PPV decreases because there will be more false positives for every true positive. This is because you’re hunting for a “needle in a haystack” and likely to find lots of other things that look similar along the way – the bigger the haystack, the more frequently you mistake things for a needle.

Therefore, as prevalence decreases, the NPV increases because there will be more true negatives for every false negative. This is because a false negative would mean that a person actually has the disease, which is unlikely because the disease is rare (low prevalence).

Examples of how PPV and NPV could vary with prevalence for a specific test can be seen below.

Hopefully, with the help of this article, the concepts of sensitivity, specificity, PPV and NPV are now clearer. The examples given should allow you to see how and why these vary as different factors change.

This blog will dive deeper into positive predictive values (PPVs) and negative predictive values (NPVs) and using them as a way to consider setting limits for sensitivity and specificity. The used definitions are shown in the standard 2x2 table (Table 1), commonly used for comparing a new qualitative assay method to a reference method or clinical truth.

Table 1: Standard 2x2 table used to compare a new assay to a reference method

Where:

Definitions of key performance statistics:

One can see from this 2x2 table, sensitivity and specificity are independent of prevalence. Let’s take a closer look at this with an example that demonstrates the independence.

If we were performing a study to estimate sensitivity and specificity, the underlying expected performance of sensitivity and specificity remains the same if the prevalence goes from 50% to 1% (Table 2a and 2b).

Table 2: Examples for sensitivity and specificity with A.) a prevalence of 50% and B.) a prevalence of 1%

It’s important to remember that sensitivity and specificity look at the performance relative to the clinical truth of the patient or refence assay. If the patient is positive or negative, what percent of the time does this assay get it right (sensitivity and specificity) relative to the clinical truth or the reference assay?

Positive and negative predictive value look at the 2x2 table from the direction of the new assay’s result. The predictive values are, given a positive (or negative) result from the new assay, the percentage of the time the new assay is correct relative to the number of positive results the new assay reports? Or in other words, what percentage of the new assay’s positive (or negative) results are actually positive (or negative) clinically or with the reference assay? Where PPV is an estimate for positive results and NPV is the same for negative results, from the new assay’s perspective.

These are key pieces of information a clinician needs to determine the next step in the diagnostic pathway. The test’s result is positive (or negative), how likely is that to be true?

For example, the set of 2x2 tables above (Test A and Test B, Table 2a and 2b) have exactly the same sensitivity and specificity (95% and 99% respectively) with Test A having prevalence of 50% and Test B having a prevalence of 1%. The PPV goes from 99% with a 50% prevalence down to 49% with a 1% prevalence. A PPV of 99% indicates that with a positive assay result there’s a 99% chance of it being correct. Likewise, with a 49% PPV, there is only a 49% chance that the patient is actually positive. Depending on the intended use of the product, one, both, or neither of these predictive values might be sufficient. The important thing to remember is that PPV (and NPV) show how likely a new assay test result is correct.

As the predictive values involve the sensitivity and specificity of the assay and the prevalence of the disease or condition in the intended use population one can see that different intended purposes with the same assay can result in vastly different predictive value performances. When determining the requirements for sensitivity and specificity of an assay it is critical to look at the expected prevalence to estimate the expected PPV and NPV and review that in the context of the intended use.

There are different uses or purposes for IVDs, e.g. screening, diagnostic, prognostic, etc. How the clinician will use the assay’s test result should drive the requirements for PPV and NPV. How likely an incorrect result is to occur when using the new assay and the different risks for false positives and false negatives must be considered. How low can the PPV and NPV be and still have a positive benefit-risk for the use of this new assay? This in turn will drive the requirements for sensitivity and specificity, given an estimated prevalence in the intended use population.

For example, let’s assume there is a need for an assay, where the PPV must be ≥ 90% and the NPV ≥ 99%, with an expected prevalence of 20%. This translates to needing a high level of confidence when the new assay’s result is negative and the clinicians can tolerate a few more new assay positives that are false. Here’s one set of sensitivity and specificity that meet the PPV and NPV requirements at 20% prevalence (Table 3).

Table 3: Set of PPVs and NPVs per prevalence for a 96% sensitivity and 98% specificity