1、Predictive Values,Bob L. Larson, DVM, PhD University of Missouri Commercial Agriculture, Beef Focus Team,Use and Interpretation of Diagnostic Tests,Help to make a diagnosis (does not stand alone) Judge severity of disease Predict subsequent clinical course of disease Estimate likely responsiveness t
2、o therapy Determine response to therapy (post therapy),Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic Tests,Must be tested against a “gold standard” Although no perfect test exists, a gold standard is an accepted, reliable test Properties of di
3、agnostic tests Sensitivity = proportion of true positives that test positive How effective is the test at identifying animals with the condition? Specificity = proportion of true negatives that test negative How effective is the test at identifying animals without the condition?,Measurement of disea
4、se events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic Tests,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic Tests,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic
5、 Tests,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic Tests,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic Tests,Measurement of disease events Infectious disease epidemiology Interpreting
6、diagnostic tests,Diagnostic Tests,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic Tests,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic Tests,Sensitivity = a/(a+c) Specificity = d/(b+d),Fals
7、e negative = c/(a+c),Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic Tests,Sensitivity = a/(a+c) Specificity = d/(b+d),False negative = c/(a+c) False positive = b/(b+d),Measurement of disease events Infectious disease epidemiology Interpreting d
8、iagnostic tests,Diagnostic tests use an arbitrary cut-off to separate two populations (normal vs. abnormal),Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic tests use an arbitrary cut-off to separate two populations (normal vs. abnormal),Measurem
9、ent of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic tests use an arbitrary cut-off to separate two populations (normal vs. abnormal),Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic tests use an arbitrar
10、y cut-off to separate two populations (normal vs. abnormal),Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic tests use an arbitrary cut-off to separate two populations (normal vs. abnormal),Measurement of disease events Infectious disease epidemi
11、ology Interpreting diagnostic tests,Diagnostic tests use an arbitrary cut-off to separate two populations (normal vs. abnormal),Where cut-off is placed determines sensitivity and specificity,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic tests
12、use an arbitrary cut-off to separate two populations (normal vs. abnormal),Where cut-off is placed determines sensitivity and specificity,Normal,Abnormal,Few False Negatives High Sensitivity,- result,+ result,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests
13、,Diagnostic tests use an arbitrary cut-off to separate two populations (normal vs. abnormal),Where cut-off is placed determines sensitivity and specificity,Normal,Abnormal,Few False Positives High Specificity,- result,+ result,Measurement of disease events Infectious disease epidemiology Interpretin
14、g diagnostic tests,Diagnostic tests use an arbitrary cut-off to separate two populations (normal vs. abnormal),Where cut-off is placed determines sensitivity and specificity,Normal,Abnormal,Trade off between False and False +,- result,+ result,Measurement of disease events Infectious disease epidemi
15、ology Interpreting diagnostic tests,Diagnostic tests use an arbitrary cut-off to separate two populations (normal vs. abnormal),Test with High Sensitivity,Normal,Abnormal,A negative result allows you to rule-out whatever you are testing,- result,+ result,SnNout,Measurement of disease events Infectio
16、us disease epidemiology Interpreting diagnostic tests,Diagnostic tests use an arbitrary cut-off to separate two populations (normal vs. abnormal),Test with High Specificity,Normal,Abnormal,A positive result allows you to rule-in whatever you are testing,- result,+ result,SpPin,Measurement of disease
17、 events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic Tests,Sensitivity and specificity are inversely related Sensitivity and specificity are properties of the diagnostic test and tell us if the test is valid Test sensitivity and specificity have little effect on the confi
18、dence a veterinarian has in an individual result.,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Because ,My illustration is not consistent with true disease dynamics in a population (the real world),Measurement of disease events Infectious disease epidem
19、iology Interpreting diagnostic tests,Because ,My illustration is not consistent with true disease dynamics in a population (the real world),Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Predictive Value What we are really interested in,Predictive values
20、tell us if a test is useful Positive Predictive Value (PPV) The probability that an animal with a positive test result is truly positive Negative Predictive Value (NPV) The probability that an animal with a negative test result is truly negative Dependent on disease prevalence and test sensitivity a
21、nd specificity,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,PPV and NPV,Assume a test for bovine Johnes is 60% sensitive and 90% specific On examination of a 5 year old dairy cow, with typical history, clinical signs and positive herd-mates We assume 90
22、% of cows with this description are positive (prevalence = 90%),Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,PPV and NPV,Assume a test for bovine Johnes is 60% sensitive and 90% specific On examination of a 5 year old dairy cow, with typical history, cl
23、inical signs and positive herd-mates We assume 90% of cows with this description are positive (prevalence = 90%),Test Sensitivity = 540/900 = 60%,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,PPV and NPV,Assume a test for bovine Johnes is 60% sensitive a
24、nd 90% specific On examination of a 5 year old dairy cow, with typical history, clinical signs and positive herd-mates We assume 90% of cows with this description are positive (prevalence = 90%),Test Sensitivity = 60%,Measurement of disease events Infectious disease epidemiology Interpreting diagnos
25、tic tests,PPV and NPV,Assume a test for bovine Johnes is 60% sensitive and 90% specific On examination of a 5 year old dairy cow, with typical history, clinical signs and positive herd-mates We assume 90% of cows with this description are positive (prevalence = 90%),Test Sensitivity = 60%,NPV = 90/4
26、50 = 20%,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,PPV and NPV,Assume a test for bovine Johnes is 60% sensitive and 90% specific On examination of a 5 year old dairy cow, with typical history, clinical signs and positive herd-mates We assume 90% of c
27、ows with this description are positive (prevalence = 90%),Test Sensitivity = 60%,NPV = 90/450 = 20%,Test Specificity = 90%,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,PPV and NPV,Assume a test for bovine Johnes is 60% sensitive and 90% specific On exam
28、ination of a 5 year old dairy cow, with typical history, clinical signs and positive herd-mates We assume 90% of cows with this description are positive (prevalence = 90%),PPV = 540/550 = 98%,NPV = 90/450 = 20%,With positive test we are 98% sure she is a true positive,With negative test we are 20% s
29、ure she is a true negative,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,PPV and NPV,Assume a test for bovine Johnes is 60% sensitive and 90% specific On examination of a 5 year old beef cow, with typical history, clinical signs and unknown herd status W
30、e assume 50% of cows with this description are positive (prevalence = 50%),Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,PPV and NPV,Assume a test for bovine Johnes is 60% sensitive and 90% specific On examination of a 5 year old beef cow, with typical h
31、istory, clinical signs and unknown herd status We assume 50% of cows with this description are positive (prevalence = 50%),Test Sensitivity = 300/500 = 60%,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,PPV and NPV,Assume a test for bovine Johnes is 60% s
32、ensitive and 90% specific On examination of a 5 year old beef cow, with typical history, clinical signs and unknown herd status We assume 50% of cows with this description are positive (prevalence = 50%),Test Sensitivity = 60%,Measurement of disease events Infectious disease epidemiology Interpretin
33、g diagnostic tests,PPV and NPV,Assume a test for bovine Johnes is 60% sensitive and 90% specific On examination of a 5 year old beef cow, with typical history, clinical signs and unknown herd status We assume 50% of cows with this description are positive (prevalence = 50%),Test Sensitivity = 60%,NP
34、V = 450/650 = 69%,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,PPV and NPV,Assume a test for bovine Johnes is 60% sensitive and 90% specific On examination of a 5 year old beef cow, with typical history, clinical signs and unknown herd status We assume
35、50% of cows with this description are positive (prevalence = 50%),Test Sensitivity = 60%,NPV = 450/650 = 69%,Test Specificity = 90%,PPV = 300/350 = 86%,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,PPV and NPV,Assume a test for bovine Johnes is 60% sensi
36、tive and 90% specific On examination of a 5 year old beef cow, with typical history, clinical signs and unknown herd status We assume 50% of cows with this description are positive (prevalence = 50%),PPV = 300/350 = 86%,NPV = 450/650 = 69%,With positive test we are 86% sure she is a true positive,Wi
37、th negative test we are 69% sure she is a true negative,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Predictive Value of a Diagnostic Test PPV and NPV,PPV and NPV are greatly influenced by pre-test likelihood of disease Published prevalence data Your cl
38、inical skills and judgment,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,PPV and NPV,The immunohistochemistry test for persistently infected BVD animals is very good: 96% sensitive and 99% specific However, the prevalence of PI animals in the general cal
39、f population is low (0.3%) So, how does a very good test perform in a population with low prevalence?,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,PPV and NPV,The immunohistochemistry test for persistently infected BVD animals is very good: 96% sensitiv
40、e and 99% specific However, the prevalence of PI animals in the general calf population is low (0.3%) So, how does a very good test perform in a population with low prevalence?,Test Sensitivity = 96%,Test Specificity = 99%,Measurement of disease events Infectious disease epidemiology Interpreting di
41、agnostic tests,PPV and NPV,The immunohistochemistry test for persistently infected BVD animals is very good: 96% sensitive and 99% specific However, the prevalence of PI animals in the general calf population is low (0.3%) So, how does a very good test perform in a population with low prevalence?,Te
42、st Sensitivity = 96%,NPV = 98,703/98,715 = 99.9%,Test Specificity = 99%,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,PPV and NPV,The immunohistochemistry test for persistently infected BVD animals is very good: 96% sensitive and 99% specific However, th
43、e prevalence of PI animals in the general calf population is low (0.3%) So, how does a very good test perform in a population with low prevalence?,Test Sensitivity = 96%,NPV = 98,703/98,715 = 99.9%,Test Specificity = 99%,PPV = 288/1,285 = 22%,Measurement of disease events Infectious disease epidemio
44、logy Interpreting diagnostic tests,PPV and NPV,The immunohistochemistry test for persistently infected BVD animals is very good: 96% sensitive and 99% specific However, the prevalence of PI animals in the general calf population is low (0.3%) So, how does a very good test perform in a population wit
45、h low prevalence?,Test Sensitivity = 96%,NPV = 98,703/98,715 = 99.9%,Test Specificity = 99%,PPV = 288/1,285 = 22%,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,PPV and NPV,= 100 head,PPV and NPV,Evaluating a component of the physical examination Observat
46、ion for present of dyspnea Auscultation with stethoscope Etc.,Test Sensitivity = 95%,Test Specificity = 96%,Pregnancy Determination via Rectal Palpation Assume high sensitivity and specificity (truth?),Prevalence is 95%,PPV = 99.8%,NPV = 50%,Measurement of disease events Infectious disease epidemiol
47、ogy Interpreting diagnostic tests,PPV and NPV,Take Home,As prevalence falls, PPV (positive predictive value) must also fall and NPV (negative predictive) value must rise, regardless of sensitivity and specificity of the test. As prevalence increases, PPV must increase and NPV must fall, regardless o
48、f the sensitivity and specificity of the test. Your clinical judgment often sets the pre-test probability (prevalence),Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic Testing Strategy,Must know what you are asking Ruling-out a preliminary diagno
49、sis Ruling-in a preliminary diagnosis Screening for regulatory or biosecurity reasons Describing a herd production situation,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,Diagnostic Testing Strategy,Must understand predictive value of test in the population you are testing,Must know what you are asking Ruling-out a preliminary diagnosis Ruling-in a preliminary diagnosis Screening for regulatory or biosecurity reasons Describing a herd production situation,Measurement of disease events Infectious disease epidemiology Interpreting diagnostic tests,
