1、,Information Collection - Key Strategy,Motivation To reduce uncertainty which makes us choose “second best” solutions as insurance Concept Insert an information-gathering stage (e.g., a test) before decision problems, as an option,D,Decision Problem,Test,Decision Problem,Operation of Test,EV (after
2、test) EV (without test) Why? Because we can avoid bad choices and take advantage of good ones, in light of test results Question: Since test generally has a cost, is the test worthwhile? What is the value of information? Does it exceed the cost of the test?,Value of Information - Essential Concept,V
3、alue of information is an expected value Expected value after test “k”= pk(Dk*)Pk = probablility, after test k, of an observation which will lead to an optimal decision (incorporating revised probabilities due to observation) Dk* Expected Value of information = EV (after test) - EV (without test) =
4、pk(Dk*) - pk(Ej)Oij,k,k,k,Expected Value of Perfect Information - EVPI,Perfect information is a hypothetical concept Use: Establishes an upper bound on value of any test Concept: Imagine a “perfect” test which indicated exactly which Event, Ej, will occur By definition, this is the “best” possible i
5、nformation Therefore, the “best” possible decisions can be made Therefore, the EV gain over the “no test” EV must be the maximum possible - an upper limit on the value of any test!,EVPI Example,Question: Should I wear a raincoat? RC - Raincoat; RC - No Raincoat Two possible Uncertain Outcomes (p = 0
6、.4) or No Rain (p = 0.6),Remember that better choice is to take raincoat, EV = 0.8,EVPI Example (continued),EV (after test) = 0.4(5) + 0.6(4) = 4.4 EVPI = 4.4 - 0.8 = 3.6,Application of EVPI,A major advantage: EVPI is simple to calculate Notice: Prior probability of the occurrence of the uncertain e
7、vent must be equal to the probability of observing the associated perfect test result As a “perfect test”, the posterior probabilities of the uncertain events are either 1 ot 0 Optimal choice generally obvious, once we “know” what will happen Therefore, EVPI can generally be written directly No need
8、 to use Bayes Theorem,Expected Value of Sample Information - EVSI,Sample information are results taken from an actual test 0 pjk Calculate best decision Dk* for each test result TRk (a k-fold repetition of the original decsion problem) Calculate EV (after test) = pk(Dk*)Calculate EVSI as the differe
9、nce between EV (after test) - EV (without test) A BIG JOB,k,EVSI Example,Test consists of listening to forecasts Two possible test results Rain predicted = RP Rain not predicted = NRP Assume the probability of a correct forecast = 0.7 p(RP/R) = P(NRP/NR) = 0.7 P(NRP/R) = P(RP/NR) = 0.3 First calcula
10、tion: probabilities of test results P(RP) = p(RP/R) p(R) + P(RP/NR) p(NR) = (0.7) (0.4) + (0.3) (0.6) = 0.46P(NRP) = 1.00 - 0.46 = 0.54,EVSI Example (continued 2 of 5),Next: Posterior ProbabilitiesP(R/RP) = p(R) (p(RP/R)/p(RP) = 0.4(0.7/0.46) = 0.61 P(NR/NRP) = 0.6(0.7/0.54) = 0.78 Therefore, p(NR/R
11、P) = 0.39 & p(R/RNP) = 0.22,EVSI Example (continued 3 of 5),Best decisions conditional upon test results,EV (RC) = (0.61) (5) + (0.39) (-2) = 2.27 EV (RC) = (0.61) (-10) + (0.39) (4) = -4.54,EVSI Example (continued 4 of 5),EV (RC) = (0.22) (5) + (0.78) (-2) = -0.48 EV (RC) = (0.22) (-10) + (0.78) (4
12、) = 0.92,Best decisions conditional upon test results,EVSI Example (continued 5 of 5),EV (after test) = p(rain pred) (EV(strategy/RP) + P(no rain pred) (EV(strategy/NRP) = 0.46 (2.27) + 0.54 (0.92) = 1.54 EVSI = 1.54 - 0.8 = 0.74 EVPI,Practical Example - Is a Test Worthwhile?,If value is Linear (i.e
13、., probabilistic expectations correctly represent valuation of outcomes under uncertainty) Calculate EVPI If EVPI 50% EVPI Reject test (Real test are not close to perfect) Calculate EVSI EVSI cost of test Reject test Otherwise, accept test,Is Test Worthwhile? (continued),If Value Non-Linear (i.e., p
14、robabilistic expectation of value of outcomes does NOT reflect attitudes about uncertainty) Theoretically, cost of test should be deducted from EACH outcome that follows a test If cost of test is known A) Deduct costs B) Calculate EVPI and EVSI (cost deducted) C) Proceed as for linear EXCEPT Question is if EVPI(cd) or EVSI(cd) 0? If cost of test is not known A) Iterative, approximate pragmatic approach must be used B) Focus first on EVPI C) Use this to estimate maximum cost of a test,
