On the Foundation of Guidelines for Health Economic Assessment

In recent years, there has been increased interest in setting up guidelines for carrying out cost-effectiveness analysis of medical interventions, and some such guidelines have indeed been established. In the paper, we present a model of information retrievement and use in which we can studie the role of guidelines. The main result, which is a version of the well-known theorem of Blackwell (1948), shows that in cases where there are sufficiently many decisions to be made on the basis of the information obtained, there can be no objective ranking of methods, except the trivial one stating that more information is better than less. The consequence is that guidelines, and the very detailed version known as the reference case approach, may have administrative advantages but can be harmful when considered as an aid towards better decisions.


Introduction
In the course of the last decade, the use of cost-effectiveness analysis as an aid to decision making in health care has been steadily increasing, with the effect that not only a large number of such analyses have been carried out, but also the discussion of methods and foundations has been growing.Among the issues which have been debated is the possibility of drawing up guidelines for carrying out a cost-effectiveness analysis.Such guidelines have indeed been established various time intervals, and the trend toward towards greater detailedness of guidelines has in the last years been evident, possibly as a response to the demand from the pharmaceutical industry wanting exact instructions as to which documentation has to be presented, partly by authorities wanting a clear description of the information to be demanded from producers wanting to market a new drug.
In many cases, the scientific community has been invited to participate in drawing up guidelines, so that the cost-effectiveness analyses which are made in accordance with the guidelines are those that are "best" from a purely theoretical point of view.The question of whether such scientifically correct guidelines can at all be found seems not to have attracted much, if any, attention.However, a similar question has been asked long ago in another field, which however is related, namely that of finding the "correct" accounting standards, rules for how the accounts of a company should be drawn up and presented.As pointed out by Demski (1973), such correct accounting standards simply do not exist.
What is behind this seemingly paradoxical situation -that science cannot point to the right way of doing cost-effectiveness analysis -is a problem which was considered by statistical information theory even earlier.Indeed, one of the main results of this theory, known as Blackwell's theorem (Blackwell, 1951), states that if a decision maker has to choose an act with uncertain consequences, and she has a choice of different methods for obtaining information prior to this choice, then the only ranking of such information services which does not depend on the preferences of the decision maker is the simple, but unfortunately quite trivial rule: "More information is better than less".In other words, there can be no method for collecting and presenting information which is superior to all other methods no matter how the decision maker looks at the outcomes.
It might be argued that this dependence of the guidelines on the decision maker which has to use the information in her decisions is not a real problem; guidelines are indeed established by specified decision makers (public health care organizations, NICE, and others).However, in what follows we can actually sharpen the statement of Blackwell's theorem slightly, so that it will apply even when the decision maker and her preferences are uniquely specified, as long as there are sufficiently many alternative actions to choose among -as will indeed be the case when the decision maker is in charge of a national health care system.So, even in this case, there is no theoretical foundation of guidelines considered as "golden standard" or best practice.Guidelines may be convenient or useful for many other purposes, but alas, they cannot be the last of science.This result adds another dimension to the scepticism towards the increased emphasis on detailed guidelines observed in recent years, as expressed e.g. by Birch and Gafni (2004).
The paper is structured as follows: In Section 2, we introduce the statistical information theory and explain the relation to cost-effectiveness analysis.This section ends with a formulation and proof of the version of Blackwell's theorem which is tailored to our purpose.In Section 3, we connect the abstract result on information methods to the case at hand, cost-effectiveness analysis, and we show why guidelines cannot be scientifically wellfounded here, although it may well be so in other fields of medical decision making.We close the paper with some comments in Section 4.

Cost-effectiveness analysis as an information service
In the present section, we briefly introduce the background for our model, which is the approach to the theory of information introduced by Marschak, Blackwell, and others; a more detailed exposition of this theory can be found e.g. in Hirshleifer and Riley (1992).The model introduced pertains to all decision making under uncertainty, but we keep the discussion as close as possible to our principal application, which is cost-effectiveness analysis, showing as we proceed how the abstract concepts fit in with our overall purpose.
We consider a situation where a decision maker -which in our application may be a health care organization deciding upon the best way of treating the patients under their responsibility -has to make a decision which is subject to some uncertainty.This uncertainty will be modelled in a very simple way, since we assume that there is a finite set of uncertain states, S = {s 1 , . . ., s n }, each of which affect the results of the treatment.The choices of the decision maker is given by a set A of decisions; for our purposes, a decision is a function a : S → X, taking each uncertain state s ∈ S to an outcome a(s) in a space of possible outcomes (to be specified later).In our application to health care, a decision is a particular treatment of patients (which may be a procedure in hospital or the administration of a particular medical drug); the effects of such treatments are usually not fully known, and neither are their costs.In our present case, we have assumed that there are some identifiable but random factors which are behind this uncertainty, which is indeed a simplification as compared with reality where the uncertainty about future consequences of a treatment tend to have a more complex nature.
However, this simple structure allows us to obtain some results which hopefully give insight into the more complicated reality.We shall assume that the decision maker has initial beliefs about the likelihood of each of the underlying states s 1 , . . ., s n influencing outcome of the decisions, formulated as a ("prior") probability distribution (P(σ 1 ), . . ., P(σ p )). Also, the decision cares about the consequences obtained, meaning that she has a utility function U : X → R assigning utility or degree of satisfaction to each possible outcome.Relying only on the initial beliefs, standard decision making under uncertainty calls for the choice of decision which maximizes expected utility over all decisions a ∈ A. This behavior may not correspond fully to what decision makers in health care organizations actually do, but it is at least in line which what they pretend to do.
In our discussion so far, there has been no mentioning of information; decisions were based on assessments of probabilities but not on any observation; to allow for this -and thereby formally introducing "evidence-based medicine" into the model, we should add the option of collecting and processing of information before the decision is made, which is what cost-effectiveness analysis is about.In particular, following the recent guidelines of NICE ( 2004), information about the medical effects of a treatment should be gathered, if possible by a randomized clinical trial, which should also contain observations about healthrelated quality of life, and the information should be collected and processed according the specified rules.What the guidelines specify, is an information method, that is a procedure for collecting date and producing a signal (in our case, a report of the cost-effectiveness analysis, containing relevant cost-effectiveness ratios as well as other ingredients (such as sensitivity analysis, confidence limits etc.) In our simplified formal world, an information method is a pair (Σ, p), where Σ = {σ 1 , . . ., σ m } is a finite set if siugnals, and p is a system of conditional probabilities p(σ j |s i ), for j = 1, . . ., m, i = 1, . . ., n, interpreted as the probability that signal σ j is reported given that the true state of nature is s i .Thus, in our model the information obtained reveals the true state of nature only partially, since there is a random error to be taken into account.This approach is in line with the usual approach to the treatment of uncertainly in cost-effectiveness analysis, cf.e.g.Drummond and McGuire (2002).
Given any observed signal σ j , the decision maker may compute posterior probabilities P * (•|σ j ) of the states of nature using Bayes' formula, where P(σ j ) = n h=1 p(σ j |s h )P(s h ) is the probability of observing the signal σ j .Having now the updated probabilities over states of nature, conditional on the observed signal σ j , the choice of the decision maker must be the decision which maximizes over all a ∈ A. This optimal decision, given the signal σ j , is denoted a[σ j ].The overall expected value (of taking optimal decisions, contingent on the observed signals) of the optimal decision, denoted a[σ j ], will depend on the signal observed; before actually carrying out the observation, we may evaluate the potential gains in expected utility as that is the difference between the ex ante expected utility with and without the information method.We call V(Σ, p; U) the value of the information method (at the utility function U).
The value of information, as derived above, depended on the utility function U of the decision maker.Consequently, the choice if information method, if indeed such a choice is open to the decision maker, will in its turn depend on the utility function.In the case we have in mind, where an information method is a particular way of collecting and presenting data on different medical interventions, this means that the method for performing these operations should be chosen in accordance with the desires and goals of the decision maker.Simple and acceptable as this sounds, it does carry a controversial message, namely that it is in general impossible to prescribe a single such method, independently of the decision maker who is going to use the results.In other word, ranking different methods of performing cost-effectiveness analysis seems not in general possible without recourse to a concrete decision maker, so that guidelines which are applicable to all users cannot be constructed in a scientific way.In the following we give the precise formulation of this result.
For this, we need the notion that one information method is more informative that another.Let (Σ, p), ( Σ, p) be information methods, with |Σ| = m, | Σ| = m.Then (Σ, p) is more informative than ( Σ, p) if for each signal σĵ ∈ Σ such that P(•| σĵ ) 0, there are nonnegative numbers r ĵ,1 , . . ., r ĵ,m with m j=1 r ĵ, j = 1 such that An equivalent formulation of this condition is that there is an ( m × m) matrix R with all column sums 1 such that where P is the (n × m)-matrix with characteristic element P(s i | σĵ ), P the (n × m) matrix with characteristic element P(s i |σ j ), and R is ( m × m) with elements r ĵ, j , ĵ = 1, . . ., m, j = 1, . . ., m.
The following is a formulation of the classical result by Blackwell (1948) adapted to our situation.We work with classes of acceptable utility functions which are more restricted than what is usually seen in the formulations of Blackwell's theorem, where it is usual to allow all possible utility functions.The proof given below follows Bielinska-Kwapisz (2003).
Theorem 1.Let (Σ, p), ( Σ, p) be information methods, let U be a set of utility functions, A a set of actions such that for all vectors q ∈ R n , there are U ∈ U, a, a ∈ A such that U(a(s i )) = q i , U(a (s i )) = 0, i = 1, . . ., n.Then the following are equivalent: (i) (Σ, p) is more informative than ( Σ, p), (ii) V(Σ, p; U) ≥ V( Σ, p; U) for all U ∈ U.
Proof: (ii)⇒(i): Suppose on the contrary that V(Σ, p; U) ≥ V( Σ, p; U) for all U ∈ U but that (Σ, p) is not more informative than ( Σ, p); then by definition we have that there is a signal σĵ * of the information method ( Σ, p) with P(•| σĵ * ) 0 such that By separation of convex sets, there must then be q ∈ R n , q 0, such that Now we use the assumption on U and A to find U ∈ U and a 1 , a 2 ∈ A such that U(a 1 (s i )) = q i , U(a 2 (s i )) = 0, i = 1, . . ., n; in the following we assume that A contains only these two acts.Then for each σ ∈ Σ we have that 6 Assessing the value of the information method ( Σ, p) similarly, we get It follows that V( Σ, p; U) > V(Σ, p; U), a contradiction, showng that (i) must hold.

Conditions for meaningfulness of guidelines
In the previous section, we established a version of Blackwell's theorem tailored to our problem, dealing with cost-effectiveness analysis in general and rules for conducting such analyses in particular.In the present section, we draw the lines from the abstract world of information systems to the more relevant context of guidelines for cost-effectiveness analysis -as well as to guidelines for other aspects of medical decision making.
For this, we take a closer look on the conditions on U and A stated in the theorem, which state that there is a rather large supply (in terms of utility levels achieved in the different states of nature) of decisions which are in principle available.We now consider cases where this richness of action possibilities is a consequence of the underlying structure.
Such a case is provided by the environment in which cost-effectiveness analysis is carried out.We outline briefly its theoretical background (or rather, one possible theoretical background, as there may be several, cf.e.g.Brouwer and Koopmanschap (2000)): We consider medical interventions (in a broad sense) in a society with M individuals, each characterized by a vector (x i , h i ) describing current consumption of commodities and current state of health; the consumption part of the vector in its turn is a vector describing consumption of each of the l goods available, x i = (x i1 , . . ., x il ), and similarly the health component has the form h i = (h i1 , . . ., h iK ), where h ik measures the kth characteristic of health (which may be ability to move around, or ability to hear, or some other aspect of health).An intervention is described by the change ( x i , h i ) which it causes in the state (of consumption and health) of the individual i, for i = 1, . . ., m.
In the standard approach (cf.e.g.Hansen, Hougaard, Keiding, Østerdal (2004)) it is assumed that each individual i has a utility function u i defined on the consumption-health pairs (x i , h i ), and the decision maker or society assesses allocations ((x 1 , u 1 ), . . ., (x m , u m )) in accordance with a social welfare function S defined on m-tuples of utility levels, so that allocations are judged by the achieved value U((x 1 , u 1 ), . . ., (x m , u m )) = (S (u 1 (x 1 , h 1 ), . . ., u m (x m , h m )), an approach often termed as "welfarist".In our present setup, we do not insist that allocations are ranked exactly in this way, but only that the decisions in society are made in accordance with a utility function of the type S ((x 1 , h 1 ), . . ., (x m , h m )).Since an intervention is described as a change (from status-quo) in interventions, it follows that may think of the decision maker's utility function as defined directly on these the outcomes of the interventions, changes of allocation, namely by To make this setup fit with our model of the previous section, we assume that the changes in allocation are subject to random displacements, so that ( x i , h i ) depend on the state of nature s h , h = 1, . . ., m, so that an intervention takes the form ( x i (s h ), h i (s h )) m i=1 n h=1 .In our application, the uncertainty mainly pertains to the effects of the medical treatment; the draw on material ressources may of course also be subject to randomness, although to a smaller extent.What does distinguish the two commodity and health components is however the possibility of compensation payments (between individuals and from outside); in the statement of our main result below we assume effects, positive or negative, on the health components of the individuals, may be offset by suitable money compensations.Theorem 2. Let A be a set of interventions of the form a = ( x i (s h ), h i (s h )) m i=1 n h=1 depending on an uncertain state of nature, which contain also the zero intervention ("do nothing"), and let U be a set of utility functions of the decision maker.Assume that (i) A allows for all possible money compensations of health effects: For each intervention (( x 1 , h 1 ), . . .( x m , h m )), state s h , and each commodity allocation ( x1 , . . ., xm ), the set A contains an intervention a such that (ii) for each U ∈ U, U is monotonic in the commodity components and the marginal rate of substitution between commodity and and health (of any individual) is bounded (from above and below).
If the cost-effectiveness methodology (Σ, p) is better than ( Σ, p) for all U ∈ U, then (Σ, p) is a more detailed version of ( Σ, p) (in the sense of being more informative).
Proof: The theorem is basically a reformulation of Theorem 1 and all that is needed is to show that A contains interventions such that for each vector q = (q 1 , . . ., q m ) ∈ R m , there is an intervention ( But this follows easily from the assumptions (i) and (ii), which together show that there is an intervention, constructed from the original one combined with suitable commodity displacements in each state, such that the final utility level in each state corresponds to the vector q.The result now follows immediately.
It may be noticed that our assumptions in Theorem 2 are stronger than what is really needed; indeed, we need not compensate everybody, what matters is only that some compensations can be made which make up for any loss and gain that the decision maker would experience from the uncompensated intervention.Also, it suffices to consider very small displacements since what mattered in Theorem 1 was not the absolute values of the utilities in each state but only their relative values.

Discussion
The implications for practice of the results are that guidelines for cost-effectiveness are of dubious value., at least when considered as a source of information.Given that the assumptions of Theorem 2 are reasonably weak and seem satisfied in practical situations where cost-effectiveness analysis is performed, the conclusion is that there can be no "best" way of setting up such an analysis.The reference case approach, specifying that observations on medical and other outcomes of an intervention should be carried out according to a standardized scheme, may have administrative merits but it has no scientific basis.The only rule that can be stated is that more detailed observation is better than less detaild information, which indeed is trivial.
Why, then, are guidelines so widespread and estimated?Again, Theorem 2 points to an answer: For purely medical interventions, the setup differs considerably from that of cost-effectiveness analysis, since there is no economic component, and obviously monetary transfers make no change in the medical outcome.In such cases, information methods may indeed by ranked to an extent which permits a choice of a best one, singled out as reference case.