choice-metrics.com

Hi,
Would it be possible to have an expert check of my syntax.

Design
;alts = alt1*, alt2*, optout
;rows = 36
;eff = (mnl, d)
;block = 3
;model:

U(alt1) = b1.effects[0.000011|0.00001|-0.00001]*FEV[15,5,-5,0] + b2.effects[0.00001]*IVdays[1,0] + b3.effects[0.000011|0.00001]*Abdo[2,1,0] + b4.effects[0.000012|0.000011|0.00001]*Lexp[15,10,5,0] + b5.effects[0.000011|0.00001]*QoL[2,1,0] + b6.effects[0.000011|0.00001]*Neb[2,1,0] + b7.effects[0.000011|0.00001]*Physio[2,1,0] /

U(alt2) = b1*FEV + b2*IVdays + b3*Abdo + b4*Lexp + b5*QoL + b6*Neb + b7*Physio /

U(optout)= b0[0]

$

My questions are:
Have I correctly specified the opt-out?
My experiment is looking at patients’ preferences for adding a new medication to their existing treatment. It consists of two unlabelled alternatives and an opt-out. The opt-out is to stay on current treatment only (the baseline) – so should be considered as a status-quo. All attributes specify changes in various outcomes from their baseline with existing treatment (specified as ‘0’ in the model).

We are at the pilot stage and I have no information on priors – other than sign. I have received expert advice elsewhere to treat unknown attributes as categorical in the first instance. Since a (potential) secondary objective is to model WTP using FEV as the numeraire – is it reasonable to treat these as categorical in the design stage, and switch to continuous in subsequent analysis?

Many thanks in advance
Rory

Yes this specification of the optout is correct, either the optout has a constant or alt1 and alt2 have a (same) constant.

Yes you can treat variables at categorical at the design stage and later estimate them as continuous variable, that sounds fine. Note that you can also compute WTP if FEV is categorical, you just get multiple WTP values (which does make it more difficult to interpret).

Michiel

Hi there,

I have been collecting initial data with the above design with a view to updating my priors soon and now have a few follow-on questions – based on a total on n=20 completes so far.

With such a small sample, the results of the experiment are not yet very stable – with each new respondent, there can be quite a big shift in the coefficients. Generally though the results are sensible - though one parameter has a non-significant negative coefficient where it should logically be positive.

Q1 – if one has limited certainty/ confidence in the priors obtained up to a certain point – what should guide one’s judgement of when to stop and update the design (I have budget for a total sample of ~110).

Q2 – How would you handle the prior that has an unexpected negative sign?

From the results I have to date, it is clear that some attributes are dominating while others are close to zero. Using Bayesian priors, I have found that Ngene creates designs with overlap primarily in those attributes that have the smaller coefficients – in some of the models I have looked at, as many as 50% of the 36 designs have overlap in these attributes with smaller coefficients.

Q3 – Linked to Q1 above, if I have limited certainty in the priors collected to date, I am concerned that this overlap will lose me information on attributes that may turn out to be of greater importance than the data currently indicates. Do you have any suggestions to mitigate against this – is there a way to limit overlap?

Q4 – Simulation draws – what would be your recommendation for a large number (16) of Bayesian priors? I have found that Ngene crashes with 3 gauss draws (2 works though)…

Many thanks in advance
Rory

Q1 - Parameter estimates from a small pilot study often are not statistically significant, but that is often not needed. I usually use the information to create Bayesian priors, [(n,mu,sigma)], where mu is the parameter estimate (with the correct sign) and sigma is set to the standard error. This Bayesian prior indicates that you have uncertainty about the priors.

Q2 - If you believe that the prior should be positive, then I would use something like [(n,0.00001,sigma)], which says that you do not really know the value of the parameter, or [(u,0,x)] where x is some upper bound. It is important to use the correct sign of the prior for removing strictly dominant alternatives in choice tasks.

Q3 - Typically, overlap happens for attributes that are dominant (very important), which is a good thing if they are indeed dominant. This would generally not happen if the coefficients are small, but it would happen if the Bayesian distribution is very wide. For example, Bayesian prior [(n,0.001,2)] has a small coefficient on average, but a large standard deviation, which means that when taking draws from this distribution, it will draw values like 1.5, 2, or 3, which are very large and it seems as if they are dominant attributes. You may want to take a conservative approach, for example driving all parameter estimates and standard errors by two.

Q4 - I usually try to avoid having so many Bayesian priors and make priors only Bayesian for the most important attributes (because these will influence utility most) and simply set others to fixed. If you want to use 16 Bayesian priors, I would recommend using a large number of sobol draws, e.g. ;bdraws = sobol(5000).

Michiel