choice-metrics.com

Hi there,

I have designed a DCE with the following syntax and have a few questions before I use this to run a pilot study:

Design
;alts = alt1, alt2, alt3
;rows = 60
;require:
alt1.LBP = alt2.LBP ,
alt1.Comm = alt2.Comm
;eff = (mnl,d,mean)
;block = 5, minmax, noimprov(10 secs)
;alg=mfederov(stop=total(100000 iterations))
;bdraws = gauss(2)
;model:
U(alt1) = b0+b1.dummy[(n,1.38,0.08) ]*LBP[1,0] +b2.dummy[(n,1.32,0.18)|(n,1.32,0.18)|(n,1.32,0.18)]*Comm[1,2,3,0] +b3.dummy[(n,0.74,0.2)|(n,0.74,0.2)|(n,0.74,0.2)] *Opioid [1,2,3,0] + b4[(n,-0.56,0.14)]* OpioidPain[1,0] + b5[(n,-0.56,0.14)]* OpioidAEs[1,0] /
U(alt2) = b0 + b1.dummy[(n,1.38,0.08)] * LBP + b2.dummy[(n,0.32,0.18)|(n,1.32,0.18)|(n,1.32,0.18)]* Comm + b6.dummy[(n,0.52,0.13)]* NSAID[1,0] + b7[(n,-0.54 ,0.13)]* NSAIDPain[1,0] + b8[(n,-0.53,0.13)]* NSAIDAEs[1,0] /
U(alt3) = 0
$

1. The design has two scenario variables (LBP and Comm in the below syntax) which remain constant across alternatives. I am wondering if these should be coded as interaction effects or main effects and what would the implications of each be?
2. We would like to alternate the options between each choice set in the survey, with the aim to reduce bias in the answers, however does this require a specific coding or is this something we just do when setting up the survey?
3. The priors I have are from dummy data, random simulations from Qualtrics. I have been advised not to use these for the main study, but is it ok to use these to help with the design for the pilot study and then use the priors from the pilot for the main study? I understand I cannot run an orthogonal design for the pilot as we have an uneven number of levels across alternatives.
4. Alt 3 is the opt out option, however does this still need to include the scenario variables? is this coded correctly?

Greatly appreciate your help!

Melanie

1. Scenario variables for a labelled experiment can be added as main effects (as you have) as well as interaction effects. Both have a different interpretation. Your main effects indicate how LBP and Comm affect preferences towards Opioid and NSAID, while interaction effects would be interpreted as differences in sensitivity to the other attributes depending on levels of LBP and Comm. So you could consider adding interaction effects, but maybe this is not relevant for your study.

2. What do you mean with "alternate the options"?

3. Priors need to reflect behaviour of respondents. Simulations do not reflect any behaviour and should never be used, simulations in survey tools are merely to check whether all variables are assigned correctly and to check whether you can estimate a model, but not for any other purpose. Orthogonal designs exist for combinations of even and odd numbers of levels, but orthogonal designs cannot handle scenario variables that are fixed across alternatives. In this case, it is based to set the priors to zero, which means you do not have information about the priors (which is the same assumption as in orthogonal designs). This is a fine strategy for a pilot study.

4. The optout alternative does not need to be given a utility function, so you can leave out the utility function such that it is defaulted to zero.

Below I provide a recommended syntax to start with. It sets priors to zero, it assumes that the first two alternatives are labelled and therefore have a different constant, and also the parameters for LBP and Comm can be different for different labels. I have defaulted to the swapping algorithms since this achieved attribute level balance, which is often useful for a pilot study. I also added ;con since the alternative-specific constants may be of interest.

Code: Select all

Design
;alts = Opioid, NSAID, optout
;rows = 60
;block = 5, minmax, noimprov(10 secs)
;con
;eff = (mnl,d)
;model:
U(Opioid) = b0
          + b1.dummy[0]     * LBP[1,0]
          + b2.dummy[0|0|0] * Comm[1,2,3,0]
          + b3.dummy[0|0|0] * Opioid[1,2,3,0]
          + b4[0]           * OpioidPain[1,0]
          + b5[0]           * OpioidAEs[1,0]
          /
U(NSAID) = b6
         + b7.dummy[0]      * LBP[LBP]
         + b8.dummy[0|0|0]  * Comm[Comm]
         + b9.dummy[0]      * NSAID[1,0]                 
         + b10[0]           * NSAIDPain[1,0]
         + b11[0]           * NSAIDAEs[1,0]
$

Michiel

Hi Michiel,

Your responses have been very helpful! it is such a relief to be able to clarify questions with you on this forum.

Just in regards to question 2, apologies for not being clearer but what I meant was when we set up the survey we wanted to alternate option A (opioid) and option B (NSAID) between each choice set, so not all option A's would be opioid and not all option B's would be the NSAID option. My understanding for doing this is to avoid the left-right bias. However, do we need to code differently if we wish to do this?

Best wishes,
Melanie

The randomisation of the order of the alternatives is done in the survey instrument (such as Qualtrics or SurveyEngine). The experimental design remains the same, only the order in which they are shown to respondent is randomised. Note that randomisation only appears across respondents, not within respondents to avoid confusion.

So one respondent would always see Opioid on the left, whereas another respondent would always see NSAID on the left. When you pool the data from all respondents you can add an order dummy to your utility functions to account for left-to-right bias.

Michiel

Thanks Michiel! again very helpful

Sorry Just one more thing!

it just occurred to me how do I know which alternative is the reference when interpreting the output? I realise we have no output with Zero priors at this stage, but when coding for the Bayesian D-efficient design will the opt out be the reference?

If you are referring to the constants in the model, then currently the optout is normalised to zero hence the constants for Opioid and NSAID are relative to the optout.

If you are referring to the order dummies in the model, you can select any reference. I generally choose the last alternative as the base level and estimate dummy coefficients for the alternatives that are presented first and second. You would enter these dummy variables in the utility functions of all three alternatives after you have collected the data. In your case, I would allow the following three orderings in the survey instrument:

Opioid, NSAID, optout
Optout, Opioid, NSAID,
NSAID, Opioid, optout
optout, NSAID, Opioid

If the survey instrument does not easily allow you do randomise to this order, you can simply create four versions of your design and randomly select a version to assign to each respondent.

Michiel