choice-metrics.com

Dear Michiel,

I have a quick question on adding a dummy coded, scenario variable into my design without first including it as a main effect in the utility function in ngene. I can't find the answer in the manual, in this forum or in your slides.

Brief background
My experiment requires individuals to choose between two treatments, A or B, based on 3 attributes:
(1)[NHB] as a categorical variable with 4 levels;
(2)[CERT] as a categorical variable with 4 levels;
(3)[WAIT] as a continuous variable.

I have two scenario variables:
(4) [LIFE] as a continuous variable,
(5) [QOL] as a categorical variable with 4 levels.

I planned on interacting LIFE and QOL with two of the attributes (1-3) to include in the design as contextual effects. I did not plan to (and don't think I should) include LIFE and QOL as main effects - consistent with your advice.

Here is my code, I have indicated where I have issues with i5,i6 and i7 in ?comments? in the code chunk below.

Also, if I could check my choice task (row) calculation S = K/(J-1). As I have 1 ASC [1 parameter to estimate], 2 continuous variables [2 additional parameters], 3 categorical variables, each with 4 levels (as interactions) [3(4-1) = 9 additional parameters]. S= 12 parameters. So I am correct using 36 with 3 blocks to get have sufficient variation to calculate multiple catigorical variables? Thought as it's an efficient design I'm not sure I would have to stick to a multiple of 12 for ALB, but 12 choice tasks per respondent is probably manageable.

Also if you have any concerns about how I have estimated my priors (for the pilot study) please do indicate.

Code: Select all

design
;alts = DrugA*, DrugB*
;rows = 36
;block= 3
;eff  =(mnl,d)
;alg = mfederov
;reject:
         DrugA.LIFE = 0.5 and DrugA.WAIT = 0.5,
         DrugA.LIFE = 0.5 and DrugA.WAIT = 1,
         DrugA.LIFE = 0.5 and DrugA.WAIT = 2,
         DrugA.LIFE = 1   and DrugA.WAIT = 1,
         DrugA.LIFE = 1   and DrugA.WAIT = 2,
         DrugA.LIFE = 2   and DrugA.WAIT = 2,
 
         DrugB.LIFE = 0.5 and DrugB.WAIT = 0.5,
         DrugB.LIFE = 0.5 and DrugB.WAIT = 1,
         DrugB.LIFE = 0.5 and DrugB.WAIT = 2,
         DrugB.LIFE = 1   and DrugB.WAIT = 1,
         DrugB.LIFE = 1   and DrugB.WAIT = 2,
         DrugB.LIFE = 2   and DrugB.WAIT = 2
 
;require:
         DrugA.LIFE = DrugB.LIFE,
         DrugA.QOL  = DrugB.QOL
 
;model:
U(DrugA) = c1
         + b1.dummy[0.0001|0.0002|0.0003]   *  NHB[1,2,3,0]
         + b2.dummy[0.0001|0.0002|0.0003]   *  CERT[1,2,3,0]
         + b3[-0.0001]                      *  WAIT[0,0.5,1,2]
         + i4[0]                            *  WAIT * LIFE[0.5,1,2,5]    ?LIFE is simple to interact without being added as main effects as it is continuous
         + i5.[0]                           *  WAIT * QOL.dummy[1]    ?No matter how I code these, I get error messages
         + i6.[0]                           *  WAIT * QOL.dummy[2]    ?No matter how I code these, I get error messages
         + i7.[0]                           *  WAIT * QOL.dummy[3]    ?No matter how I code these, I get error messages
 
         /
U(DrugB) = b1.dummy[0.0001|0.0002|0.0003]   *  NHB[1,2,3,0]
         + b2.dummy[0.0001|0.0002|0.0003]   *  CERT[1,2,3,0]
         + b3[-0.0001]                      *  WAIT[0,0.5,1,2]
         + i4[0|0|0]                        *  WAIT * LIFE[0.5,1,2,5]
         + i5.[0]                           *  WAIT * QOL.dummy[1]   
         + i6.[0]                           *  WAIT * QOL.dummy[2]   
         + i7.[0]                           *  WAIT * QOL.dummy[3]   
 
$
 


I also tried coding it as below, but that did not work either:

Code: Select all

;model:
U(DrugA) = c1
         + b1.dummy[0.0001|0.0002|0.0003]   *  NHB[1,2,3,0]
         + b2.dummy[0.0001|0.0002|0.0003]   *  CERT[1,2,3,0]
         + b3[-0.0001]                      *  WAIT[0,0.5,1,2]
         + i4[0]                            *  WAIT * LIFE[0.5,1,2,5]
         + i5.dummy[0|0|0]                  *  WAIT * QOL.dummy[1,2,3,0]
 
         /
U(DrugB) = b1.dummy[0.0001|0.0002|0.0003]   *  NHB[1,2,3,0]
         + b2.dummy[0.0001|0.0002|0.0003]   *  CERT[1,2,3,0]
         + b3[-0.0001]                      *  WAIT[0,0.5,1,2]
         + i4[0|0|0]                        *  WAIT * LIFE[0.5,1,2,5]
         + i5.dummy[0|0|0]                  *  WAIT * QOL[1,2,3,0]
$


Thanks in advance for any advice - much appreciated!

It is indeed tricky to add categorical variables only as an interaction effect in Ngene because of how currently the syntax works. We have developed a new way of defining dummy variables that will allow you to directly specify them in interactions only, but that is currently being implemented and is not ready yet. But there is another way to do it, see the script below. The trick is to define two models, namely model m1 where everything is added as a main effect, and model m2 where it is added as an interaction effect only. Model m1 serves only to define the categorical variable, so that it can be used in an interaction in m2. The script below only optimises model m2 since it states eff = m2(mnl,d). A nice little trick

Note that the modified Federov will relax attribute level balance, so the numerical/contribution attribute levels may not appear equally. In that case, you may consider imposing some constraints in their appearance, such as wait[0,0.5,1,2](6-12,6-12,6-12,6-12), noting that it may be difficult to get a high level of attribute level balance because of the constraints you have imposed.

Regarding your question about the number of rows, yes 36 will be sufficient. It does not need to be a multiple of 12, you could also do rows = 40 and block = 10 or any other combination.

Code: Select all

design
;alts(m1) = DrugA*, DrugB*
;alts(m2) = DrugA*, DrugB*
;rows = 36
;block= 3
;eff  = m2(mnl,d)
;alg = mfederov
;reject:
         DrugA.LIFE = 0.5 and DrugA.WAIT = 0.5,
         DrugA.LIFE = 0.5 and DrugA.WAIT = 1,
         DrugA.LIFE = 0.5 and DrugA.WAIT = 2,
         DrugA.LIFE = 1   and DrugA.WAIT = 1,
         DrugA.LIFE = 1   and DrugA.WAIT = 2,
         DrugA.LIFE = 2   and DrugA.WAIT = 2,
 
         DrugB.LIFE = 0.5 and DrugB.WAIT = 0.5,
         DrugB.LIFE = 0.5 and DrugB.WAIT = 1,
         DrugB.LIFE = 0.5 and DrugB.WAIT = 2,
         DrugB.LIFE = 1   and DrugB.WAIT = 1,
         DrugB.LIFE = 1   and DrugB.WAIT = 2,
         DrugB.LIFE = 2   and DrugB.WAIT = 2
 
;require:
         DrugA.LIFE = DrugB.LIFE,
         DrugA.QOL  = DrugB.QOL
 
;model(m1):
U(DrugA) = c1
         + b1.dummy[0.0001|0.0002|0.0003]   *  NHB[1,2,3,0]
         + b2.dummy[0.0001|0.0002|0.0003]   *  CERT[1,2,3,0]
         + b3[-0.0001]                      *  WAIT[0,0.5,1,2]
         + b4.dummy[0|0|0]                  *  QOL[1,2,3,0]
         + b5                               *  LIFE[0.5,1,2,5]
         /
U(DrugB) = b1.dummy[0.0001|0.0002|0.0003]   *  NHB[1,2,3,0]
         + b2.dummy[0.0001|0.0002|0.0003]   *  CERT[1,2,3,0]
         + b3[-0.0001]                      *  WAIT[0,0.5,1,2]
         + b4.dummy                         *  QOL[1,2,3,0]
         + b5                               *  LIFE[0.5,1,2,5]
 
;model(m2):
U(DrugA) = c1
         + b1.dummy[0.0001|0.0002|0.0003]   *  NHB[1,2,3,0]
         + b2.dummy[0.0001|0.0002|0.0003]   *  CERT[1,2,3,0]
         + b3[-0.0001]                      *  WAIT[0,0.5,1,2]
         + i4[0]                            *  WAIT * LIFE[0.5,1,2,5]
         + i5.[0]                           *  WAIT * QOL.dummy[1]   
         + i6.[0]                           *  WAIT * QOL.dummy[2]   
         + i7.[0]                           *  WAIT * QOL.dummy[3]   
 
         /
U(DrugB) = b1.dummy[0.0001|0.0002|0.0003]   *  NHB[1,2,3,0]
         + b2.dummy[0.0001|0.0002|0.0003]   *  CERT[1,2,3,0]
         + b3[-0.0001]                      *  WAIT[0,0.5,1,2]
         + i4[0|0|0]                        *  WAIT * LIFE[0.5,1,2,5]
         + i5.[0]                           *  WAIT * QOL.dummy[1]   
         + i6.[0]                           *  WAIT * QOL.dummy[2]   
         + i7.[0]                           *  WAIT * QOL.dummy[3]   
 
$


Michiel

Dear Michiel,

Thank you for your useful advice. Using this trick works and I am much nearer to finalising my experimental design. However I have a few additional questions on including scenario variables that I am unable to answer from your slides or the ngene user manual that I hope you can shed light on. For reference, my questions are regarding my design (for m2) which I have included below.

Brief background
My experiment requires individuals to choose between two treatments, A or B, based on 2 main attributes:
(1)[CERT] as a categorical variable with 4 levels;
(2)[WAIT] as a continuous variable.

I have three scenario variables that will change across choice tasks but will be held constant across alternatives for simplicity:
(3) [LIFE] as a continuous variable,
(4) [QOL] as a categorical variable with 4 levels.
(5) [IOB] as a categorical variable with 3 levels

Q1)
For my experimental design in ngene that includes scenario variables, am I correct in assuming that I should specify all possible interactions (i.e. between scenario variables and main effects) when creating the design? For example in my design I interact LIFE, QOL, IOB (all of my scenario variables) with each of my main effects (WAIT and CERT)? Alternatively, I could have just interacted them with WAIT - doing either seems to have little impact on my D-Error (<-0.2 in both cases, but not the same). I think it would be best to include all interactions as I have done to get the most efficient design?

Q2)
One of my scenario variables (IOB) is actually really a product attribute (how well it works), but it seems more useful to consider it as a scenario variable because it is likely to have a large impact on relative importance/utility of WAIT and CERT as a scenario variable might. In contrast, my other scenario variables: LIFE (peoples life expectancy) and QoL (Quality of Life) are what I would consider "true" scenario variables that do not apply to product itself but rather the individual. In substantive terms and for ngene, is it ok to specify a design with "true" scenario variables (ones not concerning the product, but rather the 'scenario') and a scenario variable that is somewhat a product attribute but that I want to evaluate as a scenario variable?

Q3)
Related to the question above, I compared two models where (A) IOB was included as a main effect and held constant across choices using a constraint vs (B) IOB is included as a scenario variable (interacted with the two main effects) and is also held constant across choice using a constraint - this is the case for my model below. To my suprise, for Model A the D-Error was <8000, whereas for the model below it is <0.2. Therefore, in Ngene, specifying the constraint DrugA.IOB = DrugB.IOB, and then including IOB as an interaction vs main effect has a very large impact. Are you able to possibly explain this is simple terms?

Q4)
For the model below (m2) I get the error message "the modified federov candidate set size of 2000 could not be achieved......." which I can see from previous forum posts is fine (and can be removed by adding the number it reports in the design (alg=mfederov(candidates = xxx)) , but it also adds it is "98.12% due to constraints". I just wanted to check that I can still ignore this error message if it's due to constraints as I am aware I am adding some quite strict restraints by having three attributes constants across alternatives.

Thanks so much in advance for any help.

Code below:

Code: Select all

design
;alts(m1) = DrugA*, DrugB*
;alts(m2) = DrugA*, DrugB*
;rows = 40
;block= 4
;eff  = m2(mnl,d)
;alg = mfederov
;reject:
         DrugA.LIFE = 0.5 and DrugA.WAIT = 0.5,
         DrugA.LIFE = 0.5 and DrugA.WAIT = 1,
         DrugA.LIFE = 0.5 and DrugA.WAIT = 2,
         DrugA.LIFE = 1   and DrugA.WAIT = 1,
         DrugA.LIFE = 1   and DrugA.WAIT = 2,
         DrugA.LIFE = 2   and DrugA.WAIT = 2,
 
         DrugB.LIFE = 0.5 and DrugB.WAIT = 0.5,
         DrugB.LIFE = 0.5 and DrugB.WAIT = 1,
         DrugB.LIFE = 0.5 and DrugB.WAIT = 2,
         DrugB.LIFE = 1   and DrugB.WAIT = 1,
         DrugB.LIFE = 1   and DrugB.WAIT = 2,
         DrugB.LIFE = 2   and DrugB.WAIT = 2
 
;require:
         DrugA.LIFE = DrugB.LIFE,
         DrugA.QOL  = DrugB.QOL,
         DrugA.IOB  = DrugB.IOB
 
;model(m1):
U(DrugA) = c1
         + b1.dummy[0.0001|0.0002]          *  IOB[1,2,0]
         + b2.dummy[0.0001|0.0002|0.0003]   *  CERT[1,2,3,0]
         + b3[-0.0001]                      *  WAIT[0,0.5,1,2]
         + b4.dummy[0|0|0]                  *  QOL[1,2,3,0]
         + b5                               *  LIFE[0.5,1,2,5]
         /
U(DrugB) = b1.dummy[0.0001|0.0002]          *  IOB[1,2,0]
         + b2.dummy[0.0001|0.0002|0.0003]   *  CERT[1,2,3,0]
         + b3[-0.0001]                      *  WAIT[0,0.5,1,2]
         + b4.dummy                         *  QOL[1,2,3,0]
         + b5                               *  LIFE[0.5,1,2,5]
 
;model(m2):
U(DrugA) = c1
         + b1.dummy[0.0001|0.0002|0.0003]   *  CERT[1,2,3,0]
         + b2[-0.0001]                      *  WAIT[0,0.5,1,2]

         + i3[0]                            *  WAIT * LIFE[0.5,1,2,5]
         + i4.[0]                           *  WAIT * QOL.dummy[1]   
         + i5.[0]                           *  WAIT * QOL.dummy[2]   
         + i6.[0]                           *  WAIT * QOL.dummy[3]   
         + i7.[0.0001]                      *  WAIT * IOB.dummy[1]
         + i8.[0.0002]                      *  WAIT * IOB.dummy[2]

         + i9.[0]                           *  CERT * LIFE[0.5,1,2,5]
         + i10.[0]                          *  CERT * QOL.dummy[1]   
         + i11.[0]                          *  CERT * QOL.dummy[2]   
         + i12.[0]                          *  CERT * QOL.dummy[3]   
         + i13.[0.0001]                     *  CERT * IOB.dummy[1]
         + i14.[0.0002]                     *  CERT * IOB.dummy[2]

         /
U(DrugB) = b1.dummy[0.0001|0.0002|0.0003]   *  CERT[1,2,3,0]
         + b2[-0.0001]                      *  WAIT[0,0.5,1,2]

         + i3[0|0|0]                        *  WAIT * LIFE[0.5,1,2,5]
         + i4.[0]                           *  WAIT * QOL.dummy[1]   
         + i5.[0]                           *  WAIT * QOL.dummy[2]   
         + i6.[0]                           *  WAIT * QOL.dummy[3]   
         + i7.[0.0001]                      *  WAIT * IOB.dummy[1]
         + i8.[0.0002]                      *  WAIT * IOB.dummy[2]

         + i9.[0]                           *  CERT * LIFE[0.5,1,2,5]
         + i10.[0]                          *  CERT * QOL.dummy[1]   
         + i11.[0]                          *  CERT * QOL.dummy[2]   
         + i12.[0]                          *  CERT * QOL.dummy[3]   
         + i13.[0.0001]                     *  CERT * IOB.dummy[1]
         + i14.[0.0002]                     *  CERT * IOB.dummy[2]

$


Q1) You do not NEED to include all interactions, but you CAN include all interactions. Each interaction has a different meaning in the model and you need to think about which interaction effects would make most sense behaviourally. If all of them would make sense, then include all of them. It does not hurt including all of them a priori, even if you remove some later in model estimation.

Q2) I think this is fine, but note that making an attribute a scenario variable means that you can no longer look at trade-offs with this attribute since it is becomes constant across all alternatives. So you will not be able to derive marginal rates of substitution or relative importance of this attribute. Instead, the IOB attribute will explain the sensitivity towards other attributes.

Q3) You cannot include IOB as a main effect unless you add an opt-out alternative. If drugA.IOB = drugB.IOB, then there will never be a trade-off with IOB in the data and hence parameter b1 cannot be estimated. This is signaled by the very large D-error, indicating that the model is not identifiable and cannot be estimated. So either add an opt-out alternative, or only include IOB as an interaction effect. With an opt-out alternative, where IOB can be omitted (since it is a different labelled alternative), the main effect represents the impact of IOB of choosing a drug or choosing no drug.

Q4) Yes you can ignore the error message, as long as there are a sufficient number of choice tasks in the candidate set to create the design that it is fine. Of course, the more constraints you add, the less efficient the design becomes, so only add constraints that are necessary to make the choice tasks more realistic.

Michiel

Thanks Michiel - much appreciated!

Dear Michiel,

For the purposes of obtaining priors, I have a question relating to the code above, specifically when it comes to the interaction of a categorical scenario variable, with another categorical variable.

In this ngene example, [CERT] is coded as a categorical dummy variable with 4 levels. However, for interaction i9 to i14, when [CERT] is interacted with the other variables, the level of [CERT] is not specified, it is simply written CERT * IOB.dummy[1] for example. In this instance, which categorical level of CERT is ngene interacting? To me, I would expect to see here something like "CERT.dummy[1] * IOB.dummy[1], but this is not the case?

My reason for asking is that I am creating a model in R (Apollo) to obtain priors. When obtaining a beta for the interaction terms, I wouldn't know how to specify the model in R without the information above and an understanding of which level of CERT ngene is interacting?

I have copied the segment of code I am referring to below for your reference:

Code: Select all

;model(m2):
U(DrugA) = c1
         + b1.dummy[0.0001|0.0002|0.0003]   *  CERT[1,2,3,0]
         + b2[-0.0001]                      *  WAIT[0,0.5,1,2]

         + i3[0]                            *  WAIT * LIFE[0.5,1,2,5]
         + i4.[0]                           *  WAIT * QOL.dummy[1]   
         + i5.[0]                           *  WAIT * QOL.dummy[2]   
         + i6.[0]                           *  WAIT * QOL.dummy[3]   
         + i7.[0.0001]                      *  WAIT * IOB.dummy[1]
         + i8.[0.0002]                      *  WAIT * IOB.dummy[2]

         + i9.[0]                           *  CERT * LIFE[0.5,1,2,5]
         + i10.[0]                          *  CERT * QOL.dummy[1]   
         + i11.[0]                          *  CERT * QOL.dummy[2]   
         + i12.[0]                          *  CERT * QOL.dummy[3]   
         + i13.[0.0001]                     *  CERT * IOB.dummy[1]
         + i14.[0.0002]                     *  CERT * IOB.dummy[2]

         /


Thanks in advance for you advice,

Rob

Hi Rob,

Yes you understand it correctly, you need to specify interactions between dummy coded variables as cert.dummy[1] * qol.dummy[1] etc.
If you specify cert * qol.dummy[1] it simply considers cert as a numerical variable with levels 0,1,2,3.

Michiel

Thanks, Michiel for your quick reply.