choice-metrics.com

Hi Team,

I am creating a candidate set for the explicit partial design. My instrument has seven items, each with five levels. I want to overlap any two of the seven attributes between Choice A and Choice B for each task. Since there are 21 rotation overlap combinations (e.g., (item1, item2), (item1, item3), (item1, item4)...), I plan to create choice tasks for each of the 21 combinations and combine them into a candidate set. Below is my code for running the first combination using Ngene:


Design
;alts = alt1*, alt2*
;rows = 200
;eff = (mnl,d)
;alg = mfederov
;require:
alt1.item1 = alt2.item1,
alt1.item2 = alt2.item2

;model:
U(alt1) = b0[0] + b1.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)]*item1[1,2,3,4,5] + b2.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)]*item2[1,2,3,4,5] + b3.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)]*item3[1,2,3,4,5] + b4.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)]*item4[1,2,3,4,5] + b5.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)]*item5[1,2,3,4,5] + b6.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)]*item6[1,2,3,4,5] + b7.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)]*item7[1,2,3,4,5] /
U(alt2) = b1.dummy*item1 + b2.dummy*item2 + b3.dummy*item3 + b4.dummy*item4 + b5.dummy*item5 + b6.dummy*item6 + b7.dummy*item7

$

I have the following questions:
1. The results generate an undefined MNL D-error. I have tried adjusting the number of rows (increase or decrease), but the generated design still shows an undefined MNL D-error. How can I fix this? The code runs smoothly with a defined D-error when I do not include the required condition and mfederov, so I believe it is not related to the model specification. Can I still use the choice tasks in the design with an undefined MNL D-error to construct my candidate set? I read in the manual that there does not need to be a lot of intelligence in the candidate set.

2. Am I correct in thinking that there is no explicit requirement for the size of the candidate set, as long as it covers the rotation combinations I need in the choice tasks, with randomly generated combinations of non-overlapping attribute levels for the remaining five attributes? Ngene will automatically select appropriate choice tasks from the candidate set when I input the dataset back into Ngene for running this partial design using the mfederov algorithm. If this is true, I assume I do not need to rely on Ngene to generate the candidate set, but could instead create it externally using R to generate a certain number of choice tasks, with my 21 combinations of overlapped attributes and a random sample of any size for the non-overlapped attributes?

Many thanks for your help and clarification.

1. Your D-error is undefined (infinite) because you cannot estimate model parameters b1 and b2 if the levels across both alternatives are always the same. If item1 and item2 are scenario variables then you need to interact them with the other attributes. You can also add an opt-out but then the parameters for b1 and b2 only indicate the preference to selecting the optout versus not selecting the optout.

2. Yes correct, and you can create the candidate set externally in R. I usually restrict the candidate set size to about 2000-5000 rows. Larger is possible but then the modified Federov algorithm becomes quite slow and would hardly improve the efficiency of the design.

Michiel

Hi Michiel,

1. Regarding point 1, I would like to clarify whether I need to interact items 1 and 2 with all of the other five items. Specifically, should I be interacting item 1 with item 3, item 1 with item 4, item 1 with item 5, item 1 with item 6, item 1 with item 7, and similarly, interacting item 2 with item 3, item 2 with item 4, item 2 with item 5, item 2 with item 6, item 2 with item 7? I understand that these interaction terms are needed for the experimental design. However, for the subsequent modelling analysis once I have collected all DCE responses, do I need to include the interaction terms in my model, or can I only use the main effects for the analysis?

2. I try to include interacting item 1 with item 3 and interacting item 2 with item 3 as a starting point, but I encountered the following error: An interaction term is associated with a dummy or effects coded parameter. Interactions of dummy or effects coded attributes are not currently supported. 'i1.dummy[0]*item1[1,2,3,4,5]*item3[1,2,3,4,5]'" after I run it. Should I use continuous coding instead for the interaction terms, or is there another way to fix this? I am also unsure if my code is correct though:

Design
;alts = alt1*, alt2*
;rows = 200
;eff = (mnl,d)
;alg = mfederov
;require:
alt1.item1 = alt2.item1,
alt1.item2 = alt2.item2

;model:
U(alt1) = b0[0] + i1.dummy[0]*item1[1,2,3,4,5]*item3[1,2,3,4,5] + i2.dummy[0]*item2[1,2,3,4,5]*item3[1,2,3,4,5] + b3.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)]*item3[1,2,3,4,5] + b4.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)]*item4[1,2,3,4,5] + b5.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)]*item5[1,2,3,4,5] + b6.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)]*item6[1,2,3,4,5] + b7.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)]*item7[1,2,3,4,5] /
U(alt2) = i1.dummy*item1*item3 + i2.dummy*item2*item3 + b3.dummy*item3 + b4.dummy*item4 + b5.dummy*item5 + b6.dummy*item6 + b7.dummy*item7

$


Many thanks for your time.

The model you specify in Ngene is THE SAME as the model you specify during model estimation. So if Ngene tells you that the model is not identified, then you will NOT be able to estimate the model. In other words, you CANNOT estimate a model whereby item1 and item2 are main effects if they have the same level as there are no trade-offs being made across alt1 and alt2 with respect to item1 and item2. Interactions between categorical variables result in a lot of variables. For example, if you want to interact item1 with item3 you will get 16 additional parameters. Your model will get quite complex and it will be unlikely that you will be able to obtain statistically significant parameter estimates for all of them.

+ i11 * item3.dummy[1] * item1.dummy[1]
+ i12 * item3.dummy[1] * item1.dummy[2]
+ i13 * item3.dummy[1] * item1.dummy[3]
+ i14 * item3.dummy[1] * item1.dummy[4]
+ i21 * item3.dummy[2] * item1.dummy[1]
+ i22 * item3.dummy[2] * item1.dummy[2]
+ i23 * item3.dummy[2] * item1.dummy[3]
+ i24 * item3.dummy[2] * item1.dummy[4]
+ i31 * item3.dummy[3] * item1.dummy[1]
+ i32 * item3.dummy[3] * item1.dummy[2]
+ i33 * item3.dummy[3] * item1.dummy[3]
+ i34 * item3.dummy[3] * item1.dummy[4]
+ i41 * item3.dummy[2] * item1.dummy[1]
+ i42 * item3.dummy[2] * item1.dummy[2]
+ i43 * item3.dummy[2] * item1.dummy[3]
+ i44 * item3.dummy[2] * item1.dummy[4]

You will need to think about the research questions in your study, and which interaction effects you need. Maybe you do not need all interaction effects.

Only adding variables as interaction effects and NOT as main effects is a topic regularly discussed on this forum and I have provided several examples. This needs to be done as follows, where you specify model m1 to define the categorical variable item1 but only optimise model m2 that uses item1 as interaction effect only. In this example I only assumed item1 to be a scenario variable, you would need to do the same for item2. And you may want to interact item1 and item2 with more attributes than just item3, depending on your research question.

Code: Select all

Design
;alts(m1) = alt1*, alt2*
;alts(m2) = alt1*, alt2*
;rows = 200
;eff = m2(mnl,d)
;alg = mfederov
;require:
alt1.item1 = alt2.item1

;model(m1):
U(alt1) = b0[0]
        + b1.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)] * item1[1,2,3,4,5]
        + b2.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)] * item2[1,2,3,4,5]
        + b3.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)] * item3[1,2,3,4,5]
        + b4.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)] * item4[1,2,3,4,5]
        + b5.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)] * item5[1,2,3,4,5]
        + b6.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)] * item6[1,2,3,4,5]
        + b7.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)] * item7[1,2,3,4,5]
        /
U(alt2) = b1.dummy*item1 + b2.dummy*item2 + b3.dummy*item3 + b4.dummy*item4 + b5.dummy*item5 + b6.dummy*item6 + b7.dummy*item7

;model(m2):
U(alt1) = b0[0]
        + b2.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)] * item2[1,2,3,4,5]
        + b3.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)] * item3[1,2,3,4,5]
        + b4.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)] * item4[1,2,3,4,5]
        + b5.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)] * item5[1,2,3,4,5]
        + b6.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)] * item6[1,2,3,4,5]
        + b7.dummy[(u,-1,0)|(u,-0.5,0)|(u,-0.25,0)|(u,-0.1,1)] * item7[1,2,3,4,5]
        + i11 * item3.dummy[1] * item1.dummy[1]
        + i12 * item3.dummy[1] * item1.dummy[2]
        + i13 * item3.dummy[1] * item1.dummy[3]
        + i14 * item3.dummy[1] * item1.dummy[4]
        + i21 * item3.dummy[2] * item1.dummy[1]
        + i22 * item3.dummy[2] * item1.dummy[2]
        + i23 * item3.dummy[2] * item1.dummy[3]
        + i24 * item3.dummy[2] * item1.dummy[4]
        + i31 * item3.dummy[3] * item1.dummy[1]
        + i32 * item3.dummy[3] * item1.dummy[2]
        + i33 * item3.dummy[3] * item1.dummy[3]
        + i34 * item3.dummy[3] * item1.dummy[4]
        + i41 * item3.dummy[4] * item1.dummy[1]
        + i42 * item3.dummy[4] * item1.dummy[2]
        + i43 * item3.dummy[4] * item1.dummy[3]
        + i44 * item3.dummy[4] * item1.dummy[4]
        /
U(alt2) = b2.dummy*item2 + b3.dummy*item3 + b4.dummy*item4 + b5.dummy*item5 + b6.dummy*item6 + b7.dummy*item7
        + i11 * item3.dummy[1] * item1.dummy[1]
        + i12 * item3.dummy[1] * item1.dummy[2]
        + i13 * item3.dummy[1] * item1.dummy[3]
        + i14 * item3.dummy[1] * item1.dummy[4]
        + i21 * item3.dummy[2] * item1.dummy[1]
        + i22 * item3.dummy[2] * item1.dummy[2]
        + i23 * item3.dummy[2] * item1.dummy[3]
        + i24 * item3.dummy[2] * item1.dummy[4]
        + i31 * item3.dummy[3] * item1.dummy[1]
        + i32 * item3.dummy[3] * item1.dummy[2]
        + i33 * item3.dummy[3] * item1.dummy[3]
        + i34 * item3.dummy[3] * item1.dummy[4]
        + i41 * item3.dummy[4] * item1.dummy[1]
        + i42 * item3.dummy[4] * item1.dummy[2]
        + i43 * item3.dummy[4] * item1.dummy[3]
        + i44 * item3.dummy[4] * item1.dummy[4]
$

Michiel

Thank you for your guidance, Michiel.

I will still include attribute 1 and attribute 2 in my main effects model estimation, as my attribute overlap is rotated. For example, each participant completes 10 tasks. In some tasks, attributes 1 and 2 are the same across Choice A and Choice B. In other tasks, attributes 3 and 4 are the same across Choice A and B, and so on. Therefore, attributes 1 and 2 are not always the same between A and B for all tasks.

What I am doing from our previous conversation is generating the candidate set. Initially, I planned to generate the 21 rotation combination designs and then combine them into a single candidate set. Fixing attributes 1 and 2 is the first rotation combination design. However, I understand your point about not being able to estimate the model parameters for attributes 1 and 2 (i.e., parameters b1 and b2) if the levels across both alternatives are always the same based on my previous Ngene code.

I will also consider generating the candidate set in R also. I really appreciate your guidance on incorporating interaction terms and I will think carefully.