Efficient design with interactions

[Andrew]

Dear Ngene users,

I am trying to create an efficient design with two-way interactions for a discrete choice experiment (dce). The unlabeled dce has 3 alternatives and 4 attributes (6 level x 3 level x 3 level x 3 level).
I want all 6 levels of attribute1 (time) to interact with all levels of the remaining atttributes (risk). There would be 54 interactions to be specified (6*3 + 6*3 + 6*3). If I am not completely wrong there will be all in all 69 parameters (6+3+3+3+18+18+18) with degrees of freedom of 47. I use effects-coding for analysis (mixed logit and latent class model). I guess there will be 200 to 300 respondents in the survey. There are no priors but I can assume sign of parameters. Probably, attribute1 (time) will have the most impact on decisions.

This is my code without interactions which finds designs fast and easily:

Code: Select all

Design
;alts = alt1, alt2, alt3
;rows = 120
;block = 10
eff = (mnl,d,mean)
;bdraws = halton(200)
;model:
U(alt1) =
b1.effects[(n,-0.02,0.01)|(n,-0.01,0.01)|(n,-0.01,0.01)|(n,0.01,0.01)|(n,0.01,0.01)] * att1[1,2,3,4,5,6] +
b2.effects[(n,-0.015,0.01)|(n,-0.01,0.01)] * att2[1,2,3] +
b3.effects[(n,-0.015,0.01)|(n,-0.01,0.01)] * att3[1,2,3] +
b4.effects[(n,-0.015,0.01)|(n,-0.01,0.01)] * att4[1,2,3] /
U(alt2) =
b1 * att1 +
b2 * att2 +
b3 * att3 +
b4 * att4 /
U(alt3) =
b1 * att1 +
b2 * att2 +
b3 * att3 +
b4 * att4
$


I tried to add all 54 interactions to the design and used following specifications for all alts:

Code: Select all

i1 * att1.effects[1] * att2.effects[1] +
i2 * att1.effects[1] * att2.effects[2] +
i3 * att1.effects[1] * att2.effects[3] +
...
...
...
i52 * att1.effects[6] * att4.effects[1] +
i53 * att1.effects[6] * att4.effects[2] +
i54 * att1.effects[6] * att4.effects[3] +


No design was found. And the code seems ridiculously long.

My question, how do I specify these interactions the right way?

Thank you very much. Any help is appreciated.

Regards,
Andrew

[Michiel Bliemer]

A few initial comments:

1) Your current priors are close to zero and you did not really specify a clear ranking in the attribute levels nor did you ask Ngene to check for strictly dominant alternatives, so you may as well set priors equal to zero. In that case, you may be better off creating an orthogonal design with a foldover, which ensures orthogonality between all main effects and all two-way interaction effects.

2) Estimating dummy or effects coded variables is usually difficult, you will typically need a large number of respondents. I doubt that many of your 69 parameters would be statistically significant, this is a large number of parameters for a relatively small sample size, but maybe it is OK. If each parameter is considered random in a mixed logit model, you will need to estimate 138 parameters, which will be even harder.

3) I think that your model with all interactions is not identifiable. You are including interaction effects between reference levels, which I believe you cannot do. This is easiest to understand with dummy coding where the reference levels are always set to zero and hence interactions such as att1.dummy[1] * att2.dummy[2] would always be zero and hence cannot be included, and the same holds I think for effects coded in which the reference levels are confounded. This is more an estimation and identification question, so please ensure that your model is identifiable, otherwise Ngene will not be able to find any design.

Michiel

[Andrew]

Michiel,

thank you very much for the quick response. It was very helpful.

I removed all interactions with reference levels. Of course it is not necessary as with effects-coding estimates of reference levels can easily be calculated as inverted sum of remaining level coefficients of an attribute. So the calculated number of degrees of freedom was wrong as well. Anyway, the number of rows specified will be enough.

I got a design with following code now:

Code: Select all

? design
Design
;alts = alt1, alt2, alt3
;rows = 144
;block = 12
;eff = (mnl,d,mean)
;bdraws = halton(500)
;model:
U(alt1) =
          b1.effects[(n,-0.0003,0.0001)|(n,-0.0002,0.0001)|(n,-0.0001,0.00005)|(n,0.0001,0.00005)|(n,0.0002,0.0001)] * att1 [1,2,3,4,5,6] +
          b2.effects[(n,-0.0002,0.0001)|(n,-0.001,0.00005)] * att2 [1,2,3] +
          b3.effects[(n,-0.0002,0.0001)|(n,-0.001,0.00005)] * att3 [1,2,3] +
          b4.effects[(n,-0.0002,0.0001)|(n,-0.001,0.00005)] * att4 [1,2,3] +
          i1[0] * att1.effects[1] * att2.effects[1] +
          i2[0] * att1.effects[1] * att2.effects[2] +
          i3[0] * att1.effects[1] * att3.effects[1] +
          i4[0] * att1.effects[1] * att3.effects[2] +
          i5[0] * att1.effects[1] * att4.effects[1] +
          i6[0] * att1.effects[1] * att4.effects[2] +
          i7[0] * att1.effects[2] * att2.effects[1] +
          i8[0] * att1.effects[2] * att2.effects[2] +
          i9[0] * att1.effects[2] * att3.effects[1] +
          i10[0] * att1.effects[2] * att3.effects[2] +
          i11[0] * att1.effects[2] * att4.effects[1] +
          i12[0] * att1.effects[2] * att4.effects[2] +
          i13[0] * att1.effects[3] * att2.effects[1] +
          i14[0] * att1.effects[3] * att2.effects[2] +
          i15[0] * att1.effects[3] * att3.effects[1] +
          i16[0] * att1.effects[3] * att3.effects[2] +
          i17[0] * att1.effects[3] * att4.effects[1] +
          i18[0] * att1.effects[3] * att4.effects[2] +
          i19[0] * att1.effects[4] * att2.effects[1] +
          i20[0] * att1.effects[4] * att2.effects[2] +
          i21[0] * att1.effects[4] * att3.effects[1] +
          i22[0] * att1.effects[4] * att3.effects[2] +
          i23[0] * att1.effects[4] * att4.effects[1] +
          i24[0] * att1.effects[4] * att4.effects[2] +
          i25[0] * att1.effects[5] * att2.effects[1] +
          i26[0] * att1.effects[5] * att2.effects[2] +
          i27[0] * att1.effects[5] * att3.effects[1] +
          i28[0] * att1.effects[5] * att3.effects[2] +
          i29[0] * att1.effects[5] * att4.effects[1] +
          i30[0] * att1.effects[5] * att4.effects[2] /

U(alt2) =
          b1 * att1 +
          b2 * att2 +
          b3 * att3 +
          b4 * att4 +
          i1 * att1.effects[1] * att2.effects[1] +
          i2 * att1.effects[1] * att2.effects[2] +
          i3 * att1.effects[1] * att3.effects[1] +
          i4 * att1.effects[1] * att3.effects[2] +
          i5 * att1.effects[1] * att4.effects[1] +
          i6 * att1.effects[1] * att4.effects[2] +
          i7 * att1.effects[2] * att2.effects[1] +
          i8 * att1.effects[2] * att2.effects[2] +
          i9 * att1.effects[2] * att3.effects[1] +
          i10 * att1.effects[2] * att3.effects[2] +
          i11 * att1.effects[2] * att4.effects[1] +
          i12 * att1.effects[2] * att4.effects[2] +
          i13 * att1.effects[3] * att2.effects[1] +
          i14 * att1.effects[3] * att2.effects[2] +
          i15 * att1.effects[3] * att3.effects[1] +
          i16 * att1.effects[3] * att3.effects[2] +
          i17 * att1.effects[3] * att4.effects[1] +
          i18 * att1.effects[3] * att4.effects[2] +
          i19 * att1.effects[4] * att2.effects[1] +
          i20 * att1.effects[4] * att2.effects[2] +
          i21 * att1.effects[4] * att3.effects[1] +
          i22 * att1.effects[4] * att3.effects[2] +
          i23 * att1.effects[4] * att4.effects[1] +
          i24 * att1.effects[4] * att4.effects[2] +
          i25 * att1.effects[5] * att2.effects[1] +
          i26 * att1.effects[5] * att2.effects[2] +
          i27 * att1.effects[5] * att3.effects[1] +
          i28 * att1.effects[5] * att3.effects[2] +
          i29 * att1.effects[5] * att4.effects[1] +
          i30 * att1.effects[5] * att4.effects[2] /

U(alt3) =
          b1 * att1 +
          b2 * att2 +
          b3 * att3 +
          b4 * att4 +
          i1 * att1.effects[1] * att2.effects[1] +
          i2 * att1.effects[1] * att2.effects[2] +
          i3 * att1.effects[1] * att3.effects[1] +
          i4 * att1.effects[1] * att3.effects[2] +
          i5 * att1.effects[1] * att4.effects[1] +
          i6 * att1.effects[1] * att4.effects[2] +
          i7 * att1.effects[2] * att2.effects[1] +
          i8 * att1.effects[2] * att2.effects[2] +
          i9 * att1.effects[2] * att3.effects[1] +
          i10 * att1.effects[2] * att3.effects[2] +
          i11 * att1.effects[2] * att4.effects[1] +
          i12 * att1.effects[2] * att4.effects[2] +
          i13 * att1.effects[3] * att2.effects[1] +
          i14 * att1.effects[3] * att2.effects[2] +
          i15 * att1.effects[3] * att3.effects[1] +
          i16 * att1.effects[3] * att3.effects[2] +
          i17 * att1.effects[3] * att4.effects[1] +
          i18 * att1.effects[3] * att4.effects[2] +
          i19 * att1.effects[4] * att2.effects[1] +
          i20 * att1.effects[4] * att2.effects[2] +
          i21 * att1.effects[4] * att3.effects[1] +
          i22 * att1.effects[4] * att3.effects[2] +
          i23 * att1.effects[4] * att4.effects[1] +
          i24 * att1.effects[4] * att4.effects[2] +
          i25 * att1.effects[5] * att2.effects[1] +
          i26 * att1.effects[5] * att2.effects[2] +
          i27 * att1.effects[5] * att3.effects[1] +
          i28 * att1.effects[5] * att3.effects[2] +
          i29 * att1.effects[5] * att4.effects[1] +
          i30 * att1.effects[5] * att4.effects[2]
$


But Ngene won't find a design when use ;alts = alt1*, alt2*, alt3* to check for dominant alternatives. Maybe I should reconsider the use of specified priors in the syntax. Though, we are certain about sign of parameters and ranking. I tried different specifications (larger priors and close to zero). But it didn't make any difference. Ngene detects too many cases of dominance and didn't come up with any design.

Could you give any advice how to specify my priors in a better way?

And I thought about using orthogonal design with foldover. But I would need at least 36 rows, with foldover 72. I am not sure if I can block the foldover design into more than 2 blocks. And the low number of respondents might be an issue too.

Thank you very much again.

Regards,
Andrew

[paulm]

One simple thing you could do is use fixed priors rather than distributions. Yes, your design will pay a price for that uncertainty, but you have a much better chance of getting one at all.

So instead of [(n,-0.0003,0.0001)|(n,-0.0002,0.0001)|(n,-0.0001,0.00005)|(n,0.0001,0.00005)|(n,0.0002,0.0001)], try [-0.03|-0.02|-0.01|0.01|0.02]. These are pretty small estimates for an effects coded attribute, so should work ok in practice.

Paul

[Michiel Bliemer]

I agree with Paul.
Also, you are constraining your design too much. You are asking for 144 rows that do not include any dominant choice task but is still attribute level balanced, since the default algorithm is the swapping algorithm. The swapping algorithm is not a good algorithm to find such a heavily constrained design.

You should be able to check for dominance by using the modified Federov algorithm (which relaxes attribute level balance) by setting something like:
;alg = mfederov(candidates = 1000)

But note that this algorithm is much slower than the swapping algorithm, so it may take a long time for it to improve efficiency.

Do you really need 144 rows? You can also use 72 rows.

[Andrew]

Paul and Michiel,

thank you very much for your helpful tips. I gonna consider them in my syntax. Accordingly, I gonna create a few designs with slightly different specifications (fixed priors and new algorithm), compare design criteria, create dummy data, check the size of standard error and whether I will be able to estimate what I want to estimate. I would get back to you in case I have any additional question.

Regarding the number of rows, I guess I can use 72 instead of 144 rows. I defined more rows in order get a smaller d-error.

Thank you very much again. Great support as always!

Kind regards,
Andrew

[Andrew]

I have one follow-up question regarding comparability of experimental designs. For example, when I increase number of rows and blocks I get a smaller d-error. What about designs with different algorithm, zero priors and non-zero priors. Would the d-error of designs directly be comparable? Which parameter of design specification do I have to hold constant in order to compare designs?

Thank you.
Kind regards,
Andrew

[Michiel Bliemer]

D-errors can only be compared between designs with fixed:
* priors
* alternatives and attributes
* rows
* model type

Blocking does not effect edficiency. Some algorithms are better in finding more efficient designs than others, depending on which conatraints you apply, but they also make different underlying assumptions. The default swapping algorithm maintains attribute levels balance, while the modified Federov algorithm does not (and hence can achieve better D-errors).

Increasing the number of rows always increases efficiency, but it also means that you have to show more choice tasks to each respondent or that you need a larger sample size because of increased number of blocks. Setting the number of rows should therefore not be done based on efficiency but rather on how many choice tasks a person can handle, and how much variation in the data you would like, which also depends on the number of parameters you are estimating.

Michiel

[Andrew]

Michiel,

this was again very helpful. Thank you very much.

Kind regards,
Andrew