choice-metrics.com

Hi,

I have an experiment where we want to have a fixed number of attributes differing in each choice set (so nine dimensions with either four or five levels, and we want four to differ in each choice set). We also need to be able to estimate a large number of two factor interactions between each of the levels of each of the coefficients with the willingness to pay numeraire (which is A below).

I have run it successfully without this constraint:

Design
;alts = alt1, alt2
;rows = 100
;eff = (mnl,wtp(ref1))
;wtp = ref1(i1,i2,i3,i4,i5,i6,i7,i8,i9,i10,i11,i12,i13,i14,i15,i16,i17,i18,i19,i20,i21,i22,i23,i24,i25,i26,i27,i28,i29,i30,i31,i32/b2)
;alg = swap(stop=noimprov(120 secs))
;model:
U(alt1) = b1[0] + b2[0.1] * A[10,5,2,1] + b3.dummy[0|0|0|0] * B[4,3,2,1,0] + b4.dummy[0|0|0|0] * C[4,3,2,1,0] + b5.dummy[0|0|0|0] * D[4,3,2,1,0] + b6.dummy[0|0|0|0] * E[4,3,2,1,0] + b7.dummy[0|0|0|0] * F[4,3,2,1,0] + b8.dummy[0|0|0|0] * G[4,3,2,1,0] + b9.dummy[0|0|0|0] * H[4,3,2,1,0] + b10.dummy[0|0|0|0] * I[4,3,2,1,0] + i1[0] * A * B.dummy[1] + i2[0] * A * B.dummy[2] + i3[0] * A * B.dummy[3] + i4[0] * A * B.dummy[4] + i5[0] * A * C.dummy[1] + i6[0] * A * C.dummy[2] + i7[0] * A * C.dummy[3] + i8[0] * A * C.dummy[4] + i9[0] * A * D.dummy[1] + i10[0] * A * D.dummy[2] + i11[0] * A * D.dummy[3] + i12[0] * A * D.dummy[4] + i13[0] * A * E.dummy[1] + i14[0] * A * E.dummy[2] + i15[0] * A * E.dummy[3] + i16[0] * A * E.dummy[4] + i17[0] * A * F.dummy[1] + i18[0] * A * F.dummy[2] + i19[0] * A * F.dummy[3] + i20[0] * A * F.dummy[4] + i21[0] * A * G.dummy[1]
+ i22[0] * A * G.dummy[2] + i23[0] * A * G.dummy[3] + i24[0] * A * G.dummy[4] + i25[0] * A * H.dummy[1] + i26[0] * A * H.dummy[2] + i27[0] * A * H.dummy[3] + i28[0] * A * H.dummy[4] + i29[0] * A * I.dummy[1] + i30[0] * A * I.dummy[2] + i31[0] * A * I.dummy[3] + i32[0] * A * I.dummy[4] /
U(alt2) = b2 * A + b3.dummy * B + b4.dummy * C + b5.dummy * D + b6.dummy * E + b7.dummy * F + b8.dummy * G + b9.dummy * H + b10.dummy * I + i1 * A * B.dummy[1] + i2 * A * B.dummy[2] + i3 * A * B.dummy[3] + i4 * A * B.dummy[4] + i5 * A * C.dummy[1] + i6 * A * C.dummy[2] + i7 * A * C.dummy[3] + i8 * A * C.dummy[4] + i9 * A * D.dummy[1] + i10 * A * D.dummy[2] + i11 * A * D.dummy[3] + i12 * A * D.dummy[4] + i13 * A * E.dummy[1] + i14 * A * E.dummy[2] + i15 * A * E.dummy[3] + i16 * A * E.dummy[4] + i17 * A * F.dummy[1] + i18 * A * F.dummy[2] + i19 * A * F.dummy[3] + i20 * A * F.dummy[4] + i21 * A * G.dummy[1]+ i22 * A * G.dummy[2] + i23 * A * G.dummy[3] + i24 * A * G.dummy[4] + i25 * A * H.dummy[1] + i26 * A * H.dummy[2] + i27 * A * H.dummy[3] + i28 * A * H.dummy[4] + i29 * A * I.dummy[1] + i30 * A * I.dummy[2] + i31 * A * I.dummy[3] + i32 * A * I.dummy[4]
$

I asked a colleague of mine about how to enforce this overlap condition, and she suggested using a large number of constraints like this one:

;cond:
if (alt1.A <> alt2.A and alt1.B <> alt2.B and alt1.C <> alt2.C and alt1.D <> alt2.D, alt1.E=alt2.E and alt1.F=alt2.F and alt1.G=alt2.G and alt1.H=alt2.H and alt1.I=alt2.I), etc etc

Thus, we would specify each combination of attributes that differ, and force the others to be the same. This gives a lot of constraints (112 I think), and causes Ngene to fall over. I tried fractional but couldn’t find a fraction which would prevent this.

I also tried defining the condition in terms of reject (i.e reject any profile where 5 are the same, or 6 are different), but that ended up with even more constraints, and the same problem of not being able to find an answer.

;reject:
Alt1.A = Alt2.A and Alt1.B=Alt2.B and Alt1.C=Alt2.C and Alt1.D = Alt2.D and Alt1.E=Alt2.E and Alt1.F=Alt2.F, etc etc

Is there a more sensible way of trying to do this?

Thanks in advance for any suggestions on this,
Richard

The design that you are looking for consists of what the literature calls 'partial profiles'. In a partial profile, only a subset of attributes are displayed to the respondent, assuming that all other attributes not shown have identical levels.

There does not exist an easy way to do this in Ngene. Using constraints is indeed quite tedious. I am thinking of possibly using a heterogeneous design by stacking multiple subdesigns in which only a subset of attributes change their levels. But this will likely lead to many subdesigns and therefore a very large design.

Alternatively, you could try to create all possible allowed choice tasks (e.g. using Excel), in which you only include choice tasks in which 4 attributes have the same level and respondents only need to trade off on the other 4 attributes. This list of choice tasks is called a candidate set. The latest version of Ngene that we have is able to read in such a candidate set from Excel and then select the best 100 choice tasks from this candidate set. This is a feature that will be released in version 1.1.3, but we could provide you with a special built that includes this feature. It does mean that you would need to put a bit of effort in creating the candidate set yourself.

Thanks Michiel - that's really helpful. I appreciate the prompt reply.

I wondered about the candidate option, and had a look at the numbers. If I am calculating correctly, there are 5^9 x 4 possible profiles, meaning (5^18 x 4^2) possible choice sets. Of these, I tried to work out how many had the correct amount of overlap, and I think it is too many to produce a full list of all possible choice sets. I think the proportion of choice sets with the correct overlap is (9!/5!4!)/2^9 (or about 25%), meaning there is a ballpark (5^9 x 4) choice sets that meet the constraint. (I can identify the complete set of generators between A and B that have the right degree of overlap, but presume that isnt useful for Ngene).

One option I discussed with Debbie Street was to use a BIBD to determine which dimensions are changing, then one OMEP to define the initial levels of the differing dimensions, and generators to come up with the option B levels. You can then use another OMEP to populate the non-differing levels. If I understand your suggestion about stacking multiple subdesigns, that sounds fairly similar (but I may of course have completely missed your point!). The method I described produces something that looks plausible, but is very fiddly to assemble and I can't easily rule out human error. Also, it doesn't allow non-zero priors so you are likely to get dominant pairs.

I think that route is likely to be the most fruitful but, if you have any ideas about how to do this better, I would be grateful to hear them.

Richard

Partial profiles have been described in the literature and have been using BIBD designs and other designs as a master design to decide which attributes to vary in each choice task. This all works well, but unfortunately is not implemented in the current version of Ngene yet.

Instead of generating all possible candidates, you could take a subset by simply randomly generating choice tasks that are allowed, it is not necessary to generate all of them (which you indicate could be a very large number).

Excellent. I will try that. I will contact you offline to discuss.

Thanks again,
Richard