choice-metrics.com

Hello,

I am currently in the midst of designing an unlabelled choice experiment and I am unsure about including constraints to ensure realistic choice sets.
The respondents are supposed to choose amongst 3 unlabelled options or an opt-out alternative ("I do not want to choose any of the alternatives.").

In order to avoid unrealistiv choice sets, I have to account for the fact that the cost attribute is dependent on other attributes in reality: A higher level in "A" as well as "B" should always be combined with a larger or equal level in the "cost" attribute compared to lower levels.

Code: Select all

Design
;alts = opt1*, opt2*, opt3*, optout
;rows = 180
;block = 30
;eff = (mnl, d)

? Conditions go here

;model:
U(opt1) = b1[0.000001]*A[1,2,3] + b2[0.000001]*B[1,2,3] + b3*C[20,40,60,80] + b4[-0.000001]*cost[20000:80000:10000] + b5[0.000001]*E[500:800:100] + b6[0.000001]*F[2,4,6,8]  /

U(opt2) = b1*A+ b2*B+ b3*C+ b4*cost+ b5*E+ b6*F/

U(opt3) = b1*A+ b2*B+ b3*C+ b4*cost+ b5*E+ b6*F/

U(optout) = b7[0]

$


I have 3 questions regarding this design:

1) Are these kind of constraints viable or do they lead to difficulties in generating a design or interpreting the results in the end? I am wondering about the latter, since I do not know of unlabelled choice experiments in the literature that impose such cost constraints.

2) How would I go on to implement this in my design? I am unsure whether ";cond", ";require" or ";reject" is the correct way to implement these constraints.

3) Is my specification of the opt-out alternative correct? I found this specification in the forum but the Ngene manual (https://choice-metrics.com/NgeneManual120.pdf) (p. 109) says to not include a utility function for the opt-out option at all.

I am looking forward for any tips/comments.
Tom

1) You can include constraints, but you should always avoid creating perfectly correlated variables, so you should not use if(opt1.A=1, opt1.cost=20000) but rather if(opt1.A=1, opt1.cost=[20000,30000,40000]) or something similar.

2) Using the default swapping algorithm, you can only use conditional constraints with ;cond: if(...). I tried them in the script below, but if you run it Ngene will say that it cannot find a design because there are too many cases of dominance.

Code: Select all

Design
;alts = opt1*, opt2*, opt3*, optout
;rows = 180
;block = 30
;eff = (mnl, d)

;cond:
if(opt1.A=1, opt1.cost=[20000,30000,40000]),
if(opt1.A=2, opt1.cost=[40000,50000,60000]),
if(opt1.A=3, opt1.cost=[60000,70000,80000]),
if(opt2.A=1, opt2.cost=[20000,30000,40000]),
if(opt2.A=2, opt2.cost=[40000,50000,60000]),
if(opt2.A=3, opt2.cost=[60000,70000,80000]),
if(opt3.A=1, opt3.cost=[20000,30000,40000]),
if(opt3.A=2, opt3.cost=[40000,50000,60000]),
if(opt3.A=3, opt3.cost=[60000,70000,80000])

;model:
U(opt1) = b1[0.000001]*A[1,2,3] + b2[0.000001]*B[1,2,3] + b3*C[20,40,60,80] + b4[-0.000001]*cost[20000:80000:10000] + b5[0.000001]*E[500:800:100] + b6[0.000001]*F[2,4,6,8]  /

U(opt2) = b1*A+ b2*B+ b3*C+ b4*cost+ b5*E+ b6*F/

U(opt3) = b1*A+ b2*B+ b3*C+ b4*cost+ b5*E+ b6*F/

U(optout) = b7[0]

$

So next, you would need to switch to the modified Federov algorithm, see script below, and use reject constraints.

Code: Select all

Design
;alts = opt1*, opt2*, opt3*, optout
;rows = 180
;block = 30
;eff = (mnl, d)
;alg = mfederov

;reject:
opt1.A=1 and opt1.cost=50000, opt1.A=1 and opt1.cost=60000, opt1.A=1 and opt1.cost=70000, opt1.A=1 and opt1.cost=80000,
opt1.A=2 and opt1.cost=20000, opt1.A=2 and opt1.cost=30000, opt1.A=2 and opt1.cost=70000, opt1.A=2 and opt1.cost=80000,
opt1.A=3 and opt1.cost=20000, opt1.A=1 and opt1.cost=30000, opt1.A=1 and opt1.cost=40000, opt1.A=3 and opt1.cost=50000,
opt2.A=1 and opt2.cost=50000, opt2.A=1 and opt2.cost=60000, opt2.A=1 and opt2.cost=70000, opt2.A=1 and opt2.cost=80000,
opt2.A=2 and opt2.cost=20000, opt2.A=2 and opt2.cost=30000, opt2.A=2 and opt2.cost=70000, opt2.A=2 and opt2.cost=80000,
opt2.A=3 and opt2.cost=20000, opt2.A=1 and opt2.cost=30000, opt2.A=1 and opt2.cost=40000, opt2.A=3 and opt2.cost=50000,
opt3.A=1 and opt3.cost=50000, opt3.A=1 and opt3.cost=60000, opt3.A=1 and opt3.cost=70000, opt3.A=1 and opt3.cost=80000,
opt3.A=2 and opt3.cost=20000, opt3.A=2 and opt3.cost=30000, opt3.A=2 and opt3.cost=70000, opt3.A=2 and opt3.cost=80000,
opt3.A=3 and opt3.cost=20000, opt3.A=1 and opt3.cost=30000, opt3.A=1 and opt3.cost=40000, opt3.A=3 and opt3.cost=50000

;model:
U(opt1) = b1[0.000001]*A[1,2,3] + b2[0.000001]*B[1,2,3] + b3*C[20,40,60,80] + b4[-0.000001]*cost[20000:80000:10000] + b5[0.000001]*E[500:800:100] + b6[0.000001]*F[2,4,6,8]  /

U(opt2) = b1*A+ b2*B+ b3*C+ b4*cost+ b5*E+ b6*F/

U(opt3) = b1*A+ b2*B+ b3*C+ b4*cost+ b5*E+ b6*F/

U(optout) = b7[0]

$


Note that the modified Federov algorithm does not impose attribute level balance, so you may see in the resulting design that it mainly uses outer levels for your numerical attributes. You can either impose attribute level constraints in your utility functions, such as b1[...] * A[1,2,3](40-80,40-80,40-80). Alternatively, you can dummy code all variables. The latter is my preferred way of dealing with this if you have uninformative near-zero priors and this automatically results in a design that is reasonable attribute level balanced.

Further, note that the modified Federov algorithm is quite slow when you have so many rows (in your case, 180), so you will need to run it for a long time.

3) Yes it is correct. The utility function of the optout is either a constant only (as in your case), or zero (and then you put the constant in opt1, opt2, and opt3). If it is zero, then you simply leave out the alternative in the specification.

Michiel

PS: Please post Ngene related questions in the Ngene forum.

Dear Michiel,

thank you so much for your reply. If I understand your proposal correctly, it would restrict the cost levels of each level of attribute "A" to a certain range of costs that slightly overlap. Does this not affect the interpretation of preferences for attribute "A", as level 3 of attribute "A" (almost) always has higher cost levels than level 2 and 1? Can we still tell whether respondents choose level 1 of "A" because of a preference for level 1 or because it is always cheaper than levels 2 and 3? Is this why you included the overlap in costs? My idea was to have this cost restriction within each choice set, allowing every price level for each level of "A":(cost|A=1) <= (cost|A=2) <= (cost|A=3).

Thank you for your help.
Tom

With the proposed constraints, that are indeed specifically chosen to overlap, you can separately estimate preferences towards A and C.

My reject constraints can also be written as:

;reject:
opt1.A=1 and opt1.cost>=50000,
opt1.A=2 and opt1.cost<=30000, opt1.A=2 and opt1.cost>=70000,
opt1.A=3 and opt1.cost<=20000,
opt2.A=1 and opt2.cost>=50000,
opt2.A=2 and opt2.cost<=30000, opt1.A=2 and opt1.cost>=70000,
opt2.A=3 and opt2.cost<=20000,
opt3.A=1 and opt3.cost>=50000,
opt3.A=2 and opt3.cost<=30000, opt1.A=2 and opt1.cost>=70000,
opt3.A=3 and opt3.cost<=20000

This means that, within each choice task, if A=1 then the cost should be smaller than 50000, etc. Is this not the same as what you propose?

Michiel

Dear Michiel,

thank you for your response.

Regarding that you said that dummy coding all variables is your preferred method of achieving level balance, how would I do that? Is the following correct assuming that higher costs yield more negative utility?

Code: Select all

b4.dummy[-0.000001|-0.000002|-0.000003|-0.000004|-0.000005|-0.000006]*cost[20000:80000:10000]


And would I still be able to include the cost parameter as a continuous variable in my models estimation alter on?

Kind regards
Tom

Like this, where the LAST level in Ngene is the reference level:

Code: Select all

U(opt1) = b1.dummy[ 0.01| 0.02]                         * A[2,3,1]
        + b2.dummy[ 0.01| 0.02]                         * B[2,3,1]
        + b3.dummy[ 0   | 0   | 0   ]                   * C[20,40,60,80]
        + b4.dummy[-0.01|-0.02|-0.03|-0.04|-0.05|-0.06] * cost[30000,40000,50000,60000,70000,80000,20000]
        + b5.dummy[ 0.01| 0.02| 0.03]                   * E[600,700,800,500]
        + b6.dummy[ 0.01| 0.02| 0.03]                   * F[4,6,8,2]  /


Yes, you can estimate a single coefficient (a linear effect) considering the variables as numerical attributes afterwards.

Michiel