DCE with status quo and dominant alternatives

[JvB]

Hello,

I am trying to generate an appropriate design for my DCE, when facing some questions that you might be able to give some information about from your experience:
I have a DCE with one status quo and two alternatives. The level 1.6 of the first attribute is only used in the status quo, for the other two dummy coded variables the status quo level [0] is also used in the alternatives (i.a. due to identifiability issues).
The draft for the syntax is below:

Design
;alts = alt1, alt2, SQ
;rows = 12
;eff = (mnl,d)
;cond:
if(alt1.amount=0<>alt1.period=0),
if(alt2.amount=0<>alt2.period=0)
;model:
U(alt1) = b1[...]*contrib[1.8,2.5,3.2] + b2.dummy[...]*amount[3,2,1,0] + b3.dummy[...]*period[3,2,1,0] /
U(alt2) = b1*contrib + b2*amount + b3*period /
U(SQ) = b1[...]*contrib[1,6] + b2[...]*amount[0] + b3[...]*period[0]

The following issues came to my mind when writing the syntax:
- does it matter using 3,2,1,0 as values for attr2 and attr3 even if the levels itself are different for attr2 and attr3? and do I have to define the levels names somewhere, even if I in the end will import the design into another software to generate the survey?
- did I implement right the constraint trying to say that if level 0 of attr2 is shown attr3 is not allowed to show its level 0 at the same time (being a constraint for both alternatives that´s why I mentioned it for alt1 and alt2)?
- how to define the priors because I am generating the design for my pretest and still do not have any specific priors from literature (thats why I´ve only put three dots in the syntax so far for the priors)? I might know a tendency as all attributes have a natural preference order (lower levels being better than higher levels) but no specific priors so far. Any hints on how to deal with this situation? Also especially with regards to avoid dominant alternatives due to this natural preference order?
- In the end I will try to find heterogeneous preferences between males and females, makes it sense though to include this socio economic variable in the model specification as a covariate?
- I will start with the mnl model as the simplest one (as recommended in the Ngene manual) but perspectively I´d like to make model averaging for mnl and mmnl just not knowing if to use the mmnl oder the panel mmnl approach?

Thank you in advance.
Best regards.

[Michiel Bliemer]

Hi,

1. I always recommend to put the description of each dummy coded level as a comment in the sytax so that you know which level belongs to what categorical description. In the survey instrument, you simply type the text that you would like to show for each level.

2. The correct constraint would have been: if(alt1.amount = 0, alt1.period <> 0). I have replaced it with a reject constraint since the require constraint is needed for setting the dummy coded variables to a specific value in the status quo alternative. Require/reject constraints require the modified Federov algorithm. Since this algorithm relaxes attribute level balance, I added the constraint (4,4,4) to make sure that each level for contrib appears equally. Such constraints are generally not needed for dummy coded variables.

3. See syntax below. Note that your status quo alternative is always dominant since it has the lowest (best) value for each of the attributes, so please check if this is correct. To check for dominance, you can use ;alts = alt1*, alt2*, SQ*. Using this will now not generate any design because DQ is dominant.

4. You can add a covariate for gender, but at the design stage this is quite unusual. So unless you expect behaviours between male/female to be very different, then I would recommend not including it at the design stage and only add covariates in the estimation stage.

5. I recommend creating a design for the mnl model only. While it is possible to generate designs for the panel mixed logit model, the latter is very difficult since it requires a very large amount of computation time. Using the cross-sectional mixed logit model is not appropriate for choice experiments unless you only give a single choice task to each respondent. I strongly suggest that you optimise the design for the mnl model, which will also work for estimating panel mixed logit models. What you may want to consider is increasing the number of rows to create more variation in your data, which is often needed for estimating more advanced models. So using ;rows = 24 with ;block = 2 for example.

Code: Select all

Design
;alts = alt1, alt2, SQ
;rows = 12
;eff = (mnl,d)
;alg = mfederov
;require:
SQ.amount = 0, SQ.period = 0
;reject:
alt1.amount = 0 AND alt1.period = 0,
alt2.amount = 0 AND alt2.period = 0
;model:
U(alt1) = b0[0]
        + b1[-0.0001]                       * contrib[1.8,2.5,3.2](4,4,4)
        + b2.dummy[-0.0003|-0.0002|-0.0001] * amount[3,2,1,0]  ? 0 = ..., 1 = ..., 2 = ..., 3 = ...
        + b3.dummy[-0.0003|-0.0002|-0.0001] * period[3,2,1,0]  ? 0 = ..., 1 = ..., 2 = ..., 3 = ...
        /
U(alt2) = b1 * contrib
        + b2 * amount
        + b3 * period
        /
U(SQ)   = b1 * contrib_sq[1.6]
        + b2 * amount
        + b3 * period
$

Michiel

[JvB]

Dear Michiel,

thank you very much for your helpful answers. I really appreciate your support.
From your modified syntax I can see that you´ve added b0[0] to the alt1 utility function. Can you tell what b0 stands for and why you´ve put it to a fixed level of [0] and only in the utility of alt1? Furthermore, why did you choose [-0.0003|-0.0002|-0.0001] as priors for attr2 and attr3 and why does attr1 only have one prior [-0.0001] ? Maybe I am asking stupid questions but I am a bit confused now.
From what you are telling about the dominant alternatives I have noticed that a lower number is always better than a higher number, but following that logic for attr2 and attr3 the SQ level would be 3 instead of 0. So the status quo will not be a dominant alternative as the first part of the syntax would be modified to:

Design
;alts = alt1, alt2, SQ
;rows = 12
;eff = (mnl,d)
;alg = mfederov
;require:
SQ.amount = 3, SQ.period = 3
;reject:
alt1.amount = 3 AND alt1.period = 3,
alt2.amount = 3 AND alt2.period = 3

As the SQ level [3] should be the base level for attr2 and attr3 I might need to change the order from "amount[3,2,1,0]" to "amount[0,1,2,3]"? Not being sure if the priors or other parts of the model specification need to be changed due to this change in order of level and SQ level changed to 3 instead of 0?

I´ve read in the manual (bottom of page 114) that for dummy coded variables it might be an issue that the design might become sparse in terms of non-zero values. Do you see any issue regarding this with my design?
With regards to the manual I also did not completely get the idea of optimizing for sample size? Can I get an idea of a needed sample size for my design with that or can I optimize my design supposing a specific planned sample size?

Thank you very much in advance.
Best regards!

[Michiel Bliemer]

I added the constant in the wrong alternative, I should have added it to SQ (or to both alt1 and alt2), see below. It may be expected that people prefer to choose the alternative that is most familiar to them, so I would generally add a constant in the model to test the null hypothesis of b0 = 0.

If you are dummy coding an attribute with 4 levels, you are estimating 3 parameters, therefore you need to use 3 priors. If you are not using dummy coding then you are only estimating 1 parameter and therefore you only need to specify one 1 prior. The value -0.0001 for b1 indicates that higher contrib means lower utility. The prior values for the dummy coded parameters indicate the utility relative to the base, which is by default the last level in Ngene. So assuming that 3 is the base level, then with b2.dummy[0.0003|0.0002|0.0001] I indicate that level 0 has the highest utility and that utility decreases with increasing levels for amount. These values are essentially zero, but I am using these near-zero values to indicate ONLY the preference order of the attribute levels so that I can automatically let Ngene remove the dominant alternatives.

Updated syntax is below, which now automatically avoids dominant alternatives.

Code: Select all

Design
;alts = alt1*, alt2*, SQ*
;rows = 24
;block = 2
;eff = (mnl,d)
;alg = mfederov
;require:
SQ.amount = 3, SQ.period = 3
;reject:
alt1.amount = 3 AND alt1.period = 3,
alt2.amount = 3 AND alt2.period = 3
;model:
U(alt1) = b1[-0.0001]                       * contrib[1.8,2.5,3.2](8,8,8)
        + b2.dummy[0.0003|0.0002|0.0001] * amount[0,1,2,3]  ? 0 = ..., 1 = ..., 2 = ..., 3 = ...
        + b3.dummy[0.0003|0.0002|0.0001] * period[0,1,2,3]  ? 0 = ..., 1 = ..., 2 = ..., 3 = ...
        /
U(alt2) = b1 * contrib
        + b2 * amount
        + b3 * period
        /
U(SQ)   = b0[0]
        + b1 * contrib_sq[1.6]
        + b2 * amount
        + b3 * period
$

I do not believe that there is an issue with dummy coding here, you only have 4 levels.

Optimising for sample size can only be done if you have informative priors obtained from a pilot study so that you can replace 0.0003, 0.0002 and 0.0001 etc with actual dummy coded parameter values. Without informative priors, sample size computations are meaningless.

Michiel

[JvB]

Dear Michiel,

thank you very much for these insights. Now everything is very clear.
I only have one more questions: I might want to check for reliability with the help of fixed choice tasks (probably three fixed tasks, two tasks being the same). Does it make sense to do so and is it possible to put that to the syntax as well? And if yes, can I vary the position of the fixed tasks per candidate set?

Thank you very much!

[Michiel Bliemer]

I assume that you would like to check for consistency (same repeated choice task) or whether people pay attention (choice task with dominant alternative)? These you can simply add manually, Ngene would not generate these because this decreases efficiency in the design.

I think that using a dominant alternative in a choice task towards the end of the experiment is OK (I would not put it at the beginning of the experiment because the respondent may not take the experiment seriously if they receive a "silly choice task"), although you will of course have to remove data from this choice task before model estimation. This means that you will lose some efficiency.

Using the same choice task and then removing the respondent from the data if they fail to choose the same choice is something I would personally not do. Is it inconsistent that sometimes I buy cake and other times I buy cookies? People like variety and alternatives can have similar utilities. Further, choices are up to a probability, so choice probabilities of 60-40% means that it there is a 60% chance of selecting alternative 1 and 40% chance of selecting alternative 2, which means that it is not unexpected to see a respondent choose different alternatives for the same choice task. If you include them, you can put them anywhere in the experiment.

Michiel

[JvB]

Dear Michiel,

I have now run Ngene to generate a design with my syntax.
First of all, I received the following error message:

Error: The modified Federov candidate set size of 2000 could not be achieved. The percentages of candidates that failed are: 95.37% due dominance, 0% due constraints, and 4.63% due repeated alternatives. The candidate set size has been adjusted from 2000 to 1062.

But I suppose that it is just fine that Ngene reduced the default candidate set size and I do not need to worry about that?

Furthermore I have the following design properties, which look okay to me. Or do you see any problem here? I think the s estimate being so high is fine as I do not have any priors from pilot study so far?

D error 0.461262
A error 0.612549
B estimate 99.999997
S estimate 254428685.052612

The Sp estimates are also quite high with
91394263.407175 29554317.433439 68180272.847149 254428685.052612 29377797.23618 61213505.496704 253430680.065377
Sp t-ratios being quite low with
0.000205 0.000361 0.000237 0.000123 0.000362 0.000251 0.000123

I suppose that this is also due to the chosen priors that ONLY should avoid dominance in this design? So I should not worry about this as well?

Regarding mnl probabilities alt1, alt2 and sq have almost the same probability (each one around 0.33 in every choice task). That is a good thing, isnt´it?
Mnl utilities look for some choice sutuations a bit strange, maybe you can explain how to interpret them?
alt1 alt2 sq
0.00035 2E-05 -0.00016
-2E-05 0.00022 -0.00016

I am thinking about changing my number of choice task per candidate to 9. The modified syntax then would be:

Design
;alts = alt1*, alt2*, SQ*
;rows = 18
;block = 2
;eff = (mnl,d)
;alg = mfederov
;require:
SQ.amount = 3, SQ.period = 3
;reject:
alt1.amount = 3 AND alt1.period = 3,
alt2.amount = 3 AND alt2.period = 3
;model:
U(alt1) = b1[-0.0001] * contrib[1.8,2.5,3.2](6,6,6)
+ b2.dummy[0.0003|0.0002|0.0001] * amount[0,1,2,3] ? 0 = 300, 1 = 600, 2 = 900, 3 = unlimited amount
+ b3.dummy[0.0003|0.0002|0.0001] * period[0,1,2,3] ? 0 = 12, 1 = 42, 2 = 72, 3 = unlimited period
/
U(alt2) = b1 * contrib
+ b2 * amount
+ b3 * period
/
U(SQ) = b0[0]
+ b1 * contrib_sq[1.6]
+ b2 * amount
+ b3 * period
$

I suppose that changing (8,8,8) to (6,6,6) and rows to 18 instead of 24 (as in syntax above) will be the necessary adjustments and that changing to 9 choice tasks in general can be done? Accepting that d-error will be higher (around 0.60).
Thank you very much for your support.

[Michiel Bliemer]

1. The error message is just a warning but it is indeed fine as Ngene has automatically adjusted the candidate set.

2. Efficiency measures are case specific, this means it is not possible to judge what a good D-error or A-error value is. The only thing I can judge is that they are finite, this means that the model can be estimated. So I do not see any obvious issues.

3. S-estimates are only meaningful with appropriate priors. Since you have used near-zero priors they should be ignored, you do not need to worry about this as they will always be huge when using near-zero priors that indicate the preference order of attribute levels to allow dominance checking.

4. All utilities will be near-zero since you used near-zero priors. This results in near-equal choice probabilities. Why do you believe that your utilities look strange? It indicates that alt1 has a utility of 0.00035, alt2 has a utility of 0.00002, and sq has a utility of -0.00016, which are all near-zero as expected with your priors.

5. Yes you can switch to 18 rows if you prefer, which indeed means that the constraint becomes (6,6,6).

Michiel

[JvB]

Hi Michiel,

thank you very much for rechecking my design output!
Just one more thing came to my mind, when thinking about estimating results after fielding:
Might there be a distortion due to the fact that the level 1.6 of attr1 is only used in status quo and therefore only appears with levels3 of attr2 and attr3? So 1.6 might have a negative utility because of being shown only with attr2 level 3 and attr3 level 3 (which are not very attractive) and not because it really was not preferred?

I hope it is understandable what I´m trying to say?!
Your advice is highly appreciated!
Thank you!

[Michiel Bliemer]

Since contrib is a continuous numerical variable, it should not matter that it has a lower value in the status quo alternative since respondents will still be making trade-offs between all the attributes. It is for example quite common for a status quo alternative to have the highest or lowest level for one or more attributes. If contrib was dummy coded then there may have been an issue, but it is fine for a numerical variable where only a single coefficient is estimated for all attribute levels.

Michiel