NGene designs syntax for a 2 steps choice experiment

[admin]

Posted on behalf of acv due to a technical error when posting.

Dear all,

We are conducting a choice experiment to determine the preferences and WTA of a sample of people for different kinds of contracts. We have 4 attributes to describe the contracts with 4, 4, 2 and 4 levels respectively. The first and third attributes are qualitative ones while the second and the fourth are quantitative ones. For each alternative, all the attributes and all the levels of the attributes may appear. In each choice set, respondents will have the choice between 3 options: 2 alternatives combining the levels of our 4 attributes and an opt-out option (which is “keeping your current situation”, without any new contract). The alternatives are unlabeled.

There will be a pilot study to define the priors of our utility function, using an (orthogonal) efficient design with all the priors set to 0. And then, we will use these priors in order to generate the final choice sets for the study, using an efficient design (or maybe a Bayesian efficient design).

We used the %mktruns macro, available with SAS, to determine the number of alternatives we will have to propose to respondents. %mktruns tells us that the smallest design should have 16 rows so, with our 2 alternatives + the opt out alternative per choice set, we will have to create 8 choice sets. It should not be necessary to block our designs because filling 8 choice sets is not so difficult for the respondents.

We are using NGene to create our designs.

We have five questions arising from the preparation of our 2 designs using Ngene (pilot study and final study):

1 – Because our alternatives are unlabeled, we plan to use a constant only in the utility function of the opt-out option, considering people may have a utility while keeping their current situation. Is this correct?

2 – Is it correct to repeat the value of our priors in the alt2 utility function in the design of the final study because we have used asterisks in order to avoid dominant or identical alternatives within a choice set (based on the fact that our alternatives are unlabeled)?

3 – Our research area is environmental economics. We wonder if it would be interesting to use the Bayesian approach to have random priors around the value of the priors we would have determined with the pilot study. Or is it too early to use a Bayesian approach in a social science piece of work?

4 – We have understand with %mktruns that the number displayed for the size of design (16, using SAS) is the number we would have to use in our design for the line ;rows (;rows=16). But we noticed that NGene outputs for the design already propose two alternatives per row. Therefore, we have 16 choice cards instead of 8. So, do we have to change the number in our design (;rows=8) in order to have 8 choice sets? Or do we have to let ;rows=16 and submit 16 choice sets to our respondents? In the latter case, we would have to block the design into 2 blocks of 8 choice sets (using an extra line in our design ;block=2) because 16 choice sets per respondent would be too much.

5 – Considering the previous questions, are these designs correctly written? For the efficient design of the final study, we put random values for the priors since we did not yet conduct the pilot study.

Design for the pilot study

Design
;alts = alt1, alt2, alt3
;rows=16
;eff=(mnl,d)
;model:
U(alt1) = b1.dummy[0|0|0]*A[2,3,4,1] + b2*B[9,18,25,40] + b3.dummy*C[1,0] + b4*D[400,750,1100,1500] /
U(alt2) = b1*A + b2*B + b3*C + b4*D /
U(alt3)= asc
$

This design works, the D-error is 0.020317

Design for the final study

Design
;alts = alt1*, alt2*, alt3
;rows=16
;eff=(mnl,d)
;model:
U(alt1) = b1.dummy[0.25|-0.13|0.11]*A[2,3,4,1] + b2[-0.12]*B[9,18,25,40] + b3.dummy[0.30]*C[1,0] + b4[0.4]*D[400,750,1100,1500] /
U(alt2) = b1.dummy[0.25|-0.13|0.11]*A[2,3,4,1] + b2[-0.12]*B[9,18,25,40] + b3.dummy[0.30]*C[1,0] + b4[0.4]*D[400,750,1100,1500] /
U(alt3)= asc[0.3]
$

Note that the priors have been manually chosen. We first used higher priors (0.78 for instance) but the design did not work. With the current lower priors, the design works.

Design for the final study (Bayesian version)

Design
;alts = alt1*, alt2*, alt3
;rows=16
;eff=(mnl,d,mean)
;model:
U(alt1) =
b1.dummy[(n,0.25,0.2)| (n,-0.13,0.3)| (n,0.11,0.2)]*A[2,3,4,1] +
b2[(n,-0.12,0.2)]*B[9,18,25,40] +
b3.dummy[(n,0.30,0.3)]*C[1,0] +
b4[(n,0.4,0.5)]*D[400,750,1100,1500] /
U(alt2) = b1.dummy[(n,0.25,0.2)| (n,-0.13,0.3)| (n,0.11,0.2)]*A[2,3,4,1] +
b2[(n,-0.12,0.2)]*B[9,18,25,40] +
b3.dummy[(n,0.30,0.3)]*C[1,0] +
b4[(n,0.4,0.5)]*D[400,750,1100,1500] /
U(alt3)= asc[0.3]
$

This design does not work. NGene displays the following error message “Warning: No valid design has been found after 1000 evaluations. There may be a problem with the specification of the design. A common problem is that the choice probabilities are too extreme (close to 1 and 0), perhaps because some or all of the prior values are too large. Also, it is generally a good idea to start with a simple design (MNL, non-Bayesian), then add complexity. If you press stop, a design will be reported, which may assist in diagnosing the problem.”. It may be because our priors have been manually chosen?

Thanks a lot for your help

[acv]

Dear admin,
Thank you for having posted my message and for organizing this very helpful forum
Have a nice day,
Anne-Charlotte

[Michiel Bliemer]

1. Yes you can include a constant for the status quo alternative
2. Yes you need to use the same prior values in unlabelled alternatives, which allows you also to check for dominant alternatives with the *
3. Bayesian priors are becoming state-of-the-practice now, I would strongly suggest using Bayesian priors. You should include ;bdraws in your design since the default number of draws is not enough with 6 Bayesian priors. I would suggest using ;bdraws = gauss(3). But note that you should be VERY CAREFUL setting prior values, in your case you have put in manual values and THEY DO NOT MAKE SENSE, hence Ngene cannot find a design. You should not just set them manually without thinking what behaviour is implied. Priors should come from a pilot study and should not be determined manually unless you are an expert in doing this. Particularly your b4 is problematic, you are multiplying 0.4 with very large numbers such as 1500, such that your D attribute completely dominates the design and it cannot be optimised. Please use priors that make behavioural sense.
4. I cannot comment om %mktruns as I am not familiar with SAS. ;rows refers to the number of choice tasks you show to a respondent, so if you would like to use 8 choice tasks, then set ;rows = 8.
5. See my comment in 3.

[acv]

Dear Michiel,

Thank you very much for your reply, your helpful comments and this forum.

Regarding the priors, we still did not begin the pilot study so this is why we tested the design with “false” priors. Unfortunately, it seems we have chosen unrealistic priors. But the final design will be created with the true priors resulting from the pilot study. We will include ;bdraws in our design, thanks for the advice.

Ok for %mktruns. Yes, if we realize that the design size means the number of alternatives then we will only have to show 8 choice tasks (including 2 alternatives and the opt-out option) to our respondents so we will have to set ;rows = 8 in our design instead of ;rows = 16.

Have a good day,
Anne-Charlotte