D-efficient design for pilot test

[ZiW]

Dear Prof. Bliemer, Prof. Collins, Prof. Johnr and Ngene users

First of all, many thanks for providing such a possibility to post questions on this forum. The problems have been bothering me for quite a while.

I am a new user of Ngene and currently preparing a D-efficient design for the pilot study of a labelled choice experiment and will then generate a Bayesian Efficient design for the main survey based on the priors obtained in the pilot.

1. Target:
Compare the Current option (alt1) with three alternatives (alt2, alt3, alt4).
In order to make it not too difficult for the respondents to choose from 4 alternatives of one time, each choice situation will present three labelled alternatives, alt1 and two other Alternatives among alt2, alt3 and alt4. This has been realized by giving the attributes a huge value “9999” in the input data.

2. Design :

Code: Select all

Design
;alts = (alt1?), alt2, alt3, alt4
;rows = 40
;block = 5
;eff=(mnl,d)
;con
;alg = mfederov(candidates = full4_output_zw.xlsx)

;model:

(U(alt1) = b10 + ?)

U(alt2) = b8
+ b1 * alt2_cost1[0.5,0.75,1.0,1.25,1.5,9999]         
+ b2 * alt2_cost2[0.7,1.0,1.3,9999]         
+ b3 * alt2_wt[2,6,12,18,25,9999] /

U(alt3)    = b9
+ b1 * alt3_cost1[0.5,0.75,1.0,1.25,1.5,9999]         
+ b2 * alt3_cost2[0.7,1.0,1.3,9999]                
+ b4 * alt2_wt[2,6,12,18,25,9999]    
+ b5 * alt3_extrat[0.05,0.1,0.25,0.5,1.0,9999] /

U(alt4)    = b1 * alt4_cost1[0.5,0.75,1.0,1.25,1.5,9999]         
+ b2 * alt4_cost2[0.7,1.0,1.3,9999]                
+ b6 * alt4_wt[2,6,12,18,25,9999] /   
+ b7 * alt4_extrat[0.05,0.1,0.25,0.5,1.0,9999] /
$


3. Questions:
1) The attributes of Alt1 are invariant. Do I need to take Alt1 into account? However they don’t vary through all choice situations. If yes, should I tip only the alternative specific constant or also other attributes “alt1_cost1” and “alt1_wt”?
2) Because we haven’t any information about the coefficient prior, can I just let it be “0”?
3) A warning show up when I start to run: one or more attributes will not have level balance with the number of rows specified. Will it cause any problem or affect the pilot study? I tried to increase the number of rows to 60, and then the warning disappears. How can I determine how many rows is needed?
4) After a long time run I received a result. It shows that only two of five values of attributes alt2_wt and alt3_wt have been chosen. Is it because of “unbalanced level”? Will reducing attribute level help? E.g. alt2_wt[5,12,18,25,9999] instead of [2,6,12,18,25,9999].
I read it somewhere that the modified Federov algorithm does not guarantee attribute level balance. So I tried to give these attributes level constraints, such as ac_ZuAbgangszeit[1,2,3,4,5,9999](5-7,5-7,5-7,5-7,5-7,5-7), then an error showed up: [Modified Federov] ERROR: Unable to find a design that satisfies the specified attribute level frequency constraints. You may need to relax some attribute level constraints, combinatorial constraints (if specified), or dominance checks. The candidate set has 10000 rows. How can I solve this problem?

Thanks a lot for your help!

Zi

[Michiel Bliemer]

1.
You generally need to specify the full utility function for status quo or reference alternatives, only opt-out alternatives should only have a constant. If this alternative is the same for each respondent, then it could be something like:

U(alt1) = b10 + b1 * alt1_cost1[1] + b2 * alt1_cost2[0.7] + ... etc.

If alt1 has alternative-specific coefficients, e.g. b11 * alt1_wt[12], then this cannot be added because for b11 the attribute levels do not vary. Instead, you would need to make it generic, i.e. b4 * alt1_wt[12], or otherwise it will be included in the alternative-specific constant b10.

2.
Usually, yes. But since you are using 9999 in order to create a partial choice set design where one alternative drops out, you need to specify the design. Note that 0*9999 = 0, therefore the probability of this alternative does NOT become 0. A probability of an alternative only becomes zero if its utility become a large negative number, e.g. -100. In your case, cost is expected to have a negative value, so you can set b1[-0.001] to indicate a negative prior essentially equal to 0. Then your large value "9999" needs to be slightly larger, e.g. 99999, such that -0.001 * 99999 = -99.9999

3.
It is a warning because the modified Federov algorithm can never guarantee attribute level balance. As proposed in the Ngene manual, you can impose attribute level contraints such as b1 * alt2_cost1[0.5,0.75,1.0,1.25,1.5,9999](3-7,3-7,3-7,3-7,3-7,0-40), such that with 40 rows each attribute level appears between 3 and 7 times (noting that them appearing 5 times would mean attribute level balance). It is not a problem for your model if the data is not attribute level balanced, but you may observe that certain levels (usually the middle levels) do not appear much in your data and this can be avoided by setting constraints. Be careful not to overconstrain the model because Ngene may no longer be able to find a design that satisfies all your constraints. Using 5-7 is far too strict. If it is difficult to get it to work with constraints, you can try to formulate all attributes using dummy coding. Using dummy coding, the attribute levels will automatically be fairly balanced, i.e. you can generate and optimise a design using dummy coding while estimating a model with linear attributes.

4.
See comment above. If it takes a very long time before you see a result, you candidate set may be too large. I generally limit the candidate set to 10,000 rows, but often 1000 is enough. If you are using a large number of rows you need to let Ngene evaluate a large number of designs. With 40 rows, you would need to let Ngene evaluate at least 400,000 designs in order for the modified Federov algorithm to replace each of the 40 rows with each of the 10,000 rows.

Michiel

[ZiW]

Dear Prof. Bliemer,

Thanks for your advices. Your answers really solve all my current problems.
And sorry for the delay in responding, this project has me a bit overwhelmed. We have a lot of internal discussion recently and also changes in Design.
I'm pretty sure there are many questions i want to ask you, but i need some more time to sort out my thoughts before.

Thanks again for your kind help!

Best Regards,

Zi