mfederov and negative value for MNL utilities

[bichxuanbui]

Dear all,

I am a new Ngene user and have some troubles. I really need your help.
I am conducting an efficent design having 4 attributes with levels of 2, 3, 4 , and 4. One is dummy attribute level and others are continuous. There are 2 unlabel alternatives and 1 status quo in my choice set.
I used both command ;reject and ;require, so when I use algorithm of mfederov with candidates=96, Ngene doen't work. But when I reduce cadidates=25 the Ngene responded. So my questions are:
1. Whether candidates=25 enough for running or not?
2.I don't understand about the number showed in "Sp estimates ans Sp t-ratio" in the result. Could you please explain it?
3. I got a negative sign in the result of MNL utilities that is choice situation 6 - alt 2. Is that a problem with my design?

Below are the design and results:


Design
;alts = alt1*, alt2*, SQ
;rows = 12
;block = 2
;alg = mfederov(candidates=25)
;eff = (mnl,d)
;reject:
alt1.Env = 2 and alt1.Job = 200,
alt2.Env = 2 and alt2.Job = 200
;require:
SQ.Env = 2

;model:
U(alt1) = bcor[0.16] * Cor[20,30] + benv.dummy[2.77|1.58] * Env[0,1,2] + bjob[-0.0025] * Job[0,50,100,200] + bfee[-0.02] * Fee[20,50,100,200] /
U(alt2) = bcor * Cor + benv * Env + bjob * Job + bfee * Fee /
U(SQ) = bcor * CorSQ[13] + benv * Env + bjob * JobSQ[0] + bfee * FeeSQ[0] $

Result:

Prior bcor benv(d0) benv(d1) bjob bfee
Fixed prior value 0.16 2.77 1.58 -0.0025 -0.02
Sp estimates 1.324133 1.053339 2.616871 19.4062 0.948635
Sp t-ratios 1.703297 1.90973 1.211616 0.444924 2.012364

MNL utilities
Choice situation alt1 alt2 sq
1 2.255 1.47 2.08
2 3.55 3.97 2.08
3 1.97 3.88 2.08
4 4.47 5.38 2.08
5 4.255 3.28 2.08
6 0.675 -0.8 2.08
7 5.48 3.97 2.08
8 4.47 2.53 2.08
9 6.32 5.98 2.08
10 4.4 4.97 2.08
11 1.72 4.275 2.08
12 2.38 3.845 2.08

Thank you very much for help.

Xuan

[Michiel Bliemer]

It seems that the modified Federov algorithm has trouble allocating a design that satisfies all constraints as well as is estimable. Since you have a dummy coded variable, the reference level must appear (job = 2) must appear in each alternative in order to be estimable. Further, the modified Federov algorithm does not look at attribute level balance, so you may want to include some minimum upper and lower bounds on the number of times each attribute level appears.

I tried running your syntax with larger numbers of candidates, and sometimes it works, and sometimes is does not. This is because the algorithm has random generators.

You can easily rewrite your utility functions to make it easier for Ngene to locate a design:

Code: Select all

Design
;alts = alt1*, alt2*, SQ
;rows = 12
;block = 2
;alg = mfederov(candidates=1000)
;eff = (mnl,d)
;reject:
alt1.Env = 2 and alt1.Job = 200,
alt2.Env = 2 and alt2.Job = 200
;model:
U(alt1) = bcor[0.16] * Cor[7,17](5-7,5-7) + benv.dummy[2.77|1.58] * Env[0,1,2](3-5,3-5,3-5) + bjob[-0.0025] * Job[0,50,100,200](2-4,2-4,2-4,2-4) + bfee[-0.02] * Fee[20,50,100,200](2-4,2-4,2-4,2-4) /
U(alt2) = bcor * Cor + benv * Env + bjob * Job + bfee * Fee $

In total there are 4952 feasible choice tasks, so you can choose your number of candidates much larger. I chose 1000.
I subtracted the SQ alternative from the utility functions of the two alternatives, such that the SQ alternative becomes zero. The only thing I have changed is to make Cor have levels 7 and 17 (I subtracted 13). This model is identicaly to the one you specified.
Further, I specified bounds on the appearance of each level.

Once you have generated a design, you can simply generate the alternatives that you show to respondents by replacing 7 by 20, 17 by 30, and showing all the appropriate levels of the SQ alternative.

Answering your questions:
1. No, 25 is not enough, I have used 1000
2. Sp-estimates are sample size estimates. They indicate the number of design replications (of 12 choice tasks) you need to be able to estimate each of the parameters with a t-ratio of 1.96 (i.e. statistical significance). The Sp t-ratios are the t-ratios that you would obtain if you would only collect data from one design replication (12 choice tasks). If you use two blocks, the respondent sample size is double the number of design replications.
3. A negative utility is fine. You have two attributes that have a negative prior and hence negative utilities can occur. This is not a problem, logit models only care about utility differences, not about the absolute value.

[bichxuanbui]

Thank you very much, Bliemer, for your explanation. I have learned a lot from your answers in this forum.

I still have a little unclear about:
1. How to calculate the feasible choice tasks? For example in my case: you got it with 4952 feasible choice tasks, while I only know that the number of choice tasks is 96 (2*3*4*4).
2. How to decide the number of candidates can be used in mfederov algorithm suitably?
3. Sp-estimates. For instance, we have the Sp-estimates of benv(d1)=19.4062. It means that I need minimum sample size of 20 respondents to be able to estimate Env parameter with a ratio of 1.96. Is that right or wrong?

Thank you,

Xuan

[bichxuanbui]

Dear Michiel,

I run your syntax and get the results. And I need your help to explain some of them.

1. That is t-ratio of the priors:
Prior bcor benv(d0) benv(d1) bjob bfee
Fixed prior value 0.16 2.77 1.58 -0.0025 -0.02
Sp estimates 1.18486 1.275767 1.763236 11.427347 1.07581
Sp t-ratios 1.800622 1.735283 1.476049 0.579807 1.88968

Are the values of t-ratios low and therefore they are not statistical significant?

2. D-optimality = 30.110478%
Is it quite low? So the design efficiency is low?

Best regards,

Xuan

[Michiel Bliemer]

1. The total number of choice tasks is 96 * 96 = 9216, since you have two alternatives. You can see all feasible choice tasks by using ;rows = all and ;fact to generate a full factorial design, which also takes all your constraints into account (including dominancy constraints). In order to use ;fact, you have to remote ;eff and ;alg commands, see the Ngene manual.
2. The number in the mfederov algorithm should be between the number specified in ;rows and the number of feasible choice tasks (4952). The larger the number, the better. In general you do not have to specify this number, you usually use ;alg = mfederov without specifying a number. In that case, it will try to use the full factorial. You should only use a number if the full factorial is too large.
3. That is almost correct. You need a minimum of 20 design replications. Since you are blocking your design in two, I assume you are giving one respondent only half of the choice tasks, therefore you would need 40 respondents. The Sp estimates are always in terms of complete design replications.

Your next questions:
1. See my first answer, the Sp t-ratios are the t-ratios that you will obtain with a SINGLE design replication (12 choice tasks). Would you expect high t-ratios if you collect data from only 12 choice tasks (= 2 respondents if you block in 2)?
2. Why would you be interested in D-optimality? You are minimising the D-error. D-optimality is a measure that is defined by Street and Burgess and is only valid under their design assumptions (i.e. using zero priors and orthonormal coding of all coefficients). In case you are using a non-zero prior (as you do) and you are not using orthonormal coding, D-optimality is irrelevant.Ngene can optimise on D-optimality if you set ;orth = ood. Only under the very strict assumptions of Steet and Burgess it is possible to express the D-error in terms of a percentage. In general, and in your case, it is not possible to state what the minimum value of the D-error is, and therefore it cannot be expressed as a percentage. I refer to the papers of Street and Burgess to explain D-optimality and their design criteria.

[bichxuanbui]

Great!

Thank you very much, Michiel.

Best regards,

Xuan