Getting higher D-error value after the pilot survey

[neeraj85]

Hi,

Following points provide a quick information on the work I am carrying out.

1. A D-efficient pivot design experiment
2. 3 alternatives: 1 Status quo + 2 hypothetical alternatives
3. 4 attributes each with 5 levels
4. 2 blocks of 10 tasks each
5. Used ;eff = fish(mnl,d) for generating the designs
6. Initial set of FIXED PRIORS used by me were Negative and close to ZERO. Eg. -0.001, etc
7. I was suggested to generate 1 block at a time due to my heavily constrained design (kindly refer to the post "How many choice tasks to keep in pivot design" for more info).
8. Thus, the two D-errors which I got are approximately 0.336

Next, I used these designs to conduct a pilot study and analysed data received from 21 respondents.

Also, I used the Mixed Logit model for the estimation of the parameters.

The estimates which I got are more negative than -0.001. Eg -1.70, -2.25, -3.09, -2.67 (Everything is expected until now)

Now, I plugged in these evaluated prior values to come up with a new design.

OBSERVATION: the D-error is stabilizing around 0.937 now (Almost three times the previous value with near zero priors)

ISSUE: I was expecting D-error value to be lower than 0.336. My understanding is that plugging in the pilot study prior values would yield comparatively more efficient design.


Kindly suggest whether my understanding is correct or not.
If yes, where I should correct the issue?

Thanks in advance

Neeraj

[Michiel Bliemer]

Please note that this is not an Ngene question and should be posted in our other forum.

D-errors can only be compared when the model that you are estimating is the same and when the prior/parameter values are the same.
In your case, you are optimising based on different priors, giving you a different D-error. This D-error can be higher or lower.

[neeraj85]

My apologies for posting the thread in a wrong forum.

It makes sense that we cannot compare the D-errors given that I used a different modelling technique.

Is there a way I can check that the newly obtained prior values will be more efficient than the previously used priors?



Also, can you clarify one more confusion please:

1. I was suggested to use ;eff = fish(mnl,d) for the purpose of the pilot study.
2. I used the resulting design for data collection
3. I estimated the model using Mixed logit framework.

--> Should I use these prior values as mean values in generating the efficient rppanel design for the full survey now?


Thanks
Neeraj

[Michiel Bliemer]

A prior cannot be efficient, a design can be efficient. A design is efficient given certain priors (and efficient designs are different under different priors, you cannot compare them). So I am not sure what you mean with more efficient priors?

We always recommend optimising the design using an MNL model, but testing the design under a panel mixed logit model. To this end you can use the model averaging capabilities of Ngene by specifying two models, something like:

;eff = mnl_model(mnl,d)
;rdraws = gauss(3)
;rep = 500
;model(mnl_model):
U(alt1) = b1[0.1] * x1[1,2,3] + ...
;model(mxl_model):
U(alt1) = b1[n,0.15,0.05] * x1[1,2,3] + ...

This will optimise for the MNL model (using the coefficients from your MNL model estimations), but when opening the design you can evaluate the design under the panel mixed logit (using coefficients from the panel mixed logit estimations). Optimising your panel mixed logit model is too time consuming, so simply evaluating it is often the best solution.

[neeraj85]

Thank you very much sir.

So, I should use the following script:

;eff = mnl_model(mnl,d)
;eff = mxl_model(rppanel,d)
;rdraws = gauss(3)
;rep = 500
;model(mnl_model):
U(alt1) = b1[0.1] * x1[1,2,3] + ... --> Beta estimates from MNL
;model(mxl_model):
U(alt1) = b1[n,0.15,0.05] * x1[1,2,3] + ... --> Beta and Std. Dev obtained from Mixed logit model


Is my understanding correct?

Can you please guide me to relevant chapter of the Ngene manual where I can read more about this technique.

Also, mine was a pivot design in which I had 6 categories.


Following is the script for the same:

Design

;alts(Cat1) = alt1*, alt2*, alt3*
;alts(Cat2) = alt1*, alt2*, alt3*
;alts(Cat3) = alt1*, alt2*, alt3*
;alts(Cat4) = alt1*, alt2*, alt3*
;alts(Cat5) = alt1*, alt2*, alt3*
;alts(Cat6) = alt1*, alt2*, alt3*

;rows = 20
;block = 2
;alg = mfederov
;eff = fish(mnl,d)
;fisher(fish) = design1(Cat1[0.191], Cat2[0.163], Cat3[0.207], Cat4[0.289], Cat5[0.134], Cat6[0.016])

;model(Cat1):
U(alt1) = b2[-0.001] * tt.ref[7.5] + b3[-0.002] * tts.ref[2] + b4[-0.003] * sn.ref[5] + b5[-0.004] * vr.ref[0.85] /
U(alt2) = b2[-0.001] * tt.piv[-20%, -10%, 0%, 10%, 20%](1-4,1-4,1-4,1-4,1-4) + b3[-0.002] * tts.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b4[-0.003] * sn.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b5[-0.004] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%](1-4,1-4,1-4,1-4,1-4) /
U(alt3) = b2[-0.001] * tt.piv[-20%, -10%, 0%, 10%, 20%](1-4,1-4,1-4,1-4,1-4) + b3[-0.002] * tts.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b4[-0.003] * sn.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b5[-0.004] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%](1-4,1-4,1-4,1-4,1-4)

;model(Cat2):
U(alt1) = b2[-0.001] * tt.ref[22.5] + b3[-0.002] * tts.ref[6] + b4[-0.003] * sn.ref[12] + b5[-0.004] * vr.ref[2.45] /
U(alt2) = b2[-0.001] * tt.piv[-20%, -10%, 0%, 10%, 20%](1-4,1-4,1-4,1-4,1-4) + b3[-0.002] * tts.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b4[-0.003] * sn.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b5[-0.004] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%](1-4,1-4,1-4,1-4,1-4) /
U(alt3) = b2[-0.001] * tt.piv[-20%, -10%, 0%, 10%, 20%](1-4,1-4,1-4,1-4,1-4) + b3[-0.002] * tts.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b4[-0.003] * sn.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b5[-0.004] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%](1-4,1-4,1-4,1-4,1-4)

;model(Cat3):
U(alt1) = b2[-0.001] * tt.ref[45] + b3[-0.002] * tts.ref[11] + b4[-0.003] * sn.ref[20] + b5[-0.004] * vr.ref[5.3] /
U(alt2) = b2[-0.001] * tt.piv[-20%, -10%, 0%, 10%, 20%](1-4,1-4,1-4,1-4,1-4) + b3[-0.002] * tts.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b4[-0.003] * sn.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b5[-0.004] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%](1-4,1-4,1-4,1-4,1-4) /
U(alt3) = b2[-0.001] * tt.piv[-20%, -10%, 0%, 10%, 20%](1-4,1-4,1-4,1-4,1-4) + b3[-0.002] * tts.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b4[-0.003] * sn.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b5[-0.004] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%](1-4,1-4,1-4,1-4,1-4)

;model(Cat4):
U(alt1) = b2[-0.001] * tt.ref[75] + b3[-0.002] * tts.ref[19] + b4[-0.003] * sn.ref[30] + b5[-0.004] * vr.ref[12.0] /
U(alt2) = b2[-0.001] * tt.piv[-20%, -10%, 0%, 10%, 20%](1-4,1-4,1-4,1-4,1-4) + b3[-0.002] * tts.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b4[-0.003] * sn.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b5[-0.004] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%](1-4,1-4,1-4,1-4,1-4) /
U(alt3) = b2[-0.001] * tt.piv[-20%, -10%, 0%, 10%, 20%](1-4,1-4,1-4,1-4,1-4) + b3[-0.002] * tts.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b4[-0.003] * sn.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b5[-0.004] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%](1-4,1-4,1-4,1-4,1-4)

;model(Cat5):
U(alt1) = b2[-0.001] * tt.ref[105] + b3[-0.002] * tts.ref[26] + b4[-0.003] * sn.ref[42] + b5[-0.004] * vr.ref[19.3] /
U(alt2) = b2[-0.001] * tt.piv[-20%, -10%, 0%, 10%, 20%](1-4,1-4,1-4,1-4,1-4) + b3[-0.002] * tts.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b4[-0.003] * sn.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b5[-0.004] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%](1-4,1-4,1-4,1-4,1-4) /
U(alt3) = b2[-0.001] * tt.piv[-20%, -10%, 0%, 10%, 20%](1-4,1-4,1-4,1-4,1-4) + b3[-0.002] * tts.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b4[-0.003] * sn.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b5[-0.004] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%](1-4,1-4,1-4,1-4,1-4)

;model(Cat6):
U(alt1) = b2[-0.001] * tt.ref[135] + b3[-0.002] * tts.ref[34] + b4[-0.003] * sn.ref[50] + b5[-0.004] * vr.ref[21.0] /
U(alt2) = b2[-0.001] * tt.piv[-20%, -10%, 0%, 10%, 20%](1-4,1-4,1-4,1-4,1-4) + b3[-0.002] * tts.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b4[-0.003] * sn.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b5[-0.004] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%](1-4,1-4,1-4,1-4,1-4) /
U(alt3) = b2[-0.001] * tt.piv[-20%, -10%, 0%, 10%, 20%](1-4,1-4,1-4,1-4,1-4) + b3[-0.002] * tts.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b4[-0.003] * sn.piv[-50%, -25%, 0%, 25%, 50%](1-4,1-4,1-4,1-4,1-4) + b5[-0.004] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%](1-4,1-4,1-4,1-4,1-4) $


I am a bit confused on how to implement the technique suggested by you in case of pivot design.


Thanks
Neeraj

[Michiel Bliemer]

You can find more on model averaging of efficient designs in Section 7.4 of the manual.

Note that you can only have 1 optimisation criterion, so you can only include one ;eff command. You include the ;eff command on the model that you would like to optimise (in your case an MNL model). Ngene then only optimises based on this criterion but when inspecting the design you can also see how it evaluates under different models (in the design window you will see on the left multiple models appearing).

I have not tried it together with pivot designs, hopefully it works. If it works, it should look something like:

;eff = fish1(mnl,d)
;fisher(fish1) = design1(Cat1[0.191], Cat2[0.163], Cat3[0.207], Cat4[0.289], Cat5[0.134], Cat6[0.016]) <---- these are your MNL models with coefficients
;fisher(fish2) = design1(Cat7[0.191], Cat8[0.163], Cat9[0.207], Cat10[0.289], Cat11[0.134], Cat12[0.016]) <---- these are your mixed logit model with coefficients

You will have to specify the mixed logit models additionally.

If it works, when you optimise and you double-click on a design, you should hopefully see fish2 appearing on the left that you can inspect as well. For rppanel models, you have to very carefully set your ;rdraws and ;rep. It may also take some time for it to calculate, as it will be doing millions of random draws.

I notice that you are using fixed priors in your MNL model. We always recommend using Bayesian priors to make the design more robust.

[neeraj85]

Thank you sir.

I tried to incorporate the suggestions received from you.

However, I am getting an runtime error stating --> Error: Only one ';fisher' property can be specified at this time.

Can you please let me know how should I resolve the issue.


Following is my script:


Design

;alts(Cat1) = alt1*, alt2*, alt3*
;alts(Cat2) = alt1*, alt2*, alt3*
;alts(Cat3) = alt1*, alt2*, alt3*
;alts(Cat4) = alt1*, alt2*, alt3*
;alts(Cat5) = alt1*, alt2*, alt3*
;alts(Cat6) = alt1*, alt2*, alt3*
;alts(Cat7) = alt1*, alt2*, alt3*
;alts(Cat8) = alt1*, alt2*, alt3*
;alts(Cat9) = alt1*, alt2*, alt3*
;alts(Cat10) = alt1*, alt2*, alt3*
;alts(Cat11) = alt1*, alt2*, alt3*
;alts(Cat12) = alt1*, alt2*, alt3*

;rows = 10
;eff = fish1(mnl,d,mean)
;rdraws = gauss(3)
;bdraws = gauss(3)
;rep = 1000
;fisher(fish1) = design1(Cat1[0.191], Cat2[0.163], Cat3[0.207], Cat4[0.289], Cat5[0.134], Cat6[0.016])
;fisher(fish2) = design1(Cat7[0.191], Cat8[0.163], Cat9[0.207], Cat10[0.289], Cat11[0.134], Cat12[0.016])

;model(Cat1):
U(alt1) = b11[0] + b2[(n,-0.142,0.0239)] * tt.ref[7.5] + b3[(n,-0.0443,0.0145)] * tts.ref[2] + b4[(n,-0.0239,0.0161)] * sn.ref[5] + b5[(n,-0.715,0.196)] * vr.ref[0.85]/
U(alt2) = b22[(n,-0.166,0.191)] + b2[(n,-0.142,0.0239)] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b3[(n,-0.0443,0.0145)] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b4[(n,-0.0239,0.0161)] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b5[(n,-0.715,0.196)] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%]/
U(alt3) = b33[(n,-0.183,0.193)] + b2[(n,-0.142,0.0239)] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b3[(n,-0.0443,0.0145)] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b4[(n,-0.0239,0.0161)] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b5[(n,-0.715,0.196)] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%]

;model(Cat2):
U(alt1) = b11[0] + b2[(n,-0.142,0.0239)] * tt.ref[22.5] + b3[(n,-0.0443,0.0145)] * tts.ref[6] + b4[(n,-0.0239,0.0161)] * sn.ref[12] + b5[(n,-0.715,0.196)] * vr.ref[2.45]/
U(alt2) = b22[(n,-0.166,0.191)] + b2[(n,-0.142,0.0239)] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b3[(n,-0.0443,0.0145)] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b4[(n,-0.0239,0.0161)] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b5[(n,-0.715,0.196)] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%]/
U(alt3) = b33[(n,-0.183,0.193)] + b2[(n,-0.142,0.0239)] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b3[(n,-0.0443,0.0145)] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b4[(n,-0.0239,0.0161)] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b5[(n,-0.715,0.196)] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%]

;model(Cat3):
U(alt1) = b11[0] + b2[(n,-0.142,0.0239)] * tt.ref[45] + b3[(n,-0.0443,0.0145)] * tts.ref[11] + b4[(n,-0.0239,0.0161)] * sn.ref[20] + b5[(n,-0.715,0.196)] * vr.ref[5.3]/
U(alt2) = b22[(n,-0.166,0.191)] + b2[(n,-0.142,0.0239)] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b3[(n,-0.0443,0.0145)] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b4[(n,-0.0239,0.0161)] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b5[(n,-0.715,0.196)] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%]/
U(alt3) = b33[(n,-0.183,0.193)] + b2[(n,-0.142,0.0239)] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b3[(n,-0.0443,0.0145)] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b4[(n,-0.0239,0.0161)] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b5[(n,-0.715,0.196)] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%]

;model(Cat4):
U(alt1) = b11[0] + b2[(n,-0.142,0.0239)] * tt.ref[75] + b3[(n,-0.0443,0.0145)] * tts.ref[19] + b4[(n,-0.0239,0.0161)] * sn.ref[30] + b5[(n,-0.715,0.196)] * vr.ref[12.0]/
U(alt2) = b22[(n,-0.166,0.191)] + b2[(n,-0.142,0.0239)] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b3[(n,-0.0443,0.0145)] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b4[(n,-0.0239,0.0161)] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b5[(n,-0.715,0.196)] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%]/
U(alt3) = b33[(n,-0.183,0.193)] + b2[(n,-0.142,0.0239)] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b3[(n,-0.0443,0.0145)] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b4[(n,-0.0239,0.0161)] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b5[(n,-0.715,0.196)] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%]

;model(Cat5):
U(alt1) = b11[0] + b2[(n,-0.142,0.0239)] * tt.ref[105] + b3[(n,-0.0443,0.0145)] * tts.ref[26] + b4[(n,-0.0239,0.0161)] * sn.ref[42] + b5[(n,-0.715,0.196)] * vr.ref[19.3]/
U(alt2) = b22[(n,-0.166,0.191)] + b2[(n,-0.142,0.0239)] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b3[(n,-0.0443,0.0145)] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b4[(n,-0.0239,0.0161)] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b5[(n,-0.715,0.196)] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%]/
U(alt3) = b33[(n,-0.183,0.193)] + b2[(n,-0.142,0.0239)] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b3[(n,-0.0443,0.0145)] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b4[(n,-0.0239,0.0161)] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b5[(n,-0.715,0.196)] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%]

;model(Cat6):
U(alt1) = b11[0] + b2[(n,-0.142,0.0239)] * tt.ref[135] + b3[(n,-0.0443,0.0145)] * tts.ref[34] + b4[(n,-0.0239,0.0161)] * sn.ref[50] + b5[(n,-0.715,0.196)] * vr.ref[21.0]/
U(alt2) = b22[(n,-0.166,0.191)] + b2[(n,-0.142,0.0239)] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b3[(n,-0.0443,0.0145)] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b4[(n,-0.0239,0.0161)] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b5[(n,-0.715,0.196)] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%]/
U(alt3) = b33[(n,-0.183,0.193)] + b2[(n,-0.142,0.0239)] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b3[(n,-0.0443,0.0145)] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b4[(n,-0.0239,0.0161)] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b5[(n,-0.715,0.196)] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%]

;model(Cat7):
U(alt1) = b90[n,-1.7020,1.2635] * tt.ref[7.5] + b91[n,-2.2562,1.2533] * tts.ref[2] + b92[n,-3.0938,2.9022] * sn.ref[5] + b93[n,-2.6741,3.0650] * vr.ref[0.85] + s1[ec,u(0.2,0.8)]/
U(alt2) = b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s1 + s2[ec,u(0.5,1.0)]/
U(alt3) = b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s2

;model(Cat8):
U(alt1) = b90[n,-1.7020,1.2635] * tt.ref[22.5] + b91[n,-2.2562,1.2533] * tts.ref[6] + b92[n,-3.0938,2.9022] * sn.ref[12] + b93[n,-2.6741,3.0650] * vr.ref[2.45] + s1[ec,u(0.2,0.8)]/
U(alt2) = b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s1 + s2[ec,u(0.5,1.0)]/
U(alt3) = b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s2

;model(Cat9):
U(alt1) = b90[n,-1.7020,1.2635] * tt.ref[45] + b91[n,-2.2562,1.2533] * tts.ref[11] + b92[n,-3.0938,2.9022] * sn.ref[20] + b93[n,-2.6741,3.0650] * vr.ref[5.3] + s1[ec,u(0.2,0.8)]/
U(alt2) = b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s1 + s2[ec,u(0.5,1.0)]/
U(alt3) = b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s2

;model(Cat10):
U(alt1) = b90[n,-1.7020,1.2635] * tt.ref[75] + b91[n,-2.2562,1.2533] * tts.ref[19] + b92[n,-3.0938,2.9022] * sn.ref[30] + b93[n,-2.6741,3.0650] * vr.ref[12.0] + s1[ec,u(0.2,0.8)]/
U(alt2) = b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s1 + s2[ec,u(0.5,1.0)]/
U(alt3) = b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s2

;model(Cat11):
U(alt1) = b90[n,-1.7020,1.2635] * tt.ref[105] + b91[n,-2.2562,1.2533] * tts.ref[26] + b92[n,-3.0938,2.9022] * sn.ref[42] + b93[n,-2.6741,3.0650] * vr.ref[19.3] + s1[ec,u(0.2,0.8)]/
U(alt2) = b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s1 + s2[ec,u(0.5,1.0)]/
U(alt3) = b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s2

;model(Cat12):
U(alt1) = b90[n,-1.7020,1.2635] * tt.ref[135] + b91[n,-2.2562,1.2533] * tts.ref[34] + b92[n,-3.0938,2.9022] * sn.ref[50] + b93[n,-2.6741,3.0650] * vr.ref[21.0] + s1[ec,u(0.2,0.8)]/
U(alt2) = b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s1 + s2[ec,u(0.5,1.0)]/
U(alt3) = b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s2 $



Neeraj

[Michiel Bliemer]

Ok this means that using the fisher command is not compatible with model averaging (as I mentioned I did not test this, our software developer can probably confirm this).

In that case, you will have to create two scripts, a first script that optimizes the design with the fisher command using the MNL model and coefficients. Once an efficient design has been found, you can save it in your project,let us call it design.ngd, you're can evaluate it in a separate script using the ;alg = eval(design.ngd) command. In this second script you use the same fisher command, but in the separate models you specify mixed logit models with corresponding coefficients. Ngene will then show you how design.ngd performs under a mixed logit specification.

Michiel

[neeraj85]

I tried the procedure suggested by you. I was able to generate an efficient MNL pivot design using the script1. However, when i tried to evaluate the design.ngd (from script1) in script2 with mixed logit command and estimates, I am getting an undefined mean Bayesian RP-Panel estimate.

I continued to troubleshoot and tried the following:
1. It worked when I changed the model to MNL, RP, and EC which means that the script isn't having errors.
2. I even tried changing the number of gaussian draws and number of replications but didn't help

Is it because of the vast nature of the design that Ngene isn't able to evaluate the performance measure?

Kindly suggest some measures to fix the issue.

Following is the script2 used by me.

;alts(Cat7) = alt1*, alt2*, alt3*
;alts(Cat8) = alt1*, alt2*, alt3*
;alts(Cat9) = alt1*, alt2*, alt3*
;alts(Cat10) = alt1*, alt2*, alt3*
;alts(Cat11) = alt1*, alt2*, alt3*
;alts(Cat12) = alt1*, alt2*, alt3*

;alg = eval(test.ngd)
;rows = 10
;eff = fish2(rpecpanel,d,mean)
;rdraws = gauss(3)
;bdraws = gauss(3)
;rep = 1000
;fisher(fish2) = design1(Cat7[0.191], Cat8[0.163], Cat9[0.207], Cat10[0.289], Cat11[0.134], Cat12[0.016])

;model(Cat7):
U(alt1) = b90[n,-1.7020,1.2635] * tt.ref[7.5] + b91[n,-2.2562,1.2533] * tts.ref[2] + b92[n,-3.0938,2.9022] * sn.ref[5] + b93[n,-2.6741,3.0650] * vr.ref[0.85] + s1[ec,0.2]/
U(alt2) = b102[(n,-0.166,0.191)] + b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s1 + s2[ec,0.5]/
U(alt3) = b103[(n,-0.150,0.161)] + b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s2

;model(Cat8):
U(alt1) = b90[n,-1.7020,1.2635] * tt.ref[22.5] + b91[n,-2.2562,1.2533] * tts.ref[6] + b92[n,-3.0938,2.9022] * sn.ref[12] + b93[n,-2.6741,3.0650] * vr.ref[2.45] + s1[ec,0.2]/
U(alt2) = b102[(n,-0.166,0.191)] + b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s1 + s2[ec,0.5]/
U(alt3) = b103[(n,-0.150,0.161)] + b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s2

;model(Cat9):
U(alt1) = b90[n,-1.7020,1.2635] * tt.ref[45] + b91[n,-2.2562,1.2533] * tts.ref[11] + b92[n,-3.0938,2.9022] * sn.ref[20] + b93[n,-2.6741,3.0650] * vr.ref[5.3] + s1[ec,0.2]/
U(alt2) = b102[(n,-0.166,0.191)] + b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s1 + s2[ec,0.5]/
U(alt3) = b103[(n,-0.150,0.161)] + b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s2

;model(Cat10):
U(alt1) = b90[n,-1.7020,1.2635] * tt.ref[75] + b91[n,-2.2562,1.2533] * tts.ref[19] + b92[n,-3.0938,2.9022] * sn.ref[30] + b93[n,-2.6741,3.0650] * vr.ref[12.0] + s1[ec,0.2]/
U(alt2) = b102[(n,-0.166,0.191)] + b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s1 + s2[ec,0.5]/
U(alt3) = b103[(n,-0.150,0.161)] + b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s2

;model(Cat11):
U(alt1) = b90[n,-1.7020,1.2635] * tt.ref[105] + b91[n,-2.2562,1.2533] * tts.ref[26] + b92[n,-3.0938,2.9022] * sn.ref[42] + b93[n,-2.6741,3.0650] * vr.ref[19.3] + s1[ec,0.2]/
U(alt2) = b102[(n,-0.166,0.191)] + b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s1 + s2[ec,0.5]/
U(alt3) = b103[(n,-0.150,0.161)] + b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s2

;model(Cat12):
U(alt1) = b90[n,-1.7020,1.2635] * tt.ref[135] + b91[n,-2.2562,1.2533] * tts.ref[34] + b92[n,-3.0938,2.9022] * sn.ref[50] + b93[n,-2.6741,3.0650] * vr.ref[21.0] + s1[ec,0.2]/
U(alt2) = b102[(n,-0.166,0.191)] + b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s1 + s2[ec,0.5]/
U(alt3) = b103[(n,-0.150,0.161)] + b90[n,-1.7020,1.2635] * tt.piv[-20%, -10%, 0%, 10%, 20%] + b91[n,-2.2562,1.2533] * tts.piv[-50%, -25%, 0%, 25%, 50%] + b92[n,-3.0938,2.9022] * sn.piv[-50%, -25%, 0%, 25%, 50%] + b93[n,-2.6741,3.0650] * vr.piv[-25%, -12.5%, 0%, 12.5%, 25%] + s2 $



Neeraj

[Michiel Bliemer]

You should carefully look at your priors, because clearly they cannot be correct; they are too large and the logit model will produce undefined results.

For example:

b90[n,-1.7020,1.2635] * tt.ref[135]

You are multiplying the attribute level 135 with a prior that is around -1.7020. This is a huge value and the logit model will return a zero (i.e. exp(-1.7*135) = 0). Zero divided by zero is not defined. Further, a standard deviation of 1.235 is also very large. It is not possible that -1.7020 and 1.2635 come out of an estimation using data with attribute levels that are in the order of 135.