choice-metrics.com

Hi!

I have a question concerning the use of priors in a(n) (bayesian) efficient design.

I have performed a small pilot study (n=8) and analyzed it with Nlogit. The simple mnl model returns values with std. errors etc. Then i tried to analyze it with effects coding and Nlogit reports "error 803: Hessian is not positive ..." which could mean that there are a lot of things that are 'wrong'.
However, Nlogit still provides coefficient parameters, but without std. errors.

Now my question, can I justify the use of those parameters (effects coded) as my priors in a(n) (bayesian) efficient design? With the std. deviation of 1/3 of the prior value.
And how important is the result of the D-optimality, what is an acceptable value of this?

My syntax may look like:
Design
;alts = ct, mri
;rows = 13
;con
;eff = (mnl,d,mean)
;bdraws = gauss(2,2,3,3,3,3,2,2,3,3,3,3)
;model:
U(ct) =
ct +
time1.effects[(n,0.426,0.142)|(n,-0.065,0.022)] * tct[20,25,30] +
se1.effects[(n,-0.777,0.259)|(n,-0.666,0.222)] * sect[90,95,100] +
sp1.effects[(n,1.815,0.605)|(n,0.072,0.024)] * spct[5,15,25] /
U(mri) =
Time2.effects[(n,0.2,0.067)|(n,0.207,0.069)] * tmr[60,75,90] +
Se2.effects[(n,-0.441,0.147)|(n,-0.114,0.038)] * semr[90,95,100] +
Sp2.effects[(n,1.882,0.627)|(n,-0.002,0.0006)] * spmr[5,15,25] $

Thanks for your help!!
Regards,

Gaston Vogel

Hi Gaston

I wouldn't trust the estimates you obtained from Nlogit. The error you report, as you suggest, can have a number of causes, ranging from poorly conditioned data to poorly constructed utility functions. The possible impact of the error however might be a concern. Given that the variance-covariance matrix (the inverse of the Hessian) is used in the estimation of the parameters, I would be somewhat concerned that the parameters themselves.

In terms of your direct questions, there is no right or wrong way to do this. Provided the true population parameters lie within the prior parameter distribution, you should be safe. The use of Bayesian priors represents your level of uncertainty as to the true population parameters. If you believe that they lie within the range you set, then that is the assumption you are making. Until you do the main study and have results, nobody on the planet can argue with you as it is your assumption. The flip side to the question is what if the true parameters are outside of the range? What this means is that your design will not be very efficient, and you will probably need a larger sample size to find statistically significant estimates. The trade-off however is if you make the standard deviations too wide in the first place, then you will get a less efficient design than if it is narrow as it will be optimized over a wider possible set of estimates. That is because the design is trying to be optimal over a very wide space of uncertainty. The result of this is that you will need a larger sample size than if you had narrower range but which included the true population parameters!

Your second question is what is an acceptable level of D-optimality. The answer to that question is how long is a piece of string. The D-error, or any error measure will be dependent on the number of alternatives, the number of rows, the parameter priors, the number of attributes, the number of attribute levels, the range of the attribute levels, the design type, etc. It is in effect a meaningless number other than to say that smaller is better. The smaller the number, the lower the elements in the variance covariance matrix will be on average, based on all of the above. Note that if you change any of the above, then you cannot compare the errors. So a D-error of 1 may be very good for one design, but a bad for another with different design dimensions.

John

Thank you for your answers!
Regards, Gaston