choice-metrics.com

Dear Prof. Bierlaire,

I am conducting an unlabelled choice experiment (with 3 alternatives in each choice set: Alternative 1, Alternative 2 & No choice) as part of my master's thesis and I have some questions relating to the use of priors.

I am working with SurveyEngine where Ngene is integrated and have started my pilot study yesterday.
By now, 25 respondents have answered my survey and I would like to use the data for calculating priors, which should make my choice set design for the main study more efficient.

These are my betas (calculated by using a multinomial logic model):
ASC 1 0.968
ASC 2 1.333
CO2 0.381
Calcium 0.281
Gov. green 0.718
Gov. yellow 0.422
Organic 0.568
Price -1.124
Protein 0.268
Social green 1.117
Social yellow 0.420
Sugar 0.342

I have the following question regarding the implementation of the priors:

1) Would you recommend to divide the priors by 2? On what factors does this decision depend? And would it also be possible to divide only some priors by 2 (the ones I am particularly uncertain about, in my case for example the attribute "Social green"), or is it only possible to divide (if at all) every prior by the same number?

2) Should I also use the priors ASC 1 and ASC 2? It does not make sense to me to use them since I have an unlabelled design, however I am not sure about this.
This is how I defined ASC 1 / ASC 2 / ASC No choice (in case this is relevant in this context):
ASC_A_1 = Beta('ASC_A_1', 0, None, None, 0)
ASC_A_2 = Beta('ASC_A_2', 0, None, None, 0)
ASC_No_Choice = Beta('ASC_No_Choice', 0, None, None, 1)

3) When inserting the priors in SurveyEngine x Ngene, I can choose between fixed value, uniform distribution and normal distribution.
I suppose that fixed value is not the best option since I can not be 100% certain about my priors and I want to reduce the risk of prior misspecification.
However, how do I decide between the uniform and the normal distribution?
And if I choose for example the normal distribution, how can I get the standard deviation? In my output I only have the columns value, rob. std err, rob. t-test and rob. p-value.
The default settings for setting the Bayesian parameters in SurveyEngine x Ngene are mean, sobol, 200 draws. Would you also recommend these specifications?

Thank you in advance for your help and insights.
I searched for (academic) literature on these specific questions since I need to substantiate every step in my master's thesis with literature, however I could not finde answers there.
So I would be very grateful if you could also let me know where I can find more information on these issues.

Kind regards
Laura

Hi Laura,

Note that Michel Bierlaire is the develper of Biogeme, while I (Michiel Bliemer) am one of the developers of Ngene. Our names are similar but we are different professors

1) If you used an orthogonal design or a random design in your pilot study you may want to reduce the scale of your parameters, so that would mean dividing ALL parameters by the same value. If you used an efficient design in your pilot study then reducing the scale of your parameters is not so much needed. So you could leave all parameters as they are if you think they look reasonable, or perhaps you can shrink them a little bit, for example multiplying with 0.8 instead of 0.5.

2) Since you have an opt-out alternative, you need a constant since an opt-out alternative is a labelled alternative representing the preference for selecting option1 or option2 relative to not choosing any. You can use the same ASC value for both unlabelled alternatives in the design phase, so set them both to 1.1 or so.

3) Based on parameters from a pilot study, you would generally choose normal distributions where the mean equals the parameter estimate and the standard deviation equals the standard error, so in the Ngene script it would look like (n,mean,se). In your post you only showed the parameter estimates, but you should also use the standard errors, which indicate the reliability of the priors. Since you have a lot of parameters, I sobol(1000) would be most appropriate, but since the running time in SurveyEngine is restricted 1000 draws is not really feasible as you would probably need to run the script for hours. When using such Bayesian priors it is really best to use the desktop version of Ngene. You could restrict computation time by choosing less draws, like 200, but that would still require a lot of computation time. So if you want to use SurveyEngine to generate your design I would probably opt for fixed values, which allows finding a more efficient design in the short computation time that SurveyEngine offers.

Guidance about experimental design is scattered across many different papers. The best source I can give you is a book chapter that hopefully will appear this year or next year in the second edition of the Handbook of Choice Modelling: https://bit.ly/3OKjCVS

Michiel

Hi Michiel

first of all please excuse the confusion, and thank you for clarifying who is who..
Also, thank you very much for your reply and your helpful comments.
I implemented your suggestions and now I have my design for the main data collection.

Have a good day
Laura