choice-metrics.com

Hi there,
I used the code below designing choice sets for a pilot study. Now I want to design the final choice sets and have the following questions:
-> Based on the results from the pilot (80 respondents, each time 12 choice sets) I estimated a conditional logit. However, only some of the parameters are statistically significant. Do I only use the significant ones as priors, i.e. keeping the univariate priors for the other ones?
-> When I use a fixed prior for the attribute price based on the CL estimates instead of the univariate prior as it is shown in the code the "WTP(waqu) estimate" increases strongly. I expected that a fixed prior would decrease that figure and not increase it?
-> Also the "S estimate" as well as the "WTP(waqu) estimate" increase strongly when I use, for example, for b3 a prior following a normal distribution instead of the univariate prior - the mean value of the normal distribution is within the range of the interval specified for the univariate prior. I thought that a "normal prior" would give more information than a "univariate prior" but probably my thoughts are on the wrong track?

Thanks a lot
Jürgen

Code: Select all

design
;alts = alt1*, alt2*, SQ
;rows = 24
;block = 2
;bdraws = halton(500)
;eff = (mnl,wtp(waqu))
;wtp = waqu(b3,b4,b5,b7,b8/b9)
;model:
U(alt1) = b3[(u,0.1,0.7)]* WQuh[1,2,3,0] +
          b4[(u,0.1,0.7)]* WQob[1,2,3,0] +
          b5[(u,0.1,0.7)]* WQss[1,0] +
          b7[(u,0.1,0.7)]* WQsb[1,2,0] +
          b8[(u,0.1,0.7)]* WQda[1,2,3,0] +
          b9[(u,-0.7,-0.1)] * price[10,25,50,75,100,150]  /
U(alt2) = b3 * WQuh +
          b4 * WQob +
          b5 * WQss +
          b7 * WQsb +
          b8 * WQda +
          b9 * price                                       /
U(SQ)   = b1[-1.5]$


Hi Jürgen

I will try and answer your questions in turn

Q. Based on the results from the pilot (80 respondents, each time 12 choice sets) I estimated a conditional logit. However, only some of the parameters are statistically significant. Do I only use the significant ones as priors, i.e. keeping the univariate priors for the other ones?

A. there is no right or wrong way to do this. Michiel Bliemer typically uses the estimated standard errors from the pilot study for all parameter priors and the estimated parameters only of statistically signficant parameter estimates as the means; else if they are not statistically signifcant, he sets the mean of the prior parameter distribution to zero. We used this approach in the following paper, and it worked reasonably well. My personal preference is to use even insigificant parameters as priors, but only if they are of the expected sign. If they are not, or you don't know the sign, then I prefer to set them to zero.

Bliemer, M.C.J. and Rose, J.M. (2011) Experimental design influences on stated choice outputs: an empirical study in air travel choice, Transportation Research Part A, 45(1), 63-79.

Q. When I use a fixed prior for the attribute price based on the CL estimates instead of the univariate prior as it is shown in the code the "WTP(waqu) estimate" increases strongly. I expected that a fixed prior would decrease that figure and not increase it?

A. There are significant issues with this design and the priors that are being used which has to do with scaling. If you look at the average contribution to utility of your price parameter at the extreme of your Bayesian prior parameter distribution (i.e., -0.7) and you combine this with an average price attribute level of 68.3, the average contribution to utility is -47.83. If you compare this to the remaining attributes with say an average level of 2 and a prior of say 0.7 the contrinution to utility is on average around 1.4. As you can see price us dominating the model (you would get a great model just with price leaving everything else out) and hence these other attributes really having only a marginal impact upon utility. The design is therefore always going to struggle to find good attribute level combinations as only price really matters. Further, given that the price parameter is negative and has on average a contribution to utility of around -47.83, is say the remaining attributes contribute + 15 to overall utility, the utility for these alternatives will still be on average around -42.83. If you compare this to the SQ with a utility of -1.5, the probability that a none SQ alternative will ever get chosen is going to be close to zero if not zero. This again will make finding a design very difficult.

The reason the Bayesian and fixed designs are acting in the way you describe is that the Bayesian design can still draw priors close to -0.1 which is a much more reasonable scale (magnitude) for the size of the X you are using (but even then still probably to large - recall that the beta will be in the same magnitude as the X). In the fixed design, you are probably taking the midpoint between -0.1 and -0.7 as your prior which will be a worse than a parameter of -0.1 in terms of its scale.

You need to rescale your prior parameter for price. When I used the values... b9[(u,-0.07,-0.01)] * price[10,25,50,75,100,150] I got much more reasonable results.

Q. Also the "S estimate" as well as the "WTP(waqu) estimate" increase strongly when I use, for example, for b3 a prior following a normal distribution instead of the univariate prior - the mean value of the normal distribution is within the range of the interval specified for the univariate prior. I thought that a "normal prior" would give more information than a "univariate prior" but probably my thoughts are on the wrong track?

A. This could be caused by the previously identified problem. It is our advice to always check the probabilites and utilities of any resulting design so that you can detect these types of issues.

John

Hi John,

Many thanks for your response - it is very helpful. As I guess many people are a bit struggling with the use of priors I am sure that a paragraph or two about your experience with priors in the next version of the NGENE manual would be appreciated by many readers.

Best from Berlin,

Jürgen

Thanks Jürgen

We are working on a number of new initiatives with some really cool additions. One thing we have floated is a frequently asked question section. Trust me when I say that the issue with priors will be # 1 on this list.

Regards from Sydney

John