MNL design with Bayesian priors in Ngene

[rakhee]

Dear Ngene team,

I am attempting to design the choice tasks for my stated choice main study.
The pilot study was analysed using a MNL model.
The main study is using the priors generated from the pilot study analysis to design for a MNL model. I will optimise the main study design for the MNL model but will also evaluate for the panel MMNL model.

I am using a 6 normally distributed Bayesian priors in this design (i.e. X3, and the alternative-specifics constants RNP and PP respectively). The rest of the attributes are effect coded fixed priors (i.e. X1, X2, X4, X5, and X6). X3 has 4 levels and is a continuous variable.

I was hoping to get some advice please regarding my syntax for my main study design as I know that it is incorrect (I have detailed this later in this post).

I have included my syntax using an MNL model with priors for the main study below:
Design
;alts = GP, RNP, PP
;rows = 20
;block = 4
;eff = (mnl,d,mean)
;bdraws = gauss(3)
;alg = mfederov
;require:
GP.X1 = 1

;model:
U(GP) = b1.effects[-0.023] * X1[0,1] + b2.effects[0.176|0.017|0.003] * X2[0,1,2,3] + b3[(n, -0.244|0.026)] * X3[0,1,2,3] + b4.effects[-0.083|-0.029] * X4[0,1,2] + b5.effects[0.030] * X5[0,1] + b6.effects[-0.024|0.049] * X6[0,1,2] /
U(RNP) = b7[(n, -0.230|0.066)] + b1 * X1 + b2 * X2 + b3 * X3 + b4 * X4 + b5 * X5 + b6 * X6 /
U(PP) = b8[(n, -0.347|0.075)] + b1 * X1 + b2 * X2 + b3 * X3 + b4 * X4 + b5 * X5 + b6 * X6
$


Questions please:
1). I need some help with this design syntax please as I know that it does not work. I am using a 6 normally distributed Bayesian priors in this design (i.e. X3, and the alternative-specifics constants RNP and PP respectively). The rest of the attributes are effect coded fixed priors (i.e. X1, X2, X4, X5, and X6). X3 has 4 levels and is a continuous variable. I have run it and it comes up with the following error message:
Error: The 'model' property contains a prior that has dummy or effects coding without an appropriate suffix. 'b3'
Question 1: I don’t know how to fix this error. Can you please help?
Question 2: Is there anything else incorrect with the syntax that might affect the inclusion of the required types of priors?

2). Attribute X1 has 2 levels (i.e. [0,1]) which are effects coded with level 1 being the reference level. The GP alternative can only be available in location X1_1. I maintained the same syntax that I used for the pilot study design for this to happen.
Question 3: Is this aspect of the syntax correct please?

Please feel free to suggest any further improvements for this design syntax.
Thank you for your time, and I am grateful to the Ngene team for reviewing my query and providing any advice.

Thanks,
Rakhee.

[Michiel Bliemer]

I only spotted one minor error in specifying Bayesian priors, which need to be in the form (n,mean,stdev), not (n,mean|stdev). Fixing the GP location looks fine. Corrected syntax works:

Code: Select all

Design
;alts = GP, RNP, PP
;rows = 20
;block = 4
;eff = (mnl,d,mean)
;bdraws = gauss(3)
;alg = mfederov
;require:
GP.X1 = 1

;model:
U(GP) = b1.effects[-0.023] * X1[0,1] + b2.effects[0.176|0.017|0.003] * X2[0,1,2,3] + b3[(n, -0.244,0.026)] * X3[0,1,2,3] + b4.effects[-0.083|-0.029] * X4[0,1,2] + b5.effects[0.030] * X5[0,1] + b6.effects[-0.024|0.049] * X6[0,1,2] /
U(RNP) = b7[(n, -0.230,0.066)] + b1 * X1 + b2 * X2 + b3 * X3 + b4 * X4 + b5 * X5 + b6 * X6 /
U(PP) = b8[(n, -0.347,0.075)] + b1 * X1 + b2 * X2 + b3 * X3 + b4 * X4 + b5 * X5 + b6 * X6
$

Michiel

[rakhee]

Dear Michiel,

Thank you very much for reviewing my query and providing advice and a solution to my syntax. Apologies for this basic mistake by me. I have tried the design now too and it works perfectly, thanks.

I’ve just realised that I hadn’t included the syntax to enable me to evaluate for the panel mixed MNL model (i.e. rppanel model).
I’ve included my updated syntax below with your previous amendment.
I have run this updated syntax and it seems to work, but I have 2 questions please.

Updated syntax to optimise design for MNL and evaluate for panel mixed logit:
Design
;alts (mnl_model) = GP, RNP, PP
;alts (rppanel_model) = GP, RNP, PP

;rows = 20
;block = 4
;eff = mnl_model(mnl,d,mean)
;bdraws = gauss(3)
;alg = mfederov
;require:
GP.X1 = 1

;model (mnl_model):
U(GP) = b1.effects[-0.023] * X1[0,1] + b2.effects[0.176|0.017|0.003] * X2[0,1,2,3] + b3[(n, -0.244,0.026)] * X3[0,1,2,3] + b4.effects[-0.083|-0.029] * X4[0,1,2] + b5.effects[0.030] * X5[0,1] + b6.effects[-0.024|0.049] * X6[0,1,2] /
U(RNP) = b7[(n, -0.230,0.066)] + b1 * X1 + b2 * X2 + b3 * X3 + b4 * X4 + b5 * X5 + b6 * X6 /
U(PP) = b8[(n, -0.347,0.075)] + b1 * X1 + b2 * X2 + b3 * X3 + b4 * X4 + b5 * X5 + b6 * X6

;model (rppanel_model):
U(GP) = b1.effects[-0.023] * X1[0,1] + b2.effects[0.176|0.017|0.003] * X2[0,1,2,3] + b3[(n, -0.244,0.026)] * X3[0,1,2,3] + b4.effects[-0.083|-0.029] * X4[0,1,2] + b5.effects[0.030] * X5[0,1] + b6.effects[-0.024|0.049] * X6[0,1,2] /
U(RNP) = b7[(n, -0.230,0.066)] + b1 * X1 + b2 * X2 + b3 * X3 + b4 * X4 + b5 * X5 + b6 * X6 /
U(PP) = b8[(n, -0.347,0.075)] + b1 * X1 + b2 * X2 + b3 * X3 + b4 * X4 + b5 * X5 + b6 * X6
$

Questions:
1). Does the updated syntax above look correct to you?
2). When I look at the output for the above syntax, I note there is an output for the b3 Bayesian mean prior (i.e. Sp estimates, Sp t-ratios, Sb mean estimates, and Sb mean t-rations). However, there is no such output for the b3 Bayesian std deviation prior. Is there a reason for this? Does this mean that there is a mistake in my syntax?

I do apologise for adding to this query by asking more questions.
Again, thank you very much for your time and the advice.

Thanks,
Rakhee.

[Michiel Bliemer]

You did not formulate an rppanel mixed logit model, rather you formulated a Bayesian MNL model only in both models.
For a mixed logit model you need to specify random parameters as [n,mean,stdev], whereas for Bayesian you specify Bayesian priors as [(n,mean,stdev)]. If you want to have both random parameters and Bayesian priors, you get something like [n,(n,mean,stdev),(u,lower,upper)], where I am using a uniform distribution for the stdev Bayesian prior because the stdev cannot be negative (while normal distributions are not bounded by zero).

You will need to use priors obtained from a mixed logit estimation. For example, in mixed logit estimation you will get the following output:

b3.mean = 1.0, standard error = 0.3,
b.stdev = 0.4, standard error = 0.1.

Then you could for example use the following prior for b3: b3[n,(n,1,0.3),(u,0.2,0.8)].

Michiel

[rakhee]

Hi Michiel,
So I made the exact same mistake that you guys warned against in the discrete choice course in Sydney i.e. mistaking the Bayesian priors as the random parameters in the mixed logit model syntax.

In your last post, you indicated:
“If you want to have both random parameters and Bayesian priors, you get something like [n,(n,mean,stdev),(u,lower,upper)], where I am using a uniform distribution for the stdev Bayesian prior because the stdev cannot be negative (while normal distributions are not bounded by zero).You will need to use priors obtained from a mixed logit estimation. For example, in mixed logit estimation you will get the following output:
b3.mean = 1.0, standard error = 0.3,
b.stdev = 0.4, standard error = 0.1.
Then you could for example use the following prior for b3: b3[n,(n,1,0.3),(u,0.2,0.8)].”

Question 1: If it should be [n,(n,mean,stdev),(u,lower,upper)], then it should look like b3[n,(n,1,0.4),(u,0.2,0.8)], not b3[n,(n,1,0.3),(u,0.2,0.8)] as you stated? i.e. the stdev value should be 0.4 not 0.3 (which is the std error for b3.mean)?
Question 2: I’m not sure how you have worked out the upper and lower bound values for the uniformly distributed stdev Bayesian prior? It looks like you divided the b3.stdev value (i.e. 0.4) by 2 and multiplied by 2 to get [n,(n,mean,stdev),(u,0.2,0.8)], but I’m not sure.

So for the MNL model specified in the design syntax I used the priors obtained from an MNL estimation. I divided the all the estimates that I used by 2 to obtain a conservative value for the priors (including the Bayesian priors).
I did actually estimate a mixed logit model (random parameters MNL model) for the pilot data.
Question 2: Do I divide the values from the mixed logit estimation by 2, like I did with the MNL estimation, to obtain conservative values for the priors for the mixed logit model syntax?
Question 3: So I use all the outputs from the mixed logit estimation when specifying the rppanel mixed logit model in the design syntax?
Question 4: So in the design syntax, the values for the priors specified in the MNL model will be different from the values for the priors specified in the rppanel mixed logit model?

I got the following output for b3 from my mixed logit estimation:
b3.mean = -0.6, standard error = 0.09,
b.stdev = 0.6, standard error = 0.1
So I divided the b3.mean by 2 to get conservative values for the prior (i.e.-0.6 divided by 2 = -0.3).
But I’m not sure how the uniformly distributed b3.stdev prior for the lower and upper is worked out please.
Question 5: So the priors for both random parameters and Bayesian priors for my b3 is:
b3[n,(n,-0.3,0.6),(u,lower?,upper?)]

Thank you again for looking through my query and for the explanations. I am sorry that it has led to more questions from me.

Thanks,
Rakhee.

[Michiel Bliemer]

Q1: You are right, it should be 0.3.
Q2: I took twice the stdev (note that a t-ratio of 1.96 is consistent with a 95 per cent confidence interval) on either side, but that is just a choice you need to make, you can also take one stdev each side or take the stdev fixed instead of bayesian
Q2: Yes you can divide everything by 2 to get more conservative values, but this is not something that you need to do but merely something that some people (including me) often do if I have used an orthogonal design in the pilot phase (since scale is then typically larger)
Q3: Yes
Q4: Yes
Q5: b3[n(-0.3,0.045),[u,0.4,0.8])

Michiel

[rakhee]

Hi Michiel,

Thank you so much for the advice and explanations—I really appreciate the help.

I think that I'm almost there, but I’m still a bit confused about calculating the priors correctly due the use and interpretation of the term “stdev”.

I have interpreted that you worked out my priors for b3 in Q5 as follows:
From my mixed logit estimation:
b3.mean = -0.6, standard error = 0.09,
b3.stdev = 0.6, standard error = 0.1

If using both random parameters and Bayesian priors for b3, my priors will take the form of [n,(n,mean,stdev),(u,lower,upper)]
Where
n = normal distribution for b3.mean
mean = b3.mean divided by 2 = -0.6 divided by 2 = -0.3
stdev = standard error for b3.mean divided by 2 = 0.09 divided by 2 = 0.045
u = uniform distribution of b3.stdev
lower = lower bound for b3.stdev = b3.stdev minus (2 x standard error for b3.stdev) = 0.6 - (2 x 0.1) = 0.4
upper = upper bound for b3.stdev = b3.stdev plus (2 x standard error for b3.stdev) = 0.6 + (2 x 0.1) = 0.8
Hence b3[n(-0.3,0.045),[u,0.4,0.8]) as per your answer for Q5.

Question 6: Is my interpretation above correct?

Also, if my above interpretation is correct then I was wrong about the values for Q1 previously.

Thank you and I am really grateful for your time given to provide advice and explanations.

Many thanks,
Rakhee.

[Michiel Bliemer]

Yes that is correct.

Note that priors are merely GUESSES of the parameters and it is not an exact science in setting them, in other words I am simply using some rules of thumb that I have developed myself over the years. That does not mean that my "rules" are the best way or the only way of doing it. You mainly need to be using "common sense" priors based on information obtained from a pilot study or elsewhere. It is more of an art than a science

Michiel

[rakhee]

Hi Michiel,

Thank you for confirming that.

I really appreciate you sharing your knowledge with novices like myself.
It gives me a great starting point as I develop my ability working with stated choice experiments.
This great platform will hopefully enable me to develop the "art" as well as the science.
As always, that is really well summed up by you and you always put it all into perspective

Thank you very much and I appreciate all the help and advice.

Many thanks,
Rakhee.