Utilty balance in optimal MNL design evaluated at RP PANEL

[maria.trentinaglia]

Dear Professors,

I am new to Ngene and I am looking for your precious advice.

Before writing this post I have extensively read the already existing ones, but I think I need some help.

Let me start from the goal of the research: we want to perform a choice experiment with a latent class model, though we do not know ex ante how many classes there will be.

To this purpose, we have developed a pilot (MNL efficient design with 0 priors) to obtain parameter priors for the final design.

The final design was initially supposed to be a random parameter model until I came across one or two posts recommending to “optimise for MNL and evaluate only for RPPANEL”. I followed your advice and adapted the code to our research as follows.
All the parameters and bayesian priors are assumed to be uniformly distributed over the interval (mean - standard error; mean + standard error).
Design
;alts(model_mnl)=alt1*, alt2*, nobuy
;alts(model_rp)=alt1*, alt2*, nobuy
;eff=model_mnl(mnl, d, mean)
;alg=mfederov
;rdraws=gauss(3)
;bdraws=gauss(3)
;rows=36
;block=6
;rep=100
;model(model_mnl):
U(alt1)=b1[(u,-0.01796,-0.00978)]*W[70,80,100]+b2[(u,-0.02668,-0.01729)]*F[70,80,100]
+b3[(u,-0.77386,-0.50034)]*P[1,1.25,1.5,1.8]+b4[(u,0.401515,0.559255)|0].effects*O[1,2,3] /
U(alt2)=b1*W+b2*F+b3*P+b4*O/
U(nobuy)=asc[(u,-5.39863,-4.17598)]
;model(model_rp):
U(alt1)=b1[u,-0.11652,-0.07025]*W[70,80,100]+b2[u,-0.18457,-0.10779]*F[70,80,100]
+b3[u,-7.38243,-4.61457]*P[1,1.25,1.5,1.8]+b4[u,2.817365,4.291815|0].effects*O[1,2,3] /
U(alt2)=b1*W+b2*F+b3*P+b4*O/
U(nobuy)=asc[u,-27.8758,-20.5407]
$

The model runs smoothly, though I have one major concern related to the utility balance. In the MNL, it is 68%, whereas it is as low as 9% in the RPPANEL. I realize that this is due to the extremely high value for the constant of the no buy option, but I do not know how to improve this, as prior values have already been divided by 2.
Also, the D error for the evaluated RPPANEL is "Undefined".

Are the design set up and syntax correct? How could I improve them?

Thank you in advance for your suggestions,

Best regards,

Maria

[Michiel Bliemer]

The Bayesian efficient design for the MNL model looks fine. I assume that you estimated an MNL model and obtained parameters and standard errors to create Bayesian priors. Why not use normally distributed Bayesian priors with a standard deviation equal to the standard error?

The priors for the random parameter model do not seem appropriate, all of them are far too large. How did you obtain them? Did you estimate a panel mixed logit model? And why did you use a uniform distribution for the parameters in the mixed logit model? Note that random parameters are not the same thing as Bayesian priors.

Michiel

[maria.trentinaglia]

Thanks for your prompt feedback!

Yes, indeed, I estimated an MNL to obtain the information for the Bayesian priors. I initially used uniform distributions to have a more conservative approach, but I will re-run it using normally distributed Bayesian priors.

As for the random part... The random parameters were obtained estimated the pilot model as a RPPANEL regression in NLOGIT. The betas and ranges were then used to retrieve the random parameter values I have reported in model_rp. I am kind of struggling with the mixed logit estimation of this pilot. I am trying both with NLOGIT (obtaining these huge coefficients) and STATA (where the model fails to converge). I was wondering if the limited number of observations (I have approximately 30 respondents, each facing 8 choice tasks) may undermine the feasibility of a random parameter model.

Thanks again,
Best regards,
Maria

[Michiel Bliemer]

You should only use the betas for the rppanel model and not the standard errors since you are not using Bayesian priors in the rppanel model in Ngene, so just ignore the ranges and only take the betas.

For example, for the mixed logit you will get something like:

betas s.e.

b1_mean 0.6 0.3
b1_sigma 0.1 0.05

Then you should use for the rppanel model with fixed priors the following:

b1[n,0.6,0.1]

If you want to use an rppanel model with Bayesian priors, then you do something like:

b1[n,(n,0.6,0.3),(u,0.05,0.15)]

where I am using a uniform distribution for the normally distributed sigma since this cannot be a negative value (so the normal distribution would not be appropriate) and I assume a single standard error range from the mean.

If the model fails to converge or you get this kind of very large values, then the model parameters are not identifiable, typically a sign of insufficient data or insufficient variation in the data. So in that case just forget about the rppanel model and just optimise for the MNL model.

Michiel

[maria.trentinaglia]

Thank you very much indeed!
Best regards,
Maria Teresa