choice-metrics.com

Hi Michiel, John and Andrew,

I wanted to get your advice and confirm my understanding in relation to two issues in completing segmented consumer discrete choice studies.

In short – I have completed a segmented study on consumer preference to a new product. I segmented consumers based on their location, into have 3 groups.

My issues are:

1. Evaluating model performance across the 3 groups:

For the analysis, I completed three models, MNL, MNL with sociodemographics and a Mixed Logit Model.

I wanted to determine which model performed the best in relation to model fit.

In order to compare model, fit across the models, I used the adjusted rho-square to account for additional variables and then the non-nested hypothesis test proposed by Horowitz (1982) as they are not restricted versions of one another. Is this approach correct?

Obviously, I also look at the parameters and significance in each model etc?

2. Evaluating the MRS between segmented groups

Once the optimal model was selected, I then wanted to compare the MRS of attributes between segments. For example, segment A values attribute 1 higher than segment b.

I intended to estimate the point estimates and t-stats and confidence interval using the delta method.

Am I correct in my understanding that if the MRS estimate is significant and the confidence intervals between segments do not overlap, I can say there is a significant difference in between MRS measures.


Thanks you so much for all your assistance. I really appreciate the advice.

David

Dear David,

1. Yes adjusted rho-squared can be used to compared model fit between different models. Other model fit measures can be used as well (AIC, BIC, etc.)

2. Yes you can calculate the MRS as the ratio of two coefficients (which is often the willingness-to-pay when using the cost attribute) and calculate confidence intervals using the Delta method (note that for the mixed logit model the calculations are quite a bit more complicated, see our paper in Transportation Research Part B) in order to assess whether the MRS (WTP) in model 1 or 2 are statistically different.

Michiel

Thank you Michiel,

Your assistance and guidance is appreciated as always. One final point of clarification:

1. Have you used (do you recommend) using a non-nested test to compare Mixed logit and MNL models. Or is it sufficient to compare across adjusted rho-squared?

Some researchers state that the MMNL and MNL are non-nested models of each other and therefor an additional test such as the Horowitz test should be used.

Thank you again.

David

I am not able to answer that question as I am not really familiar with non-nested tests or the Horowitz test, perhaps someone else could answer this question.