Pivot design questions and issues - Ngene

[Elnaz]

Hi Ngene team,

Thanks for your great support and helpful manual and articles. I am trying to create an efficient design to evaluate WTP for electric vehicles in Canada. Before doing the main SP survey, we will do a pretest where we can get some ideas about parameter priors. I am thinking of doing a pivot design in which I would have three alternatives (status quo, hybrid vehicle, and pure electric vehicle). I will have six vehicle attributes, all having four levels. Regarding such experimental design on Ngene, here are my questions (please correct me if I’m wrong or missed any points):

1) Do I need to include alternative specific constant in the utility equation syntax?

2) Is that OK if some of the attributes are pivoted and some not ( e.g. attribute X1 -on the syntax below in question 6- is pivoted while X2 is not )

3) I also need to include sociodemographic characteristics in alternative utilities. Do I need to know and include all demographics in advance? (There might be more than eight of them that I want to estimate)

4) For the pretest, I know the signs of the parameters from the previous literature and I am thinking of doing Bayesian design with three segments (similar to the example in manual p.166). For the segments though, I’m not very clear about the fixed parameter in [ ] for reference alternative -- I wonder if the fixed parameter I choose for each attribute should be relevant to the one of another attributes in each segment or they are independent from one another?
for instance, if the fixed parameter for vehicle price is specified a small number, the parameter I consider for other attributes, let’s say pollution level, should be relevant to the specified price?

5) Is that OK if I optimize my design with MNL model and at the end, estimate my model with MMNL model with actual data?

6) Here is my syntax (I didn't consider the three segments this time):

Design
;alts = alt1, alt2, alt3
;rows = 48
;block = 6
;eff = (mnl,d)
;model:
U(alt1) = b1[(u,-1,0)] * X1.ref[20] + b3[(u,-0.5,0)] * X3.ref[12] + b4[(u,-0.3,0)] * X4.ref[8] + b6[(u,-0.05,0)] * X6.ref[4] /
U(alt2) = b1 * X1.piv[-25%,0%,25%,50%] + b2 * X2[0,1,2,3] + b3 * X3.piv[-80%,-60%,-40%,-20%] +
b4 * X4.piv[-15%,-5%,5%,15%] + b5 * X5[0,1,2,3] + b6 * X6.piv[-90%,-70%,-50%,-30%] +
b7 * X7[0,1,2,3] /
U(alt3) = b1 * X1.piv[-25%,0%,25%,50%] + b2 * X2[0,1,2,3] + b3 * X3.piv[-80%,-60%,-40%,-20%] +
b4 * X4.piv[-15%,-5%,5%,15%] + b5 * X5[0,1,2,3] + b6 * X6.piv[-90%,-70%,-50%,-30%] +
b8 * X8[0,1,2,3] $

I ran the above syntax and scanned all 48 scenarios I got. I noticed that Ngene keeps giving me alternatives with the same level for many scenarios. I even tried a very simple design with far less number of attributes and levels, but still got the same issue -- below is an example of one of such scenarios, you can see that x1, x3, x4 and x2 all have the same level for alt2 and alt3:

alt1 alt2 alt3
x1 20 30 30
x3 12 5 5
x4 8 8 8
x6 4 1 2
x2 2 2
x5 1 2
x7 1
x8 1
Choice question:


Many thanks and I already apologize for my lengthy questions!

[Michiel Bliemer]

1) No you do not have to, only if it makes sense to add them (typically in labelled experiments)

2) Yes

3) You can specify them using the .covar suffix (see the manual), but note that in unlabelled experiments you can only add them as interaction terms. It is not common to put them into the utility functions in the design stage (unless you think they are hugely important) so I suggest leaving them out.

4) The priors have to make sense from a behavioural point of view. Often, the ratios of the parameters indicate certain behaviour (e.g., willingness-to-pay or some other trade-off), while the absolute values are reflecting the variance of the error term. So first make sure that the ratios of the parameters make behavioural sense by looking at the trade-offs between attributes. Then the scale parameter in the logit model multiplies essentially all coefficients, so they should then be scaled all up or down, depending on how much error variance you consider.

5) An MNL optimised design is relatively efficient also for estimating panel MMNL models, so that should be fine. You can always test the design how it would perform under MMNL conditions.

6) Overlapping levels is not necessarily a problem, and in your case it is the result of the specification of your priors. For a level of x1 = 20 and x3 = 12 for example you have extremely high priors (in the most extreme case b1 = -1 and b3 = -0.5, respectively). These seem unrealistically high. You should ensure that your priors make sense, as now trade-offs cannot be made without creating dominancy in your choice tasks. Setting lower priors will enable more trade-offs between attributes and therefore less overlap.

[maria]

Dear choicemetrics team,
I would like to do a pivot desig, but I have some doubts regarding the number of rows. I could not find the answer in the manual or in the forum yet. I hope you can help me to understand it. My design is:

;alts = Ref, alt1,alt2,alt3
;rows = 36
;eff=(mnl,d)
;con
;model:

U(Ref) =
b3[-0.5]*A.ref[25]
+ b4[-0.5]*B.ref[100]
+ b5[-0.5]*C.ref[100]
+ b6[-0.5]*E.ref[100]
+ b7[-0.5]*F.ref[100]/

U(alt1) = b1[0.6]
+ b3*A.piv[-10,0]
+ b4*B.piv[-50,0]
+ b5*C.piv[-50,0]
+ b6*E.ref[100]
+ b7*F.ref[100]/

U(alt2) = b2[0.6]
+ b3*A.piv[-25,-10,0]
+ b4*B.piv[-100,-50,0]
+ b5*C.piv[-100,-50,0]
+ b6*E.piv[-100,-50,0]
+ b7*F.piv[-100%,-50%,0]
+ b8[0.3]*D.dummy[0,1]
$

The minimum number of rows should satisfy both: S=8/(4-1) and be divisible by 2 and 3. Therefore, the minimum number should be 12. However, only if I set the priors to zero I get a result for 12 rows. If I set the priors to values different to zero I need at least 36 rows to get any result. Also the D-error is very high, so maybe I have an error in the model. Do you know why the prior values modify the minimum number of choice situations?

Thank you very much for your help!

[Michiel Bliemer]

I have not tested yet, but my suspicion is that your priors are misspecified. This is the most common mistake that people make with efficient designs. If you have an attribute level of 100, you clearly cannot have a prior equal to 0.5 (since 0.5*100 = 50 is an extreme utility value, and hence Ngene will likely not be able to find any design that it can evaluate). Using a larger number of rows makes the likelihood of having at least a few choice tasks that make sense larger, but of course still is not a good design. So please use priors that you obtained from a pilot study, or significantly decrease your priors.

Note that a utility contribution of 2 or 3 is already very large. For example, if you compare an alternative with a utility of 0 with another alternative with utility 3, then the probability of choosing the first alternative is exp(0)/(exp(0)+exp(3)) = 0.047. So clearly this makes alternative 2 quite dominant, not capturing much information. Now try computing the probability when the difference in utility is equal to 50...

[maria]

Thank you very much for your fast reply. I did not thing on the values before...now I understand it! I have tried with small priors and now 12 rows is ok.

Thanks a lot!