Orthogonal, Efficient, & Bayesian

[Zuduo]

Hello Everyone,

I'm designing a survey to understand traveler's transport mode choice behaviors. I did three different designs, as elaborated below.

(1) Orthogonal design
a solution was quickly found by Ngene with a warning message, stating 12 rows are impossible, so 72 rows were generated.

Code: Select all

Design
? This will generate an orthogonal fractional factorial design for the mode choice behaviour project.
;alts = car, train, bus
;rows = 12
;orth=sim
;model:
U(car) = b2 * Journey_time[30,60,90] + b3 * Petrol_price[5,10,15] /
U(bus) =  b5* Wait_time[5,10,15]+ b6*Time[1,2,3]+b7*Ticket[5, 10, 15]+ b8 * BStation_access[1,2,3] + b9 * Dest_access[1,2,3]+b10 * Environ[1,2,3]+b11 * Internet[0,1]/
U(train)=  b12 * Wait_time[5,10,15]+b13 * Time[1,2,3]+b14 * Ticket[5, 10, 15]+b15 * TStation_access[1,2,3] + b16 * Dest_access[1,2,3]+b17 * Environ[1,2,3]+b18 * Internet[0,1]$

(2) Efficient Design with Bayesian prior
I got such warning message: No valid design has been found after 1000 evaluations. There may be a problem with the specification of the design. A common problem is that the choice probabilities are too extreme (close to 1 and 0), perhaps because some or all of the prior values are too large. Also, it is generally a good idea to start with a simple design (MNL, non-Bayesian), then add complexity. If you press stop, a design will be reported, which may assist in diagnosing the problem.

I couldn't understand why some or all of the prior values are too large in my design? the priors I assigned are uniformly distributed to accommodate the high uncertainty. Do you have any suggestions?

Code: Select all

Design
? This will generate an efficient design for the mode choice behaviour project.
;alts = car, train, bus
;rows = 12
;eff=(mnl, d, mean)
;model:
U(car) = b2 * Journey_time[30,60,90] + b3[(u,-1,0)] * Petrol_price[5,10,15] /
U(bus) =  b5[(u,-1,0)]* Wait_time[5,10,15]+ b6[(u,-1,0)]*Time[1,2,3]+b7[(u,-1,0)]*Ticket[5, 10, 15]+ b8[(u,-1,0)] * BStation_access[1,2,3] + b9[(u,-1,0)] * Dest_access[1,2,3]+b10[(u,-1,0)] * Environ[1,2,3]+b11[(u,0,1)] * Internet[0,1]/
U(train)=  b12[(u,-1,0)] * Wait_time[5,10,15]+b13[(u,-1,0)] * Time[1,2,3]+b14[(u,-1,0)] * Ticket[5, 10, 15]+b15 * TStation_access[1,2,3] + b16[(u,-1,0)]  * Dest_access[1,2,3]+b17[(u,-1,0)] * Environ[1,2,3]+b18[(u,0,1)] * Internet[0,1]$

(3) a simpler design

Out of curiosity, I simplified my second design (See the codes below) by removing the priors. This time a solution quickly reached. Then, my question is, what's the difference between this design and the first design? Why in the first design 12 rows impossible, but in this design 12 rows are Okay?

Code: Select all

Design
? This will generate an efficient design for the mode choice behaviour project.
;alts = car, train, bus
;rows = 12
;eff=(mnl, d)
;model:
U(car) = b2 * Journey_time[30,60,90] + b3 * Petrol_price[5,10,15] /
U(bus) =  b5* Wait_time[5,10,15]+ b6*Time[1,2,3]+b7*Ticket[5, 10, 15]+ b8 * BStation_access[1,2,3] + b9 * Dest_access[1,2,3]+b10 * Environ[1,2,3]+b11 * Internet[0,1]/
U(train)=  b12 * Wait_time[5,10,15]+b13 * Time[1,2,3]+b14 * Ticket[5, 10, 15]+b15 * TStation_access[1,2,3] + b16  * Dest_access[1,2,3]+b17 * Environ[1,2,3]+b18 * Internet[0,1]$


Two more general questions:

(1) is there any impact on design if a constant term is included in or removed from a utility function?

(2) when using an efficient design, lots of iterations are carried out and lots of solutions are reached but Ngene is till running. So, when will the program stop? Shall I wait for Ngene to stop? Is a solution found later better than a solution found earlier?

Sorry for asking so many questions. Any help or suggestions will be greatly appreciated.

Zuduo

[Michiel Bliemer]

To answer your questions:

[*] Your second Bayesian design cannot be found because your Bayesian priors are too large (as indicated in the warning). The Bayesian priors are draws between -1 and 0, but your attribute levels are large (5 to 15), so typically your car alternative becomes very dominant, not allowing any trade-offs and therefore the model cannot be estimated. Choosing Bayesian priors between -0.1 and 0 will result in valid designs. However, you have to ask yourself if you cannot get better priors, for example by conducting a pilot study.

[*] Your simpler design uses zero priors, hence Ngene can find a design without any problem. However, this design will be efficient under the assumption that all parameters are zero (as in an orthogonal design). This is a good design if you have no information whatsoever on the value of the parameter, not even the sign. Again, obtaining better priors will significantly improve the efficiency of your design.

[*] Yes adding a constant affects the choice probabilities, and therefore the efficiency of your design.

[*] Ngene will usually never stop, the user has to stop the iterations. Ngene will try to find better designs in each iteration, and when it finds a better design, it outputs it. So the most efficient design found so far will always be the last design reported by Ngene in the output window. For MNL designs, Ngene can find these designs rather quickly, so running it for 15 minutes or so is usually sufficient, but it does not hurt running it overnight and retreiving the best design the next morning.

Michiel

[Zuduo]

Thank you so much Michiel!