PANEL EC MODEL AND PIVOT DESIGN

[cerrosimo]

Dear all,

I want to generate a pivot design assuming a panel error component model. Using the following code, I had:

1. very good d error and s estimate for the MNL model: d error=0.0113 and s estimate= 13.972
2. good d error, but very bad s estimate for the EC model: d error=0.180 and s estimate=124704.122. The s estimate was so high because of the s1 parameter in the EC model.
3. the software was not able to compute the d error and the s estimate. I had undefined estimates. the point is that I need the EC PANEL MODEL.

Code: Select all

Design
;alts(64) = alt1, alt2, alt3
;alts(66) = alt1, alt2, alt3
;alts(68) = alt1, alt2, alt3
;alts(70) = alt1, alt2, alt3
;alts(72) = alt1, alt2, alt3
;alts(74) = alt1, alt2, alt3
;alts(76) = alt1, alt2, alt3
;alts(78) = alt1, alt2, alt3
;alts(80) = alt1, alt2, alt3
;alts(82) = alt1, alt2, alt3
;alts(84) = alt1, alt2, alt3
;alts(86) = alt1, alt2, alt3
;alts(88) = alt1, alt2, alt3
;alts(90) = alt1, alt2, alt3
;alts(92) = alt1, alt2, alt3
;alts(94) = alt1, alt2, alt3
;alts(96) = alt1, alt2, alt3
;alts(98) = alt1, alt2, alt3
;rows = 12
;eff = fish(mnl,d,mean)
;rep = 500
;bdraws = halton(300)
;rdraws = halton(300)
;fisher(fish) = design1(64[0.22], 66[0.06], 68[0.14], 70[0.04], 72[0.06], 74[0.06], 76[0.05], 78[0.06], 80[0.01], 82[0.03], 84[0.05], 86[0.05], 88[0.04], 90[0.01], 92[0.01], 94[0.05], 96[0.01], 98[0.05])

;model(64):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[64]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(66):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[66]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(68):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[68]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(70):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[70]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(72):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[72]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(74):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[74]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(76):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[76]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(78):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[78]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(80):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[80]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(82):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[82]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(84):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[84]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(86):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[86]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(88):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[88]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(90):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[90]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(92):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[92]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(94):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[94]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(96):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[96]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)]

;model(98):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[98]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,0.1,0.2)] $


 


Do you have any idea why I got these very bad results?

I maybe know why. What should I put inside the parentheses in the s1 term? The beta parameter or the standard deviation that I got from my pilot study? I put the standard deviation, but I'm not sure this is right.

Thank you.

Best

Simone

[johnr]

Hi Simon

You need to place the prior parameter in the model syntax. In the EC model, the standard deviation parameter is the beta parameter, so it is not clear what the disctinction is that you are making between the two. It is definately the EC parameter that is causing the problem. We will need to look at the undefined error and return to you on that as it is not immediately clear why this is occuring from the syntax you have sent. Hopefully we will have an answer shortly.

John

[cerrosimo]

Hi John,

Using data from a pilot study, I ran an ECM in biogeme and I got the prior parameter for my error component s1. In particular I got the a beta(s1) = -1.27 with std. error 0.173.

The problem with my design was that I put as the prior parameter the st.error(s1) (=0.173) instead of the beta(s1) (=-1.27). That's way experimental design was undefined, at least I think.

Using the right prior I built the following design:

Code: Select all

Design
;alts(64) = alt1, alt2, alt3
;alts(66) = alt1, alt2, alt3
;alts(68) = alt1, alt2, alt3
;alts(70) = alt1, alt2, alt3
;alts(72) = alt1, alt2, alt3
;alts(74) = alt1, alt2, alt3
;alts(76) = alt1, alt2, alt3
;alts(78) = alt1, alt2, alt3
;alts(80) = alt1, alt2, alt3
;alts(82) = alt1, alt2, alt3
;alts(84) = alt1, alt2, alt3
;alts(86) = alt1, alt2, alt3
;alts(88) = alt1, alt2, alt3
;alts(90) = alt1, alt2, alt3
;alts(92) = alt1, alt2, alt3
;alts(94) = alt1, alt2, alt3
;alts(96) = alt1, alt2, alt3
;alts(98) = alt1, alt2, alt3
;rows = 12
;eff = fish(mnl,d,mean)
;rep = 500
;bdraws = halton(300)
;rdraws = halton(300)
;fisher(fish) = design1(64[0.22], 66[0.06], 68[0.14], 70[0.04], 72[0.06], 74[0.06], 76[0.05], 78[0.06], 80[0.01], 82[0.03], 84[0.05], 86[0.05], 88[0.04], 90[0.01], 92[0.01], 94[0.05], 96[0.01], 98[0.05])

;model(64):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[64]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(66):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[66]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(68):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[68]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(70):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[70]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(72):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[72]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(74):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[74]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(76):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[76]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(78):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[78]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(80):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[80]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(82):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[82]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(84):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[84]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(86):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[86]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(88):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[88]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(90):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[90]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(92):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[92]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(94):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[94]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(96):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[96]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)]

;model(98):
U(alt1) = b1[-2.72] + b2[(n,-0.0172,0.0031)] * A.ref[98]             + b3[(n,3.61,0.333)] * B.ref[0.5]       /
U(alt2) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] /
U(alt3) =             b2[(n,-0.0172,0.0031)] * A.piv[-80%,-60%,-40%] + b3[(n,3.61,0.333)] * B.piv[-90%,-80%,-50%,0%] + b4[(n,-0.0252,0.0035)] * C[15,30,50,80] + s1[ec,(u,-2,-1)] $

Using this design I got:

1. d error = 0.0109 and s estimate = 13.89 for the MNL
2. d error = 0.0889 and s estimate = 60.84 for the ECM
3. When I try to check the results for the PANEL ECM, I get the same values as the ECM. Is that right? I mean normal.

Finally, I have a couple of questions.

1. Do you think I'm fine with this or Should I run a specific experimental design where I explicitly consider the ECM? (Using this sintax: ;eff = fish(ec,d,mean))

2. The s-estimate for the ECM is around 60, this means that I should use at least 60 subjects. In my study, I have 18 different reference alternatives and I'm planning to interview 30 subjects for ech segment. Do you think I'm going to face big issues?

Sorry for this very long message and thank so very much for your quick reply.

Simone

[johnr]

Hi Simon

Okay, yes, you need to use the 1.27 rather than the 0.173. The 0.173 is the standard error of the parameter. The theory of experimental design is that you want to locate a design matrix which will retrive the parameter (1.27) with as smaller possible standard error (that is smaller hopefully than 0.173). This will maximise the t-ratio of the model for a given sample size.

An error component is nothing more than a random normal distribution with mean fixed at zero.What is estimated is the standard deviation parameter. Note that in estimation, the standard deviation parameter can be +/- (it is reported as a minus here) as it is a deviation from the mean which can be either side. Most software report the absolute value of the parameter, but Biogeme does not. For Ngene, you will need to use the absolute value of this in the syntax (so 1.27, not -1.27).

I'm still not convinced that the above is what is causing the problem you observed however - it might be but I'm still investigating.

In terms of generating the design, our research has shown the following.

1. you cannot compare D-errors for different designs/models, only within a design or model. hence comparing 0.0109 to 0.0889 is not really that meaningful a comparison due to the fact that they are based on completely different AVC matrices. Comparing the D-errors within a model type does make sense as you are comparing apples with apples.

2. The longer the panel length, the better MNL designs perform in terms of pannel models with random parameters (MMNL and EC designs). That is, designs optimised for MNL tend to perform quite well for panel models and vice versa, particularly as the number of rows increases. We have done hundreds of simulation runs on this and it seems to hold in most cases. Of course, the optimal MNL design will be different to the optimal panel design, but there does appear to be a strong correlation between these designs. This correlation does not hold with cross sectional random parameter designs (MMNL or EC designs). The syntax you suggest (;eff = fish(ec,d,mean)) ) assumes a cross sectional design. Our experience is that these designs perform relatively poorly for both MNL and panel models. Hence, if you truely want a pannel model, you should use ;eff = fish(ecpanel,d,mean)) instead. Note that this will take a long long time. In such cases, what we tend to do is use the MNL model specification, and test how it will perform if you were to use the EC panel model (I suspect this is what you were doing anyway). This saves time, and as I suggest above, a good MNL design tends towards a good panel design in most cases. But always check.

3. The S-error is averaged over the combined fisher matrix that is generated in the case of model averaging. In this respect, it is not that meaningful, representing an average sample size per model treatment. Ngene allows you to drill down to the model specific values (D-, S- error) however by using the tree expansion on the left hand side of design output pane. By opening these, you can see what design treatments are likely to cause issues and may require larger samples. Note that this will require time to open each one, as Ngene has to redo all the draws for that design, and your syntax is rather complex requiring lots of draws (Bayesian and Random - in your case 300 Bayesian * 300 Random parameter draws = 90,000 draws * 500 respondents = 45,000,000 draws being simulated - I hope you can see why I am suggesting using MNL design optimisation - each design evaluated requires 45 million draws).

4. Note however that the S-error is only a guide. it is based on the priors you use, and as these are likley to be off (if you knew the exact estimates, why do the study in the first place), then the S-estimate will be a guide also. See Bliemer, M.C.J. and Rose, J.M. (2011) Experimental design influences on stated choice outputs: an empirical study in air travel choice, Transportation Research Part A, 45(1), 63-79.

Hope this helps.

John

[cerrosimo]

Hi John,

this helps a lot. Thank you very much!

1.

An error component is nothing more than a random normal distribution with mean fixed at zero.What is estimated is the standard deviation parameter. Note that in estimation, the standard deviation parameter can be +/- (it is reported as a minus here) as it is a deviation from the mean which can be either side. Most software report the absolute value of the parameter, but Biogeme does not. For Ngene, you will need to use the absolute value of this in the syntax (so 1.27, not -1.27).

This makes sense. I didn't think about this issue. Thank you very much!

2.

The longer the panel length, the better MNL designs perform in terms of pannel models with random parameters (MMNL and EC designs). That is, designs optimised for MNL tend to perform quite well for panel models and vice versa, particularly as the number of rows increases. We have done hundreds of simulation runs on this and it seems to hold in most cases. Of course, the optimal MNL design will be different to the optimal panel design, but there does appear to be a strong correlation between these designs. This correlation does not hold with cross sectional random parameter designs (MMNL or EC designs). The syntax you suggest (;eff = fish(ec,d,mean)) ) assumes a cross sectional design. Our experience is that these designs perform relatively poorly for both MNL and panel models. Hence, if you truely want a pannel model, you should use ;eff = fish(ecpanel,d,mean)) instead. Note that this will take a long long time. In such cases, what we tend to do is use the MNL model specification, and test how it will perform if you were to use the EC panel model (I suspect this is what you were doing anyway). This saves time, and as I suggest above, a good MNL design tends towards a good panel design in most cases. But always check.

You were right, that's what I'm doing!

3.

The S-error is averaged over the combined fisher matrix that is generated in the case of model averaging. In this respect, it is not that meaningful, representing an average sample size per model treatment. Ngene allows you to drill down to the model specific values (D-, S- error) however by using the tree expansion on the left hand side of design output pane. By opening these, you can see what design treatments are likely to cause issues and may require larger samples. Note that this will require time to open each one, as Ngene has to redo all the draws for that design, and your syntax is rather complex requiring lots of draws (Bayesian and Random - in your case 300 Bayesian * 300 Random parameter draws = 90,000 draws * 500 respondents = 45,000,000 draws being simulated - I hope you can see why I am suggesting using MNL design optimisation - each design evaluated requires 45 million draws).

I see your point, this makes sense. I experienced that waiting time!

Thank you again for your precious help!

Simone

[cerrosimo]

Hi John,

I have other few questions about the aforesaid design.

1. I’m not really satisfied with designs produced by Ngene after 60000 evaluations. I carefully examined many alternative designs, but I always have 3 or 4 over 12 choice tasks where one alternative clearly dominates the other two. I know the design is quite complicated, but how can I handle this issue?

Could this issue arise because I need more Bayesian and Random draws? I tried to do it on my computer but at a certain point Ngene crashes.

2. If there is no solution to the aforesaid issue, may I get rid of the 4 choice tasks where one alternative clearly dominates the other two? I think I can given that this design is not orthogonal. What do you suggest? Delete or include those choice tasks? I have seen few studies using such choice tasks to control subjects’ understanding of the game.

3. As you can see I need to estimate only 5 parameters in my utility functions (considering the EC), however, in my econometric analysis I will develop these utility functions and I’ll have other 4 parameters to estimate (for example, risk aversion and discount rate parameters) (at this point I’ll have 9 parameters). In case I get rid of the aforesaid 4 choice tasks, I have 8 choice tasks left and 9 parameters to estimate. Could I have problem in estimating all parameters that I need to estimate? (Sorry, this is a very silly question!)

Hope this is clear.

Thank you

Simone

[johnr]

Hi Simon

I'm about to board a plane so this will be brief.

1. You can try and avoid dominance by placing a star after the alternative name, as in

;alts(66) = alt1*, alt2*, alt3*

I'm not sure, but how this works is explained somewhere in the manual. I don't have time to find the page reference however.

Note also that there are dominance and then there is dominance. It will depend on the specific priors you have assumed. What you may consider dominated may not be according to the priors you have assumed. You need to look at the probabilities for these choice tasks. If they are not say greater than 0.9, then you may have a problem with the priors you assumed. Note that probs of 0.3/0.3/0.3 are equally bad!

2. You are not generating an orthogonal design so this is not a concern. However, each choice task adds to the overall efficiency of the design. Deleting tasks may reduce efficiency. That said, tasks which are dominated will generally add little to efficiency by definition. You also need to ensure that you have enough degrees of freedom if you start deleting rows - your point in your third question. This is generally not adviseable.

3. This is where things get interesting. What are degrees of freedom? There are design degrees of freedom and data degrees of freedom. In generating the design,a) you assume only one respondent and b) you need to calculate the Fisher matrix and take its inverse to get the AVC matrix. If you do not have enough rows, you cannot do this matrix inversion as the resulting AVC matrix will be singular. Now, in data, you have more than one respondent, so you actually have more rows to play with. But you still need to be careful. I suggest you try simulations in this case. But try the * thingy first!

[cerrosimo]

Hi John,

thank you for the * trick! I'll try that.

Have a safe trip.

Simone