choice-metrics.com

Dear Michiel,

When I run efficient design or Bayesian design, I found the D error was smaller if I increased number of rows and blocks, other things keeping same. For example I got smaller D error for the design with 12 rows and 2 blocks that for the design with 6 rows and 1 block. Does that mean that a design with more rows is better than the design with less rows? How should I compare designs just with different rows and blocks?

Thanks,

Wei

In order to compare D-errors of designs of different sizes, you can compute the D-error per block by multiplying it by the number of blocks.

As an example, compare Design 1 with D-error 0.255436 obtained via the following syntax:

Code: Select all

design
;alts = alt1, alt2
;rows = 6
;block = 1
;eff = (mnl,d)
;model:
U(alt1) = b1 * x1[1,2,3] + b2 * x2[1,2,3] + b3 * x3[1,2,3] /
U(alt2) = b1 * x1[1,2,3] + b2 * x2[1,2,3] + b3 * x3[1,2,3]
$


With Design 2 with D-error 0.125658 obtained via the following syntax:

Code: Select all

design
;alts = alt1, alt2
;rows = 12
;block = 2
;eff = (mnl,d)
;model:
U(alt1) = b1 * x1[1,2,3] + b2 * x2[1,2,3] + b3 * x3[1,2,3] /
U(alt2) = b1 * x1[1,2,3] + b2 * x2[1,2,3] + b3 * x3[1,2,3]
$


Then the D-error per block in Design 1 is 0.2554 while the D-error per block in Design 2 is 2 * 0.1256 = 0.2513. In other words, Design 2 is slightly more efficient than Design 1 per block. A larger design will generally be more efficient per block (or per choice task) than a smaller design.
As a general rule of thumb, I typically advise to use a number of rows that is at least 3 times the minimum number of rows required to satisfy the degrees of freedom. In the example above, the degrees of freedom is 3 (namely, number of parameters divided by the number of alternatives minus 1), such that the recommended number of design rows is 9 or more. Having more rows means more variation in the data, which is generally a good thing.

Michiel

Thanks so much, Michiel.

Wei