choice-metrics.com

Hello Experts,
I have a fundamental question about D efficiency. This question reflects my ignorance and lack of knowledge.

In cases where orthogonal design exist (like the OOD designs in NGENE), the relative efficiency in % is calculated as the ratio of the determinants of the AVC or the X'X matrix or the current design and the 'optimal' orthogonal design. The manual discusses the details on page 80-81. I guess the maximum value of the optimal matrix C max can be calculated analytically.

Now we know that the ideal X'X matrix will be a diagonal matrix. So for any D efficient designs, why can't we compare the Determinant of the X'X of the design found in Ngene with an 'ideal" (but maybe not achievable) X'X optimal matrix? This way we could assess the D error of the design.

Thanks for any insights

Manoj Agarwal, Professor of Marketing, SUNY

Assuming that "the ideal X'X matrix will be a diagonal matrix" is not correct.

Street, Burgess and Louviere proposed a method in the marketing literature to construct "D-optimal designs" for unlabelled experiments. This entails using an orthogonal design for the attribute levels in the first alternative and then sequentially construct the attribute levels of the remaining alternatives such that there is minimum overlap in attribute levels across alternatives. They analytically determined the "best possible (inverse of the) D-error" and therefore could use this value as a benchmark for assessing the relative efficiency of the design where a design that attains this best possible D-error being 100% D-efficient.

Street, Burgess and Louviere needed to make three strong assumptions to allow analytical computation of the best possible D-error: (i) the attribute levels need to be orthogonal within each alternative, (ii) all attributes need to be coded using orthogonal contrasts, and (iii) all parameter priors are equal to zero.

Consequence of assumption (i):
Street, Burgess, and Louviere analytically determine the best value of the (inverse of the) D-error under these assumptions.It is important to note that the design with the best D-error is generally not orthogonal, and hence the covariance matrix associated with the best design is generally not diagonal. Orthogonality is merely a constraint on the design, removing this constraint allows finding a more efficient design. In Ngene we specifically refer to "optimal orthogonal designs" (OOD) since they are optimal under the condition of orthogonality, but are not optimal in general. The actual best possible D-error is unknown if we we do not impose orthogonality (the actual best D-error will generally be better than the value calculated by the equations in Street, Burgess, and Louviere).

Consequence of assumption (ii):
The design is only optimal if all attributes are coded using orthogonal polynomial contrasts. This is not a widely used coding scheme for choice models. Most analysts use dummy or effects coding for qualitative attributes, while for numerical attributes one often uses the levels directly in the utility function without the use of a coding scheme. The reason that Street, Burgess, and Louviere chose for this specific coding scheme is that it ensures that the covariance matrix is diagonal when using an orthogonal design, which in turn allows analytical determination of best D-error values. When using dummy or effects coding, the covariance matrix is no longer diagonal and the actual best possible D-error is unknown.

Consequence of assumption (iii):
Street, Burgess, and Louviere implcitly assume that all parameter priors needed to compute the (inverse of the) D-error are equal to zero, which means that all choice probabilities across the alternatives are equal and fixed, which allows analytical determination of the best D-error values. This best D-error only holds under the null hypothesis, the actual best D-error value is unknown in all other cases.

To conclude: We generally do not know the actual best D-error value, only under very specific conditions (which are rarely satisfied in practical applications) can we calculate this value by the equations provided by Street, Burgess, and Louviere.

Michiel

Thank you for the clarity and insightfulness of your response (as usual !!).
Manoj