Large values in var/cov matrix, is this ok?

[Tim]

Dear all,

I have created a code for an orthogonal design for a labeled experiment (about screening tests) that allows for the estimation of all two-way interactions and in which a no choice option (Notest) is present. I would like to have alternative specific parameters for all attributes and interactions, i have left out the constant in the utility equation for the combi alternative. I have two questions: (1) Is this code ok? I found high values in the MNL covariance matrix, for example: bo/bo = 38.40, b0/b4 = -13.36, b11/b15 = -13.02 is this a problem, I don't really understand the high values?

design
;alts = Stool, Blood, Combi, Notest
;rows = 36
;orth = sim
;block = 3
;model:
U(Stool) = b0 + b1 * SENS[50,70,90]
+ b2 * SPEC[5,15,25]
+ b3 * RR[10,20,30]
+ b4 * LE [1,0]
+ b5 * SENS * SPEC
+ b6 * SENS * RR
+ b7 * SENS * LE
+ b8 * SPEC * RR
+ b9 * SPEC * LE
+ b10 * RR * LE
/
U(Blood) = b11 + b12 * SENS[50,70,90]
+ b13 * SPEC[5,15,25]
+ b14 * RR[10,20,30]
+ b15 * LE [1,0]
+ b16 * SENS * SPEC
+ b17 * SENS * RR
+ b18 * SENS * LE
+ b19 * SPEC * RR
+ b20 * SPEC * LE
+ b21 * RR * LE
/
U(Combi) = b22 * SENSC[55,75,95]
+ b23 * SPECC[10,20,30]
+ b24 * RRC[15,25,35]
+ b25 * LEC [1,0]
+ b26 * SENSC * SPECC
+ b27 * SENSC * RRC
+ b28 * SENSC * LEC
+ b29 * SPECC * RRC
+ b30 * SPECC * LEC
+ b31 * RRC * LEC
$

I look forward to get an answer to these questions.

Kind regards,
Tim

[Michiel Bliemer]

I am not sure if this answers your questions, but an orthogonal design for estimating a logit model does not mean that the covariances will be zero. The design it generates is orthogonal, but since the logit model is not a linear model, the resulting covariances will not be zero. This is in general not that much of a problem. The diagonal elements of the covariance matrix should preferably be small, as the square roots of these are the standard errors, which we like to keep small. For the constants these diagonal elements are generally very large, since there is no variability in the constant (it has a an 'attribute' associated with it that always equals 1), hence getting low standard errors for the constants will always be problematic. This does not mean that your design is a bad design, it may be fine to use.

Do you have any information on the priors of the parameters? If not, then still it may be good to add the command ;eff = (mnl,d), such that the orthogonal design that you find is at least more efficient (assuming zero priors), which may help in decreasing the values in the covariance matrix.

Michiel

[Tim]

Dear Michiel,

Do you also mean that the high negative value for an off diagonal element in the MNL covariance matrix like b0/b4 = -13.36 is not a problem?

I am not sure if this is what you meant?

Thanks for your help!

Kind regards,

Tim

[Michiel Bliemer]

Yes, large values on off-diagonals are not necessarily a bad thing, at least not in terms of the D-error.

For example, if there are 2 parameters to be estimated, the covariance matrix will have elements [a b; c d], such that the D-error will be the (scaled) determinant, which is a*d-b*c. Note that b and c are the covariances, while a and d are the variances. The D-error will be low when a and d are small, but also when b and c are large! The reason is, that when two parameters have a large covariance, it is easier to estimate the parameters, as they for example move together in the same direction. So from a perspective of reliability of the parameter estimates (which is what the D-error expresses), it is not a problem to have large covariances, as long as the variances are small.