choice-metrics.com

Hello,

I am trying to elaborate a design considering covariates (different treatments) that should enter the model as interaction terms.

I followed the instructions in section 8.4 of the Manual, but they make reference to covariates that enter the model as main effects.

I tried to figure out how the code should be in my case and I introduced the covariates in the utility function as follows:

U(alt1) = b1[-.14]*PRICE[11,13,15,17] + b2[2.17]*AOP[0,1] + b3[1.87]*BIO[0,1] + b4[-.42]*AOP[0,1]*BIO[0,1] + b5[.5]*AOP[0,1]*Ta.covar[1] + b6[.5]*BIO[0,1]*Tb.covar[0]/

See here for the full code:
https://www.dropbox.com/s/ssunmfflqz4vt ... t.ngs?dl=0

As a result, I got this message:

Warning: No valid design has been found after 1000 evaluations. There may be a problem with the specification of the design. A common problem is that the choice probabilities are too extreme (close to 1 and 0), perhaps because some or all of the prior values are too large. Also, it is generally a good idea to start with a simple design (MNL, non-Bayesian), then add complexity. If you press stop, a design will be reported, which may assist in diagnosing the problem.

I am not sure whether I did something wrong with the code above or the software simply cannot find a suitable design.

Any suggestion?

The output file is here:
https://www.dropbox.com/s/joee8zgcto7n5 ... t.ngd?dl=0

Thanks a lot.
Maurizio

I had a quick look and the syntax works when you only include one covariate, e.g. only b5*AOP*Ta.covar, but not when you add another one, b6*BIO*Tb.covar.

The Ngene warning typically means that the model is not identifiable, i.e. it is not possible to estimate this model. If this is the case, then no design will ever be able to estimate this model. But I cannot rule out there is something else going on, so I may need to take a closer look when I have time. You can check whether a model is identifiable by simulating a sample and see if you can estimate a model based on this data. If the estimation software cannot estimate the model, then the parameters are likely not identifiable and you have to formulate another utility function.
The issue may be that AOP, BIO, Ta and Tb are essentially all dummies and interactions between all of them may be problematic since there will be a lot of zeros and therefore it perhaps not possible to disentangle the parameters, but I am not completely sure about this.

Michiel

Hello Michiel,

thanks for your answer.

I am using R and the packages mlogit and gmnl to analyse the data.

I made a test with 10 respondents and I analysed their data using the following code:

my_mnl_t <- mlogit(choice ~ price + aop + bio + aop:bio + aop:treat + bio:treat + aop:bio:treat
, data = my_data
, reflevel=4)

Despite I have only 10 respondents the analysis works (of course I got a lot of non-significant parameters).

This is the model estimation output (using mlogit):

Call:
mlogit(formula = choice ~ price + aop + bio + aop:bio + aop:treat +
bio:treat + aop:bio:treat, data = my_data, reflevel = 4,
method = "nr", print.level = 0)

Frequencies of alternatives:
4 1 2 3
0.5250 0.1250 0.1875 0.1625

nr method
17 iterations, 0h:0m:0s
g'(-H)^-1g = 5.65E-07
gradient close to zero

Coefficients :
Estimate Std. Error t-value Pr(>|t|)
1:(intercept) -1.6383e+00 1.4203e+00 -1.1535 0.24873
2:(intercept) -1.3910e+00 1.3789e+00 -1.0088 0.31308
3:(intercept) -1.5553e+00 1.3753e+00 -1.1308 0.25813
price -1.4769e-01 7.1217e-02 -2.0737 0.03810 *
aop 2.7823e+00 1.2166e+00 2.2868 0.02220 *
bio 2.3086e+00 1.2821e+00 1.8006 0.07177 .
aop:bio -1.8556e+00 1.5660e+00 -1.1849 0.23606
aop:treatT1 -5.7908e-01 1.0490e+00 -0.5520 0.58093
aop:treatT2 -1.9369e+00 1.2441e+00 -1.5569 0.11950
aop:treatTx 1.0864e-03 9.7762e-01 0.0011 0.99911
bio:treatT1 8.1876e-01 9.8101e-01 0.8346 0.40394
bio:treatT2 -1.7520e+01 2.9977e+03 -0.0058 0.99534
bio:treatTx -1.6882e+01 3.2155e+03 -0.0053 0.99581
aop:bio:treatT1 8.8079e-01 1.5751e+00 0.5592 0.57603
aop:bio:treatT2 1.9218e+01 2.9977e+03 0.0064 0.99488
aop:bio:treatTx 1.7624e+01 3.2155e+03 0.0055 0.99563
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Log-Likelihood: -70.747
McFadden R^2: 0.26755
Likelihood ratio test : chisq = 51.685 (p.value = 1.5282e-06)

If estimation works, then I assume that the model is correctly specified and that the parameters are identifiable.

I looked at the Fisher matrix that it produces:

Code: Select all

MNL fisher matrix (treat)                     
Prior   b1          b2         b3          b4         b5         b6   b0
b1      63.407318   0.124298   2.813836   -0.079616   0.062149   0   -2.301404
b2      0.124298    0.693066  -0.096307    0.289737   0.346533   0   -0.082305
b3      2.813836   -0.096307   0.549385    0.166843  -0.048153   0   -0.064147
b4     -0.079616    0.289737   0.166843    0.43638    0.144869   0   -0.012544
b5      0.062149    0.346533  -0.048153    0.144869   0.346533   0   -0.041153
b6      0           0          0           0          0          0    0
b0     -2.301404   -0.082305  -0.064147   -0.012544  -0.041153   0    0.157676


As you can see, there is no information on b6, it has all zeros. I am not sure why that is, especially since the it does not happen to b5 that is specified in an identical way. Optimising designs with covariates is not often done by our users, and while we have tested this functionality thoroughly in the past, we cannot rule out that you have stumbled on a bug in Ngene. I have asked a colleague to look into it and we hope to get back to you as soon as possible.

Michiel

Thanks Michiel,

since I am not that familiar with Ngene syntax I not completely sure I did it right.

However, I appreciate your interest in going more in depth about this issue.

Best regards.

Maurizio

We have had a closer look and we are still not entirely sure why it does not work, but we suspect that there may be an issue with the order in which Ngene interprets the covariates.

I think that there is no problem in your syntax specification, so I am afraid I do not have a solution for you at the moment. We will continue looking into it and if there is indeed an issue in Ngene that we hope to fix it in the next release.

Michiel