2 pilot, with insignificant prior

[Tiziana]

Dear All,
I wrote , because I tried to use efficient disign.

[Tiziana]

Dear all, I wrote because I tried to use efficient design. In my first pilot, the results was quite bad. Before I used a choice set of 12 cards following this cose:
Design
;alts = alt1, alt2
;rows = 12
;orth = sim
;model:
U(alt1) = b1 * PRICE[0,1,2,3] + b2 * FAT [0,1] + b3 * SALT [0,1] + b4 * FAT * SALT/
U(alt2) = b1 * PRICE + b2 * FAT + b3 * SALT + b4 * FAT * SALT

|Variable | Coefficient | Standard Error |b/St.Er.|P[|Z|>z] |
+---------+--------------+----------------+--------+---------+
BFAT .48757863 .20565668 2.371 .0177
BSALT -.34783466 .18484187 -1.882 .0599
BFATSALT .08598072 .28788224 .299 .7652
BPRICE -1.83177422 .15410516 -11.887 .0000
NOBUY -3.53599794 .23964498 -14.755 .0000
I estimated using Nlogit . Unfortunately the variable bfatsalt is not statistically significat at 5%. Therefore , my first question is if it is correct to use (u, -1, 0) for bfatsalt coefficient in the efficient design, even if the sign b4 in the estimation is positive?.

Design
;alts = alt1, alt2, alt3
;rows = 12
;eff = (mnl,d,mean)
;bdraws=gauss(3)
;model:
U(alt1) = b1[(n,-1.831,0.154)] * PRICE[0.70, 1.15, 1.60, 2.05] + b2[(n,0.484,0.205)]*FAT[0,1] + b3[(n,-0.347,0.184)]*FAT[0,1] + b4 [(u,-1,0)]* FAT * SALT/
U(alt2) = b1 * PRICE + b2 *FAT + b3 *SALT + b4 * FAT * SALT/
U(alt3) = b0[(n,-3.535,0.239)]
$
The, I had these results,

MNL efficiency measures

Fixed Bayesian mean
D error 1,053979 1,077247
A error 1,666725 1,701166
B estimate 54,132969 0,522165
S estimate 44,7978 1375,259847

Prior b1 b2 b3 b4
Fixed prior value -1,831 0,484 -0,347 -0,5
Sp estimates 1,217247 22,587327 44,7978 43,381575
Sp t-ratios 1,776506 0,412405 0,292838 0,29758
Sb mean estimates 1,286969 69,487364 1154,008298 264,855677
Sb mean t-ratios 1,752118 0,408089 0,289987 0,294288

I see that S estimate is very high 1375,25, thus I suppose that this design is quite bad..right?


Today I did a second pilot study. the results are in constrast with the previous pilot, sing the bsalt coefficient is positive while the bfat is not significant. Hence, at this point, I do not know what I should do ..
Could you suggest me somenthing else?
Thanks in advance
Tiziana

[Tiziana]

Dear all, I wrote because I tried to use efficient design. In my first pilot, the results were quite bad. Before I used a choice set of 12 cards following this code:
Design
;alts = alt1, alt2
;rows = 12
;orth = sim
;model:
U(alt1) = b1 * PRICE[0,1,2,3] + b2 * FAT [0,1] + b3 * SALT [0,1] + b4 * FAT * SALT/
U(alt2) = b1 * PRICE + b2 * FAT + b3 * SALT + b4 * FAT * SALT

|Variable | Coefficient | Standard Error |b/St.Er.|P[|Z|>z] |
+---------+--------------+----------------+--------+---------+
BFAT .48757863 .20565668 2.371 .0177
BSALT -.34783466 .18484187 -1.882 .0599
BFATSALT .08598072 .28788224 .299 .7652
BPRICE -1.83177422 .15410516 -11.887 .0000
NOBUY -3.53599794 .23964498 -14.755 .0000
I estimated using Nlogit . Unfortunately the variable bfatsalt is not statistically significat at 5%. Therefore , my first question is if it is correct to use (u, -1, 0) for bfatsalt coefficient in the efficient design, even if the sign b4 in the estimation is positive?.

Design
;alts = alt1, alt2, alt3
;rows = 12
;eff = (mnl,d,mean)
;bdraws=gauss(3)
;model:
U(alt1) = b1[(n,-1.831,0.154)] * PRICE[0.70, 1.15, 1.60, 2.05] + b2[(n,0.484,0.205)]*FAT[0,1] + b3[(n,-0.347,0.184)]*FAT[0,1] + b4 [(u,-1,0)]* FAT * SALT/
U(alt2) = b1 * PRICE + b2 *FAT + b3 *SALT + b4 * FAT * SALT/
U(alt3) = b0[(n,-3.535,0.239)]
$
The, I had these results,

MNL efficiency measures

Fixed Bayesian mean
D error 1,053979 1,077247
A error 1,666725 1,701166
B estimate 54,132969 0,522165
S estimate 44,7978 1375,259847

Prior b1 b2 b3 b4
Fixed prior value -1,831 0,484 -0,347 -0,5
Sp estimates 1,217247 22,587327 44,7978 43,381575
Sp t-ratios 1,776506 0,412405 0,292838 0,29758
Sb mean estimates 1,286969 69,487364 1154,008298 264,855677
Sb mean t-ratios 1,752118 0,408089 0,289987 0,294288

I see that S estimate is very high 1375,25, thus I suppose that this design is quite bad..right?


Today I did a second pilot study. the results are in constrast with the previous pilot, sing the bsalt coefficient is positive while the bfat is not significant. Hence, at this point, I do not know what I should do ..
Could you suggest me somenthing else?
Thanks in advance
Tiziana

[Michiel Bliemer]

Your S-estimates are quite reasonable, I assume you refer to the Bayesian S-estimates that are always larger. This is a mean value because for certain draws of your priors (close to zero) it needs a large sample size, so on average this number may blow up. Your uncertainty in your priors is quite large, so it is well possible that therer may be some 'bad' draws that blow up this number. instead of (mnl,d,mean) you can choose (mnl,d,median), which is less sensitive to 'bad' draws. In order to be able to choose the median, you will have to change the draw type to halton, sobol, or mlhs, i.e. ;bdraws = sobol(250).

Using prior (u,-1,0) you indicate that you are sure that the prior is negative, but in estimation it comes out positive, so perhaps you should make this (n,0,0.3) or something like that.

[Tiziana]

Dear Michiel, thanks for your reply. I have the results, but I think that there is somenting strange. Even if the S estimate is quite good, in my efficent design I have a duplicade cards (cards9 and cards 11), as below. I am not sure now what I have to do. Could I drop one of cards? Can I estimate efficient design with low rows ( for example 8) even if I did a pilot with 12 cards, since I was interested in some interaction between attributes? I am very gratefull to you if you can suggest me somenthing.
Thnaks in advance
Tiziana

Design
;alts = alt1, alt2, alt3
;rows = 12
;eff = (mnl,d,mean)
;bdraws=gauss(3)
;model:
U(alt1) = b1[(n,-1.73,0.12)] * PR[0.70, 1.15, 1.60, 2.05] + b2[(n,0.54,0.16)]*HC1[0,1] + b3[(n,-0.36,0.15)]*HC2[0,1] + b4 [(u,0,1)]* HC1 * HC2/
U(alt2) = b1 * PR + b2 *HC1 + b3 *HC2 + b4 * HC1 * HC2/
U(alt3) = b0[(n,-3.15,0.17)]$

MNL efficiency measures

Fixed Bayesian mean
D error 1.035113 1.051087
A error 1.617153 1.640442
B estimate 55.427853 0.543429
S estimate 42.767.496 317.630507

Prior b1 b2 b3 b4
Fixed prior value -1.73 0.54 -0.36 0.5
Sp estimates 1.381337 17.612566 42.767496 40148484
Sp t-ratios 1.667655 0.46703 0.299709 0.30933
Sb mean estimates 1.433475 25.738723 121.083278 246.161727
Sb mean t-ratios 1.652759 0.46466 0.296776 0.307444

Design
Choice situation alt1.pr alt1.hc1 alt1.hc2 alt2.pr alt2.hc1 alt2.hc2 alt1.hc1*alt1.hc2 alt2.hc1*alt2.hc2
1 0.7 0 1 1.15 1 1 0 1
2 1.6 1 0 0.7 0 1 0 0
3 0.7 0 0 1.6 1 1 0 1
4 1.15 1 1 1.6 1 0 1 0
5 1.6 1 1 0.7 0 1 1 0
6 1.15 1 1 1.15 0 0 1 0
7 0.7 1 0 1.6 1 1 0 1
8 1.15 0 0 1.15 1 0 0 0
9 2.05 0 1 2.05 0 0 0 0
10 2.05 0 1 2.05 1 0 0 0
11 2.05 0 0 2.05 0 1 0 0
12 1.6 1 0 0.7 0 0 0 0

[Michiel Bliemer]

Duplicate cards appear because you have specified the model as an alternative-specific model, so a choice task with (1,2) for the first alternative and (3,4) for the second alternative is different from (3,4) for the first alternative and (1,2) for the second alternative. So you simply have to tell Ngene that your alternatives are generic, i.e.
;alts = alt1*, alt2*, alt3

Then Ngene will ensure that such duplicate choice tasks will not occur.