How to improve the design?

This forum is for posts that specifically focus on Ngene.

Moderators: Andrew Collins, Michiel Bliemer, johnr

How to improve the design?

Postby sukunta » Thu Nov 10, 2022 4:18 pm

Dear Prof.Michiel Bliemer,
I ran the command and the priors were from a pilot study.
Code: Select all
design
;alts=alt1*,alt2*,alt3
;rows=12
;eff =(mnl,d,mean)
;rdraws = random(1000)
;con
;model:

U(alt1) = b0[(n, 18.27865, 0.324166)]+ b1.dummy[(n, -0.00579, 0.166555) | (n,0.126223, 0.132102)] * fish[2,1,0]+ b2.dummy[(n,-0.1081497, 0.1416387) | (n, -0.0120441, 0.1467851)] *vet[2,1,0]+ b3.dummy[(n,0.2731957, 0.1346745) | (n, 0.2358089, 0.1500279)] *alc[2,1,0]+b4.dummy[(n,0.2361314, 0.1924899) | (n, 0.3506409, 0.1678956)] *smk[2,1,0]
        + b5.dummy[(n, 0.3145022, 0.2566367) | (n, 0.231111, 0.1940656)] * pqt[2,1,0] /
U(alt2) = b0
        + b1 * fish
        + b2 * vet
        + b3 * alc
        + b4 * smk
        + b5 * pqt
$

The d-efficient value is so high (3.18). How do I improve this design?
Sincerly yours,
Sukunta
sukunta
 
Posts: 80
Joined: Tue Jan 12, 2016 1:28 pm

Re: How to improve the design?

Postby Michiel Bliemer » Thu Nov 10, 2022 7:13 pm

Your prior for b0 seems to cause issues; it is extremely large (18), and looking at the Fisher information matrix it is not identified when it is that large since the optout (alt3) gets a choice probability of 0. Please check that this value is correct.

Two other comments:
- You should use bdraws (Bayesian draws), not rdraws (which is for random parameters, not Bayesian priors).You should preferably use smarter quasi-random draws. I suggest you use ;bdraws = sobol(1000)
- 12 rows is almost the absolute minimum to be able to estimate this model; I would recommend increasing the number of rows, for example ;rows = 24 and ;block =2

Michiel
Michiel Bliemer
 
Posts: 1705
Joined: Tue Mar 31, 2009 4:13 pm

Re: How to improve the design?

Postby sukunta » Thu Nov 10, 2022 7:56 pm

Dear Prof.Michiel Bliemer,
Thank you so much for your recommendation. Prior to b0 is the results from a pilot study.
    Y Coef. Std. Err. z P>z [95% Conf. Interval]
    asc 18.27865 0.3241657 56.39 0 17.64329 18.914
    fish_d2 0.1262231 0.1321016 0.96 0.339 -0.1326912 0.3851374
    fish_d3 -0.0057893 0.1665554 -0.03 0.972 -0.3322318 0.3206533
    vet_d2 -0.0120441 0.1467851 -0.08 0.935 -0.2997376 0.2756493
    vet_d3 -0.1081497 0.1416387 -0.76 0.445 -0.3857566 0.1694571
    alc_d2 0.2358089 0.1500279 1.57 0.116 -0.0582403 0.5298581
    alc_d3 0.2731957 0.1346745 2.03 0.043 0.0092385 0.5371528
    smk_d2 0.3506409 0.1678956 2.09 0.037 0.0215715 0.6797103
    smk_d3 0.2361314 0.1924899 1.23 0.22 -0.1411417 0.6134046
    pqt_d2 0.231111 0.1940656 1.19 0.234 -0.1492506 0.6114726
    pqt_d3 0.3145022 0.2566367 1.23 0.22 -0.1884964 0.8175009
What can I do to improve the design?
Sincerely yours,
Sukunta
sukunta
 
Posts: 80
Joined: Tue Jan 12, 2016 1:28 pm

Re: How to improve the design?

Postby Michiel Bliemer » Thu Nov 10, 2022 8:24 pm

What is the percentage of observations where the opt-out was chosen?
Michiel Bliemer
 
Posts: 1705
Joined: Tue Mar 31, 2009 4:13 pm

Re: How to improve the design?

Postby sukunta » Thu Nov 10, 2022 8:55 pm

Dear Prof.Michiel Bliemer,
The percentage of opt-out options is zero.
Sincerely yours,
Sukunta
sukunta
 
Posts: 80
Joined: Tue Jan 12, 2016 1:28 pm

Re: How to improve the design?

Postby Michiel Bliemer » Thu Nov 10, 2022 9:07 pm

That explains it, you can only estimate the constant if the opt-out is chosen in some cases.
But the issue is easy to resolve: simply remove ;con from your syntax. This means that Ngene will no longer optimise for the constant (even though it is there in the model) and your D-errors will be normal.

You may want to try to remove the opt-out alternative in model estimation to see if that changes the other parameters.

Michiel
Michiel Bliemer
 
Posts: 1705
Joined: Tue Mar 31, 2009 4:13 pm

Re: How to improve the design?

Postby sukunta » Fri Nov 11, 2022 12:07 pm

Dear Prof.Michiel Bliemer,
Thank you so much for your recommendation. The new syntax was run as follows,
Code: Select all
design
;alts=alt1*,alt2*
;rows=24
;block=2
;eff =(mnl,d,mean)
;bdraws = sobol(1000)
;model:

U(alt1) =  b1.dummy[(n, -0.0179354, 0.1637252) | (n, 0.1334439, 0.1320447)] * fish[2,1,0]+ b2.dummy[(n, -0.0867363, 0.1379654) | (n, -0.0042972, 0.1446489)] *vet[2,1,0]+ b3.dummy[(n, 0.2570482, 0.1329933) | (n, 0.2422942, 0.1494557)] *alc[2,1,0]+b4.dummy[(n, 0.2227193, 0.1939226) | (n, 0.3261694, 0.1718854)] *smk[2,1,0]
        + b5.dummy[(n, 0.3245518, 0.2614673) | (n, 0.2210562, 0.1901518)] * pqt[2,1,0] /
U(alt2) =  b1 * fish+ b2 * vet + b3 * alc + b4 * smk+ b5 * pqt
$


The D error is 0.31489. However, SP estimate value of b1d(0) =4166, b2(0) =72695, and the rest of all is less than 177. This is ok for the main survey of 240 samples.
Sincerely yours,
Sukunta
sukunta
 
Posts: 80
Joined: Tue Jan 12, 2016 1:28 pm

Re: How to improve the design?

Postby Michiel Bliemer » Fri Nov 11, 2022 12:57 pm

Sample size estimates can be very large if the attributes are not that relevant for the choice (which your prior values currently indicate). But prior values are only guesses and these values will change when you collect more data and hence you may actually be able to estimate these parameters at a statistically significant level.

Michiel
Michiel Bliemer
 
Posts: 1705
Joined: Tue Mar 31, 2009 4:13 pm

Re: How to improve the design?

Postby sukunta » Fri Nov 11, 2022 1:16 pm

Dear Prof.Michiel Bliemer,
Thank you so much for your kindness.
Sincerely yours,
Sukunta
sukunta
 
Posts: 80
Joined: Tue Jan 12, 2016 1:28 pm


Return to Choice experiments - Ngene

Who is online

Users browsing this forum: No registered users and 6 guests