First off, I would like to say it has been a pleasure reading through the forums. It really does help me to understand experimental design better. Kudos to Michiel, John and Andrew. Keep up the good work!!
As per my title, I am building a pilot study that incorporates a dual response design (i.e a forced choice between TxA and TxB, and immediately after, an unforced choice task - TxA/TxB chosen in the first task or Neither (Status Quo). I did come across a post by John (i think) where he had a similar design (i.e. TxA, TxB, SQ, and TxA or TxB alternative if SQ was chosen). It might work for my pilot as well. I'll have to go through the paper to see if it fits my study. Anyway, here is what I have done:
- Unlabelled, non-zero small priors value, 7 attributes 3 levels each.
Design
; alts = TxA*, TxB*, SQ
;eff = (mnl,d)
;rows = 24
;block=2
;alg=mfederov
;model:
U(TxA) = b2[0.01]*freq[6,8,12] + b3[-0.01]*fla[30,50,70] + b4[-0.01]*inf[2,4,6] + b5[-0.01]*canc[6,8,10] + b6[0.01]*reg[60,80,100]+ b7[-0.01]*time[0,48,120] + b8[0.01]*test[50,70,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[0] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[40] +b7*timesq[0] + b8*testsq[50]
$
My questions are:
i. Will my syntax code work in a dual response design? I search through the forum, yes, there are SQ and Opt-out topics but none specific for a dual response design. I would like to think the syntax code will be similar for any opt-out / SQ design but I could be wrong.
ii. Ideally, I would like to optimize my experimental design for attribute non-linear effect. I tried the model averaging approach in which the second model had all attributes effect coded and applied a constraint to account for the effect coding on status quo alternative but did not quite succeeded. "Error: There were problems generating a fractional factorial of choice tasks. For the modified federov algorithm, increasing the number of candidates might assist." Even with a very large candidates number, I still fail to get a design. I am thinking it might be because effect coding was applied to all 7 attributes. Is there any other alternative to average the model for non-linear effect or the first model above is sufficient? The syntax code is as below if it helps:
Design
;alts (m1) = TxA*, TxB*, SQ
;alts (m2) = TxA*, TxB*, SQ
;eff = m1(mnl,d) + 2*m2 (mnl, d)
;rows = 36
;block = 3
;alg=mfederov(candidates=1000)
;require:
sq.freq=4, sq.fla=25, sq.inf=8, sq.canc=14, sq.reg=40, sq.time=0, sq.test=50
;model (m1):
U(TxA) = b2a[0.01]*freq[6,8,12] + b3a[-0.01]*fla[30,50,70] + b4a[-0.01]*inf[2,4,6] + b5a[-0.01]*canc[8,10,12] + b6a[0.01]*reg[60,80,100]+ b7a[-0.01]*time[0,48,120] + b8a[0.01]*test[50,70,90] /
U(TxB) = b2a*freq + b3a*fla+ b4a*inf + b5a*canc + b6a*reg + b7a*time + b8a*test /
U(SQ) = b1a[0] + b2a*freq + b3a*fla + b4a*inf + b5a*canc + b6a*reg + b7a*time + b8a*test
;model (m2):
U(TxA) = b2.effect[0|0]*freq[6,8,12] + b3.effect[0|0]*fla[30,50,70] + b4.effect[0|0]*inf[2,4,6] + b5.effect[0|0]*canc[8,10,12] + b6.effect[0|-0]*reg[60,80,100]+ b7.effect[0|0]*time[0,48,120] + b8.effect[0|0]*test[50,70,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[0] + b2*freq + b3*fla + b4*inf + b5*canc + b6*reg + b7*time + b8*test
$
Love to hear all the comments and feedbacks. Much appreciated.
Regards
Jack
A dual response pilot study design
Moderators: Andrew Collins, Michiel Bliemer, johnr
12 posts
• Page 1 of 2 • 1, 2
A dual response pilot study design
by Jack » Fri Mar 29, 2019 7:43 pm
- Jack
- Posts: 6
- Joined: Wed Mar 27, 2019 7:55 pm
Re: A dual response pilot study design
by Michiel Bliemer » Tue Apr 02, 2019 3:46 pm
First of all, could I suggest that you first to an unconditional (unforced) choice and than a conditional (forced) choice? The reason is the following. If your alternatives are {TxA, TxB, neither}, then if TxA or TxB is selected then there is no more need to ask the forced choice because it would be the same. In the studies with John we automatically select TxA or TxB in the unforced choice if TxA or TxB in the forced choice is selected. If you first ask the forced choice and a respondent chooses, say, TxA, would you allow them to select TxB in the unforced choice?
To answer your questions:
i. You want a homogeneous design for two models, i.e. the same design that you can use to estimate two models, namely one with and one without the status quo alternative. I tried the syntax below, which would do exactly what you want, but unfortunately Ngene currently does not allow the number of alternatives to be different, so it will not run.
Design
;alts(unforced) = TxA*, TxB*, SQ
;alts(forced) = TxA*, TxB*
;eff = combined(mnl,d)
;rows = 24
;block = 2
;alg = mfederov
;fisher(combined) = design1(unforced[0.5],forced[0.5])
;model(unforced):
U(TxA) = b2[0.01]*freq[6,8,12] + b3[-0.01]*fla[30,50,70] + b4[-0.01]*inf[2,4,6] + b5[-0.01]*canc[6,8,10] + b6[0.01]*reg[60,80,100]+ b7[-0.01]*time[0,48,120] + b8[0.01]*test[50,70,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[0] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[40] +b7*timesq[0] + b8*testsq[50]
;model(forced):
U(TxA) = b2[0.01]*freq[6,8,12] + b3[-0.01]*fla[30,50,70] + b4[-0.01]*inf[2,4,6] + b5[-0.01]*canc[6,8,10] + b6[0.01]*reg[60,80,100]+ b7[-0.01]*time[0,48,120] + b8[0.01]*test[50,70,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test
$
This can be resolved using the following trick: add SQ to the forced choice, but make sure that its choice probability is essentially zero. This can be achieved by setting the prior for b1 to -9 or something like that. The syntax below will optimise the design for the forced and unforced choice simultaneously:
Design
;alts(unforced) = TxA*, TxB*, SQ
;alts(forced) = TxA*, TxB*, SQ
;eff = combined(mnl,d)
;rows = 24
;block = 2
;alg = mfederov
;fisher(combined) = design1(unforced[0.5],forced[0.5])
;model(unforced):
U(TxA) = b2[0.01]*freq[6,8,12] + b3[-0.01]*fla[30,50,70] + b4[-0.01]*inf[2,4,6] + b5[-0.01]*canc[6,8,10] + b6[0.01]*reg[60,80,100]+ b7[-0.01]*time[0,48,120] + b8[0.01]*test[50,70,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[0] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[40] +b7*timesq[0] + b8*testsq[50]
;model(forced):
U(TxA) = b2[0.01]*freq[6,8,12] + b3[-0.01]*fla[30,50,70] + b4[-0.01]*inf[2,4,6] + b5[-0.01]*canc[6,8,10] + b6[0.01]*reg[60,80,100]+ b7[-0.01]*time[0,48,120] + b8[0.01]*test[50,70,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[-9] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[40] +b7*timesq[0] + b8*testsq[50]
$
ii. You can do this, but freq = 4 etc also need to appear in the possible attribute levels for freq as otherwise the dummy or effects coded variable is not properly defined. Further, since freq = 4 only appears in the SQ alternative, you cannot estimate a constant (the model would be overspecified, nonidentifiable). As an example with freq dummy coded:
;require:
TxA.freq > 4,
TxB.freq > 4,
sq.freq = 4
;model:
U(TxA) = b2.dummy[0|0|0]*freq[4,6,8,12] + b3[-0.01]*fla[30,50,70] + b4[-0.01]*inf[2,4,6] + b5[-0.01]*canc[6,8,10] + b6[0.01]*reg[60,80,100]+ b7[-0.01]*time[0,48,120] + b8[0.01]*test[50,70,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b2*freq + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[40] +b7*timesq[0] + b8*testsq[50]
$
Michiel
To answer your questions:
i. You want a homogeneous design for two models, i.e. the same design that you can use to estimate two models, namely one with and one without the status quo alternative. I tried the syntax below, which would do exactly what you want, but unfortunately Ngene currently does not allow the number of alternatives to be different, so it will not run.
Design
;alts(unforced) = TxA*, TxB*, SQ
;alts(forced) = TxA*, TxB*
;eff = combined(mnl,d)
;rows = 24
;block = 2
;alg = mfederov
;fisher(combined) = design1(unforced[0.5],forced[0.5])
;model(unforced):
U(TxA) = b2[0.01]*freq[6,8,12] + b3[-0.01]*fla[30,50,70] + b4[-0.01]*inf[2,4,6] + b5[-0.01]*canc[6,8,10] + b6[0.01]*reg[60,80,100]+ b7[-0.01]*time[0,48,120] + b8[0.01]*test[50,70,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[0] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[40] +b7*timesq[0] + b8*testsq[50]
;model(forced):
U(TxA) = b2[0.01]*freq[6,8,12] + b3[-0.01]*fla[30,50,70] + b4[-0.01]*inf[2,4,6] + b5[-0.01]*canc[6,8,10] + b6[0.01]*reg[60,80,100]+ b7[-0.01]*time[0,48,120] + b8[0.01]*test[50,70,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test
$
This can be resolved using the following trick: add SQ to the forced choice, but make sure that its choice probability is essentially zero. This can be achieved by setting the prior for b1 to -9 or something like that. The syntax below will optimise the design for the forced and unforced choice simultaneously:
Design
;alts(unforced) = TxA*, TxB*, SQ
;alts(forced) = TxA*, TxB*, SQ
;eff = combined(mnl,d)
;rows = 24
;block = 2
;alg = mfederov
;fisher(combined) = design1(unforced[0.5],forced[0.5])
;model(unforced):
U(TxA) = b2[0.01]*freq[6,8,12] + b3[-0.01]*fla[30,50,70] + b4[-0.01]*inf[2,4,6] + b5[-0.01]*canc[6,8,10] + b6[0.01]*reg[60,80,100]+ b7[-0.01]*time[0,48,120] + b8[0.01]*test[50,70,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[0] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[40] +b7*timesq[0] + b8*testsq[50]
;model(forced):
U(TxA) = b2[0.01]*freq[6,8,12] + b3[-0.01]*fla[30,50,70] + b4[-0.01]*inf[2,4,6] + b5[-0.01]*canc[6,8,10] + b6[0.01]*reg[60,80,100]+ b7[-0.01]*time[0,48,120] + b8[0.01]*test[50,70,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[-9] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[40] +b7*timesq[0] + b8*testsq[50]
$
ii. You can do this, but freq = 4 etc also need to appear in the possible attribute levels for freq as otherwise the dummy or effects coded variable is not properly defined. Further, since freq = 4 only appears in the SQ alternative, you cannot estimate a constant (the model would be overspecified, nonidentifiable). As an example with freq dummy coded:
;require:
TxA.freq > 4,
TxB.freq > 4,
sq.freq = 4
;model:
U(TxA) = b2.dummy[0|0|0]*freq[4,6,8,12] + b3[-0.01]*fla[30,50,70] + b4[-0.01]*inf[2,4,6] + b5[-0.01]*canc[6,8,10] + b6[0.01]*reg[60,80,100]+ b7[-0.01]*time[0,48,120] + b8[0.01]*test[50,70,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b2*freq + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[40] +b7*timesq[0] + b8*testsq[50]
$
Michiel
- Michiel Bliemer
- Posts: 1885
- Joined: Tue Mar 31, 2009 4:13 pm
Re: A dual response pilot study design
by Jack » Mon Aug 26, 2019 9:06 pm
Good day Ngene users,
I completed my pilot design using the syntax given by Prof Michiel above for a dual response SC experiment. Thank you so much for your input. I am now preparing to use the data analyzed from the pilot to start designing the experimental design for the main study. Given the small pilot sample size (n=15), I was only able to get 4 (out of the 7 attributes) parameter estimates at a significant value. Of the 3 non-significant parameter estimates, 2 of those did not exhibit the expected sign (highlighted in red) where both should be negative. Excerpts from the results as below:
--------+--------------------------------------------------------------------
| Standard Prob. 95% Confidence
CHOICE| Coefficient Error z |z|>Z* Interval
--------+--------------------------------------------------------------------
SQ| -.87549** .43531 -2.01 .0443 -1.72869 -.02230
FREQ| .06861** .03417 2.01 .0447 .00163 .13559
FLARE| -.01403*** .00509 -2.75 .0059 -.02401 -.00405
INF| -.09545*** .03614 -2.64 .0083 -.16629 -.02460
CANCER| .03031 .03772 .80 .4217 -.04363 .10424
REGAIN .00493 .00386 1.28 .2018 -.00264 .01249
TIME| .00094 .00203 .46 .6438 -.00304 .00492
TEST| .01533*** .00569 2.69 .0071 .00417 .02648
--------+--------------------------------------------------------------------
***, **, * ==> Significance at 1%, 5%, 10% level.
I still plan to use the parameter estimates obtained above as priors for a Bayesian efficient MNL model, but with modification, i.e. to still use the data as priors for 5 parameters (FREQ,FLARE, INF, REGAIN, TEST). The bolded red parameter will have a very small non-bayesian negative prior (-0.0001).
Q1. Before I move on with the syntax, is the above proposal acceptable? If not, what are the other solutions available to deal with non-significant parameter estimates and/or in my case non-expected parameter sign?
Reference from above and from the info i gathered, the mean and SE of each parameter was divided by 2 to have a conservative prior value for Bayesian MNL. I did not change the syntax except for the prior value.
Design
;alts(unforced) = TxA*, TxB*, SQ
;alts(forced) = TxA*, TxB*, SQ
;eff = combined(mnl,d,mean)
;rows = 24
;block = 2
;alg = mfederov
;fisher(combined) = design1(unforced[0.5],forced[0.5])
;model(unforced):
U(TxA) = b2[(n,0.035,0.017)]*freq[6,8,12] + b3[(n,-0.007,0.0025)]*fla[25,30,50,70] + b4[(n,-0.05,0.02)]*inf[2,4,6,8] + b5[-0.0001]*canc[8,10,12,14] + b6[(n,0.0049,0.002)]*reg[40,60,80,100]+ b7[-0.0001]*time[0,48,120,240] + b8[(n,0.007,0.003)]*test[50,70,80,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[0] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50]
;model(forced):
U(TxA) = b2[(n,0.035,0.017)]*freq[6,8,12] + b3[(n,-0.007,0.0025)]*fla[25,30,50,70] + b4[(n,-0.05,0.02)]*inf[2,4,6,8] + b5[-0.0001]*canc[8,10,12,14] + b6[(n,0.0049,0.002)]*reg[40,60,80,100]+ b7[-0.0001]*time[0,48,120,240] + b8[(n,0.007,0.003)]*test[50,70,80,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[-9] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50] $
Q2. There is one thing I missed asking Prof Bliemer is the syntax
;fisher(combined) = design1(unforced[0.5],force[0.5]) which I assumed it relates the probability or weight attached to the forced/unforced choice? I could be wrong and appreciate some explanation. Given that the ASC 'SQ' in the pilot was -.87549**, do I need to change the parameter value in the syntax above?
Q3. While I plan to optimize for Bayesian MNL, it will also be evaluated for MMNL model. How do I specify the value for parameter and distribution of CANCER and TIME given both are of the wrong expected sign? Can an arbitrary small mean and StDev be used as the substitute?
Appreciate the time and effort for the questions above.
Jack
I completed my pilot design using the syntax given by Prof Michiel above for a dual response SC experiment. Thank you so much for your input. I am now preparing to use the data analyzed from the pilot to start designing the experimental design for the main study. Given the small pilot sample size (n=15), I was only able to get 4 (out of the 7 attributes) parameter estimates at a significant value. Of the 3 non-significant parameter estimates, 2 of those did not exhibit the expected sign (highlighted in red) where both should be negative. Excerpts from the results as below:
--------+--------------------------------------------------------------------
| Standard Prob. 95% Confidence
CHOICE| Coefficient Error z |z|>Z* Interval
--------+--------------------------------------------------------------------
SQ| -.87549** .43531 -2.01 .0443 -1.72869 -.02230
FREQ| .06861** .03417 2.01 .0447 .00163 .13559
FLARE| -.01403*** .00509 -2.75 .0059 -.02401 -.00405
INF| -.09545*** .03614 -2.64 .0083 -.16629 -.02460
CANCER| .03031 .03772 .80 .4217 -.04363 .10424
REGAIN .00493 .00386 1.28 .2018 -.00264 .01249
TIME| .00094 .00203 .46 .6438 -.00304 .00492
TEST| .01533*** .00569 2.69 .0071 .00417 .02648
--------+--------------------------------------------------------------------
***, **, * ==> Significance at 1%, 5%, 10% level.
I still plan to use the parameter estimates obtained above as priors for a Bayesian efficient MNL model, but with modification, i.e. to still use the data as priors for 5 parameters (FREQ,FLARE, INF, REGAIN, TEST). The bolded red parameter will have a very small non-bayesian negative prior (-0.0001).
Q1. Before I move on with the syntax, is the above proposal acceptable? If not, what are the other solutions available to deal with non-significant parameter estimates and/or in my case non-expected parameter sign?
Reference from above and from the info i gathered, the mean and SE of each parameter was divided by 2 to have a conservative prior value for Bayesian MNL. I did not change the syntax except for the prior value.
Design
;alts(unforced) = TxA*, TxB*, SQ
;alts(forced) = TxA*, TxB*, SQ
;eff = combined(mnl,d,mean)
;rows = 24
;block = 2
;alg = mfederov
;fisher(combined) = design1(unforced[0.5],forced[0.5])
;model(unforced):
U(TxA) = b2[(n,0.035,0.017)]*freq[6,8,12] + b3[(n,-0.007,0.0025)]*fla[25,30,50,70] + b4[(n,-0.05,0.02)]*inf[2,4,6,8] + b5[-0.0001]*canc[8,10,12,14] + b6[(n,0.0049,0.002)]*reg[40,60,80,100]+ b7[-0.0001]*time[0,48,120,240] + b8[(n,0.007,0.003)]*test[50,70,80,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[0] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50]
;model(forced):
U(TxA) = b2[(n,0.035,0.017)]*freq[6,8,12] + b3[(n,-0.007,0.0025)]*fla[25,30,50,70] + b4[(n,-0.05,0.02)]*inf[2,4,6,8] + b5[-0.0001]*canc[8,10,12,14] + b6[(n,0.0049,0.002)]*reg[40,60,80,100]+ b7[-0.0001]*time[0,48,120,240] + b8[(n,0.007,0.003)]*test[50,70,80,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[-9] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50] $
Q2. There is one thing I missed asking Prof Bliemer is the syntax
;fisher(combined) = design1(unforced[0.5],force[0.5]) which I assumed it relates the probability or weight attached to the forced/unforced choice? I could be wrong and appreciate some explanation. Given that the ASC 'SQ' in the pilot was -.87549**, do I need to change the parameter value in the syntax above?
Q3. While I plan to optimize for Bayesian MNL, it will also be evaluated for MMNL model. How do I specify the value for parameter and distribution of CANCER and TIME given both are of the wrong expected sign? Can an arbitrary small mean and StDev be used as the substitute?
Appreciate the time and effort for the questions above.
Jack
- Jack
- Posts: 6
- Joined: Wed Mar 27, 2019 7:55 pm
Re: A dual response pilot study design
by Michiel Bliemer » Tue Aug 27, 2019 9:56 am
Q1. Yes that sounds fine. Since you used an efficient design for getting the priors, and not an orthogonal or other design, you may not need to make the priors more conservative, so you could choose to not divide by two, but either way would be fine I think.
Q2. Yes you would need to use the prior for b1 that you estimated.
Q3. You would need to estimate a panel MMNL model to obtain priors for the beta_mean and beta_stdev values, you should not use the MNL estimates. It may be difficult to estimate an MMNL model based on only 15 observations, so you could simply evaluate the panel MMNL model using zero priors for all coefficients, e.g. b[n,0,0], which will tell you whether you can estimate the model or not (although it will not provide you with sample size information).
Michiel
Q2. Yes you would need to use the prior for b1 that you estimated.
Q3. You would need to estimate a panel MMNL model to obtain priors for the beta_mean and beta_stdev values, you should not use the MNL estimates. It may be difficult to estimate an MMNL model based on only 15 observations, so you could simply evaluate the panel MMNL model using zero priors for all coefficients, e.g. b[n,0,0], which will tell you whether you can estimate the model or not (although it will not provide you with sample size information).
Michiel
- Michiel Bliemer
- Posts: 1885
- Joined: Tue Mar 31, 2009 4:13 pm
Re: A dual response pilot study design
by Jack » Wed Aug 28, 2019 9:27 am
Good day Prof,
Thank you so much for answering my questions. Much appreciated. This brings me to my next query, please bear with me.
Q1. One of my attribute (FREQ) level -i.e 4 will only appear in the SQ alternative and not TxA or TxB (where levels are 6,8,12). In my recently completed pilot study, i ran the syntax as suggested below (simplified so not to bore you with the long syntax code):
;model(unforced):
U(TxA) = b2[0.01]*freq[6,8,12] + b3......../ U(TxB) = b2*freq + b3*fla...... / U(SQ) = b1[0] + b2*freqsq[4] + b3*flasq........
;model(forced):
U(TxA) =b2[0.01]*freq[6,8,12] + b3......../ U(TxB) = b2*freq + b3*fla...... / U(SQ) = b1[-9] + b2*freqsq[4] + b3*flasq........ $
I am wondering what are (or will there be) significant differences in the efficiency of the experimental design and/or subsequently calculated parameter estimates if i add the attribute level '4' into the possible attribute levels for FREQ i.e 4,6,8,12 and uses ";require TxA.freq > 4, TxB.freq > 4, sq.freq = 4" so it becomes
Design
;alts(unforced) = TxA*, TxB*, SQ
;alts(forced) = TxA*, TxB*, SQ
;eff = combined(mnl,d,mean)
;rows = 36
;block = 3
;alg = mfederov
;require:
TxA.freq > 4,
TxB.freq > 4,
sq.freq = 4
;fisher(combined) = design1(unforced[0.5],forced[0.5])
;model(unforced):
U(TxA) = b2[0.01]*freq[4,6,8,12] + b3......../
U(TxB) = b2*freq + b3*fla...... /
U(SQ) = b1[0] + b2*freq + b3*flasq........
;model(forced):
U(TxA) =b2[0.01]*freq[4,6,8,12] + b3......../
U(TxB) = b2*freq + b3*fla...... /
U(SQ) = b1[-9] + b2*freq + b3*flasq........ $
Q2. I wrote the below syntax to check whether I can evaluate the panel MMNL model, I ran the MMNL model with my pilot data but because there is no significant estimates for all parameters, zero priors for all coefficient (n,0,0) are used. However, I am not sure how to specify the below command. Can Prof please advise?
Design
;alts(unforced_mnl) = TxA*, TxB*, SQ
;alts(forced_mnl) = TxA*, TxB*, SQ
;alts(unforced_rppanel) = TxA*, TxB*, SQ
;alts(forced_rppanel) = TxA*, TxB*, SQ
;eff = combined(mnl,d)+(rppanel,d)????
;rows = 36
;block = 3
;alg = mfederov
;bdraws=gauss(2)
;rdraws = gauss (2)
;fisher(combined) = design1(unforced_mnl[0.5],forced_mnl[0.5]) ???????
;model(unforced_mnl):
U(TxA) = b2[(n,0.07,0.03)]*freq[6,8,12] + b3[(n,-0.013,0.005)]*fla[25,30,50,70] + b4[(n,-0.11,0.04)]*inf[2,4,6,8] + b5[-0.0001]*canc[8,10,12,14] + b6[0.0001]*reg[40,60,80,100]+ b7[-0.0001]*time[0,48,120,240] + b8[(n,0.013,0.006)]*test[50,70,80,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[0] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50]
;model(forced_mnl):
U(TxA) = b2[(n,0.07,0.03)]*freq[6,8,12] + b3[(n,-0.013,0.005)]*fla[25,30,50,70] + b4[(n,-0.11,0.04)]*inf[2,4,6,8] + b5[-0.0001]*canc[8,10,12,14] + b6[0.0001]*reg[40,60,80,100]+ b7[-0.0001]*time[0,48,120,240] + b8[(n,0.013,0.006)]*test[50,70,80,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[(n,-1.05,0.46)] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50] $b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50]
;model(unforced_rppanel):
U(TxA) = b2[n,0,0]*freq[6,8,12] + b3[n,0,0]*fla[25,30,50,70] + b4[(n,0,0]*inf[2,4,6,8] + b5[n,0,0]*canc[8,10,12,14] + b6[n,0,0]*reg[40,60,80,100]+ b7[n,0,0]*time[0,48,120,240] + b8[n,0,0]*test[50,70,80,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[0] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50]
;model(forced_rppanel):
U(TxA) = b2[n,0,0]*freq[6,8,12] + b3[n,0,0]*fla[25,30,50,70] + b4[(n,0,0]*inf[2,4,6,8] + b5[n,0,0]*canc[8,10,12,14] + b6[n,0,0]*reg[40,60,80,100]+ b7[n,0,0]*time[0,48,120,240] + b8[n,0,0]*test[50,70,80,90]/
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[n,0,0] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50] $b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50] $
Thank you so much.
Jack
Thank you so much for answering my questions. Much appreciated. This brings me to my next query, please bear with me.
Q1. One of my attribute (FREQ) level -i.e 4 will only appear in the SQ alternative and not TxA or TxB (where levels are 6,8,12). In my recently completed pilot study, i ran the syntax as suggested below (simplified so not to bore you with the long syntax code):
;model(unforced):
U(TxA) = b2[0.01]*freq[6,8,12] + b3......../ U(TxB) = b2*freq + b3*fla...... / U(SQ) = b1[0] + b2*freqsq[4] + b3*flasq........
;model(forced):
U(TxA) =b2[0.01]*freq[6,8,12] + b3......../ U(TxB) = b2*freq + b3*fla...... / U(SQ) = b1[-9] + b2*freqsq[4] + b3*flasq........ $
I am wondering what are (or will there be) significant differences in the efficiency of the experimental design and/or subsequently calculated parameter estimates if i add the attribute level '4' into the possible attribute levels for FREQ i.e 4,6,8,12 and uses ";require TxA.freq > 4, TxB.freq > 4, sq.freq = 4" so it becomes
Design
;alts(unforced) = TxA*, TxB*, SQ
;alts(forced) = TxA*, TxB*, SQ
;eff = combined(mnl,d,mean)
;rows = 36
;block = 3
;alg = mfederov
;require:
TxA.freq > 4,
TxB.freq > 4,
sq.freq = 4
;fisher(combined) = design1(unforced[0.5],forced[0.5])
;model(unforced):
U(TxA) = b2[0.01]*freq[4,6,8,12] + b3......../
U(TxB) = b2*freq + b3*fla...... /
U(SQ) = b1[0] + b2*freq + b3*flasq........
;model(forced):
U(TxA) =b2[0.01]*freq[4,6,8,12] + b3......../
U(TxB) = b2*freq + b3*fla...... /
U(SQ) = b1[-9] + b2*freq + b3*flasq........ $
Q2. I wrote the below syntax to check whether I can evaluate the panel MMNL model, I ran the MMNL model with my pilot data but because there is no significant estimates for all parameters, zero priors for all coefficient (n,0,0) are used. However, I am not sure how to specify the below command. Can Prof please advise?
Design
;alts(unforced_mnl) = TxA*, TxB*, SQ
;alts(forced_mnl) = TxA*, TxB*, SQ
;alts(unforced_rppanel) = TxA*, TxB*, SQ
;alts(forced_rppanel) = TxA*, TxB*, SQ
;eff = combined(mnl,d)+(rppanel,d)????
;rows = 36
;block = 3
;alg = mfederov
;bdraws=gauss(2)
;rdraws = gauss (2)
;fisher(combined) = design1(unforced_mnl[0.5],forced_mnl[0.5]) ???????
;model(unforced_mnl):
U(TxA) = b2[(n,0.07,0.03)]*freq[6,8,12] + b3[(n,-0.013,0.005)]*fla[25,30,50,70] + b4[(n,-0.11,0.04)]*inf[2,4,6,8] + b5[-0.0001]*canc[8,10,12,14] + b6[0.0001]*reg[40,60,80,100]+ b7[-0.0001]*time[0,48,120,240] + b8[(n,0.013,0.006)]*test[50,70,80,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[0] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50]
;model(forced_mnl):
U(TxA) = b2[(n,0.07,0.03)]*freq[6,8,12] + b3[(n,-0.013,0.005)]*fla[25,30,50,70] + b4[(n,-0.11,0.04)]*inf[2,4,6,8] + b5[-0.0001]*canc[8,10,12,14] + b6[0.0001]*reg[40,60,80,100]+ b7[-0.0001]*time[0,48,120,240] + b8[(n,0.013,0.006)]*test[50,70,80,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[(n,-1.05,0.46)] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50] $b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50]
;model(unforced_rppanel):
U(TxA) = b2[n,0,0]*freq[6,8,12] + b3[n,0,0]*fla[25,30,50,70] + b4[(n,0,0]*inf[2,4,6,8] + b5[n,0,0]*canc[8,10,12,14] + b6[n,0,0]*reg[40,60,80,100]+ b7[n,0,0]*time[0,48,120,240] + b8[n,0,0]*test[50,70,80,90] /
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[0] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50]
;model(forced_rppanel):
U(TxA) = b2[n,0,0]*freq[6,8,12] + b3[n,0,0]*fla[25,30,50,70] + b4[(n,0,0]*inf[2,4,6,8] + b5[n,0,0]*canc[8,10,12,14] + b6[n,0,0]*reg[40,60,80,100]+ b7[n,0,0]*time[0,48,120,240] + b8[n,0,0]*test[50,70,80,90]/
U(TxB) = b2*freq + b3*fla+ b4*inf + b5*canc + b6*reg + b7*time + b8*test /
U(SQ) = b1[n,0,0] + b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50] $b2*freqsq[4] + b3*flasq[25] + b4*infsq[8] + b5*cancsq[14] +b6*regsq[100] +b7*timesq[0] + b8*testsq[50] $
Thank you so much.
Jack
- Jack
- Posts: 6
- Joined: Wed Mar 27, 2019 7:55 pm
Re: A dual response pilot study design
by Michiel Bliemer » Wed Aug 28, 2019 10:06 am
1) No that should not make any difference since the levels used for freq in the A and B alternatives will still be 6, 8, and 12, and for the SQ alternative you only use 4. The only difference is that it becomes a bit more difficult for Ngene to find an efficient design because you are at the same time increasing the number of possible designs as well as imposing more constraints.
2) You should not optimise for the rppanel model, only evaluate. So you can simply leave out (rppanel,d) in the ;eff command. Since you specified two models, Ngene will show you the results for both models when you open the design. Note that the computation time for the rppanel model can be huge. In your case, it will be 2^7*500 = 64,000 evaluations (assuming you use the default ;rep = 500). So when you click the rppanel button in the design window, please be patient while Ngene calculates.
Michiel
2) You should not optimise for the rppanel model, only evaluate. So you can simply leave out (rppanel,d) in the ;eff command. Since you specified two models, Ngene will show you the results for both models when you open the design. Note that the computation time for the rppanel model can be huge. In your case, it will be 2^7*500 = 64,000 evaluations (assuming you use the default ;rep = 500). So when you click the rppanel button in the design window, please be patient while Ngene calculates.
Michiel
- Michiel Bliemer
- Posts: 1885
- Joined: Tue Mar 31, 2009 4:13 pm
Re: A dual response pilot study design
by Jack » Fri Aug 30, 2019 8:59 am
Thanks Prof Michiel for the guidance and clarification. Much appreciated.
- Jack
- Posts: 6
- Joined: Wed Mar 27, 2019 7:55 pm
Re: A dual response pilot study design
by Jack » Tue Sep 03, 2019 10:04 pm
Thanks Paul for the comment. Edited my post to rework the syntax.
Last edited by Jack on Tue Sep 03, 2019 10:52 pm, edited 2 times in total.
- Jack
- Posts: 6
- Joined: Wed Mar 27, 2019 7:55 pm
Re: A dual response pilot study design
by paulm » Tue Sep 03, 2019 10:20 pm
A couple of thoughts from the peanut gallery:
First, your syntax in red doesn't make any sense. You can't be both linear and effects coded at the same time. b3.effects is a single, unified, piece of syntax. b3[].effects should cause an error. Think carefully about your parameter specification in your model and you'll see why.
Also, once you effects code FREQ and put in the restrictions, your model isn't identified anymore. TxA and TxB have a 3-level attribute for freq, with values starting at 8. The SQ option has some other value (happens to be 4) that is buried in the constant (ASC). You can't estimate all the parameters you want without testing FREQ = 4 in TxA and TxB.
Paul
First, your syntax in red doesn't make any sense. You can't be both linear and effects coded at the same time. b3.effects is a single, unified, piece of syntax. b3[].effects should cause an error. Think carefully about your parameter specification in your model and you'll see why.
Also, once you effects code FREQ and put in the restrictions, your model isn't identified anymore. TxA and TxB have a 3-level attribute for freq, with values starting at 8. The SQ option has some other value (happens to be 4) that is buried in the constant (ASC). You can't estimate all the parameters you want without testing FREQ = 4 in TxA and TxB.
Paul
- paulm
- Posts: 25
- Joined: Wed Jan 09, 2013 3:51 am
Re: A dual response pilot study design
by Michiel Bliemer » Wed Sep 04, 2019 10:48 am
I was under the assumption that Jack was estimating a single parameter, i.e. linear coding, and then there is no issue. Currently the syntax assumes linear coding. If indeed effects or dummy coding is desired, then there will be an issue as pointed out by Paul.
Michiel
Michiel
- Michiel Bliemer
- Posts: 1885
- Joined: Tue Mar 31, 2009 4:13 pm
12 posts
• Page 1 of 2 • 1, 2
Return to Choice experiments - Ngene
Who is online
Users browsing this forum: No registered users and 32 guests