Design for pilot test

This forum is for posts covering broader stated choice experimental design issues.

Moderators: Andrew Collins, Michiel Bliemer, johnr

Design for pilot test

Postby samantha » Wed Nov 14, 2018 7:44 am

Hi,

I am designing a DCE pilot test and would appreciate your expertise before I run it. The target sample are prostate cancer patients who have been treated with radiotherapy, and the goal is to determine the MRS for short (b4, levels in %) and long (b5, levels in %) term side effects, and chance of cancer recurrence (b3, levels in %) compared to length of treatment (b1, in weeks). We are hoping to determine what risk of side effects patients are willing to risk for shorter treatment time. b2 is a binary attribute for if an extra procedure is required prior to radiation (yes is 1). This DCE is unlabelled and I excluded the constant based on one of the moderators comments I read on a different post. All the coefficients are negative because the lower percentage risk, shorter treatment time, and no prior procedure should be preferred.

I have the following syntax:

Design
?pilot test, WTP denominator constant, S estimate: n min 55
;alts = alt1*, alt2*
;rows = 24
;eff = (mnl,wtp(wtp1),mean)
;rep = 500
;rdraws = halton(250)
;wtp = wtp1(b3,b4,b5/b1)
;block = 2
;model:
U(alt1) = b1[-0.5] * A[2,4,8] + b2[-0.2] * B[0,1] + b3[(n,-0.3,0.1)] * C[6,12,18] + b4[(n,-0.2,0.1)] * D[20,35,50] + b5[(n,-1.2,0.2)] * E[10,15,20] /
U(alt2) = b1 * A + b2 * B + b3 * C + b4 * D + b5 * E
$

There are previous published DCEs for prostate cancer patients which is where I found the priors, though the definitions of the attributes do not exactly match mine. Questions I have:
1. Is it best to use bayesian mnl for the design since I do not know the exact parameter values? I also don't know if they are normally distributed. Or should I just used fixed values? Does this influence if I can analyze the data with conditional, mixed, bayesian, or latent class logit?
2. A small sample size <60 is likely, for the S generated values by NGene should I be multiplying that by 2 since I have 2 blocks? So I should try to find a design with S< 30?
3. Would you suggest eff by d or WTP?
4. Covariates- do I need to include these in the design stage or can I analyze for subgroups/ preference heterogeneity afterwards? I know my sample size will be small so may not be possible to do subgroup analysis. I am interested if distance from cancer centre, employment status, and age are interacting with preference for length of treatment.
5. To analyze the results of my pilot study (and eventual main DCE) I plan on using SAS as that is free for me. I was going to use conditional logit (MNL) to find the parameters from my pilot test (n=5) and then use these new values to make a new design for my DCE. Does this sound correct?

Thank you very much for your time and expertise!

Sam
samantha
 
Posts: 7
Joined: Tue Nov 13, 2018 4:53 am

Re: Design for pilot test

Postby Michiel Bliemer » Wed Nov 14, 2018 8:23 am

At first glance, your syntax looks fine.

1. There is no such thing as Bayesian MNL, this is not a model type. There exists MNL, mixed MNL, latent class models, nested logit, etc. Bayesian is a term to indicate uncertainty about the priors, the design is Bayesian, the model is not. Bayesian estimation of a mixed MNL model is again something completely different. You can use fixed or random (Bayesian) priors, this does not effect the ability to estimate different model types. You can evaluate your design for estimating a panel mixed MNL model (rppanel in Ngene), I refer to the manual.

2. Note that the sample size estimates in Ngene are only valid if your priors are correct, so there is a lot of uncertainty about them. Yes you have to multiply the S-estimates by 2 to account for 2 blocks.

3. Using a D-efficient or WTP-efficient design will have similar performance. I tend to always use D-efficient designs (it is the most common way to optimise designs) and not WTP-efficient, but that is a matter of taste. WTP-efficient designs will theoretically be slightly better in obtaining reliable WTP measures.

4. You generally add them after data collection. Since you have an unlabelled experiment, you add covariates such as age or employment status as an interaction term (eg A*age).

5. Correct, you estimate an MNL (conditional logit) model and use the parameter estimates (beta) and standard errors (se) as Bayesian priors (n,beta,se).

Michiel
Michiel Bliemer
 
Posts: 1705
Joined: Tue Mar 31, 2009 4:13 pm

Re: Design for pilot test

Postby samantha » Fri Nov 16, 2018 6:35 am

Thank you Michiel! Your reply was very helpful, I was getting Bayesian mixed up.

Taking your advice into account, I now plan on using a d efficient mnl design with Bayesian estimates for the 2 parameters that have less information in the literature, and fixed priors for the other 3 parameters. When I evaluate this design with rp there are slightly higher d value estimates and lower S estimates, but rppanel comes back undefined except the B estimate is 0, though the probabilities and utilities are available. Can you tell my why or what I am doing wrong?

Design
?pilot test d efficient with blocking, no repeats
;alts = alt1*, alt2*
;rows = 24
;eff = (mnl,d,mean)
;block = 2
;model:
U(alt1) = b1[(n,-0.4,0.1)] * A[2,4,8] + b2[-0.2] * B[0,1] + b3[-0.3] * C[6,12,18] + b4[-0.2] * D[20,35,50] + b5[(n,-1.2,0.2)] * E[10,15,20] /
U(alt2) = b1 * A + b2 * B + b3 * C + b4 * D + b5 * E
$

?evaluate above model with rp so random parameters are b1 and b5
Design
;alts = alt1*, alt2*
;rows = 24
;eff = (rp,d,mean)
;block = 2
;eval = D efficient mnl 2 design.ngd
? MNL d = 0.015, s min-max = 30-53; RP d = 0.05, s 24-46
;model:
U(alt1) = b1[n,(n,-0.4,0.1),(u,0,0.2)] * A[2,4,8] + b2[-0.2] * B[0,1] + b3[-0.3] * C[6,12,18] + b4[-0.2] * D[20,35,50] + b5[n,(n,-1.2,0.2),(u,0.1,0.3)] * E[10,15,20] /
U(alt2) = b1 * A + b2 * B + b3 * C + b4 * D + b5 * E
$

Thank you again!
Samantha
samantha
 
Posts: 7
Joined: Tue Nov 13, 2018 4:53 am

Re: Design for pilot test

Postby Michiel Bliemer » Sun Nov 18, 2018 11:09 am

Somehow the combination of random coefficients, panel effects, and Bayesian priors results in some negative values on the diagonal of the Covariance matrix. I am not sure why. If you evaluate for fixed priors in the rppanel model then it works fine. See code below, which optimised for the MNL model but you can inspect the rppanel model properties when you open the design.

Design
?pilot test d efficient with blocking, no repeats
;alts(model1) = alt1*, alt2*
;alts(model2) = alt1*, alt2*
;rows = 24
;eff = model1(mnl,d,mean)
;bdraws = gauss(3)
;rdraws = gauss(3)
;rep = 500
;block = 2
;model(model1):
U(alt1) = b1[(n,-0.4,0.1)] * A[2,4,8] + b2[-0.2] * B[0,1] + b3[-0.3] * C[6,12,18] + b4[-0.2] * D[20,35,50] + b5[(n,-1.2,0.2)] * E[10,15,20] /
U(alt2) = b1 * A + b2 * B + b3 * C + b4 * D + b5 * E
;model(model2):
U(alt1) = b1[n,(n,-0.4,0.1),0.1] * A[2,4,8] + b2[-0.2] * B[0,1] + b3[-0.3] * C[6,12,18] + b4[-0.2] * D[20,35,50] + b5[n,(n,-1.2,0.2),0.2] * E[10,15,20] /
U(alt2) = b1 * A + b2 * B + b3 * C + b4 * D + b5 * E
$

Michiel
Michiel Bliemer
 
Posts: 1705
Joined: Tue Mar 31, 2009 4:13 pm

Re: Design for pilot test

Postby samantha » Fri Nov 23, 2018 6:47 am

Hi Michiel,

Thank you again, that code is very helpful. To help my understanding, I want to clarify why there are 2 models. Model2 is required for the rppanel evaluation due to negatives occurring within the matrix, and the SD are not uniformly distributed but rather are fixed?

Also, why did you specify gauss, do you think this is the better method for draws?

Thanks!
Sam
samantha
 
Posts: 7
Joined: Tue Nov 13, 2018 4:53 am

Re: Design for pilot test

Postby Michiel Bliemer » Fri Nov 23, 2018 9:35 am

Model2 is there merely for the rppanel evaluation so that you do not have to use different syntax for design generation and evaluation.
The reason for setting SD fixed is to resolve the negative values (which I cannot yet explain but this at least works).

Yes gaussian draws are more efficient and lead to more accurate results since they are smarter (but they only work with the mean, not the median).

Michiel
Michiel Bliemer
 
Posts: 1705
Joined: Tue Mar 31, 2009 4:13 pm

Re: Design for pilot test

Postby samantha » Thu Nov 29, 2018 12:21 am

Amazing thank you for all your help!
samantha
 
Posts: 7
Joined: Tue Nov 13, 2018 4:53 am

Re: Design for pilot test

Postby samantha » Wed Dec 05, 2018 10:16 am

Hi Michiel,

I have a follow up question- I realized b2 could be dummy coded though it only has 2 levels, 0 for no and 1 for yes (a procedure participants should select against so parameter set to -0.2). Is the following syntax properly coded for this and should I code it as a dummy variable? I also included an interaction term between A and D, the sign should be negative as the larger values should be selected away from, but the magnitude is 0.01 since I do not have prior information from the literature. Does the correlation/interaction on the design output window give any further information about this interaction?

Design
;alts(model1) = A1*, B2*
;alts(model2) = A1*, B2*
;rows = 24
;eff = model1(mnl,d,mean)
;bdraws = gauss(3)
;rdraws = gauss(3)
;rep = 500
;block = 12
;model(model1):
U(A1) = b1[(n,-0.4,0.1)] * A[2,4,8] + b2.dummy[-0.2] * B[1.0] + b3[-1.3] * C[6,12,18] + b4[-0.2] * D[20,35,50] + b5[(n,-1.2,0.2)] * E[10,15,20] + i1[-0.1] * A * D /
U(B2) = b1 * A + b2 * B + b3 * C + b4 * D + b5 * E + i1 * A * D
;model(model2):
U(A1) = b1[n,(n,-0.4,0.1),0.1] * A[2,4,8] + b2.dummy[-0.2] * B[1,0] + b3[-1.3] * C[6,12,18] + b4[-0.2] * D[20,35,50] + b5[n,(n,-1.2,0.2),0.2] * E[10,15,20] + i1[-0.1] * A * D /
U(B2) = b1 * A + b2 * B + b3 * C + b4 * D + b5 * E + i1 * A * D
$

Thanks!
samantha
 
Posts: 7
Joined: Tue Nov 13, 2018 4:53 am

Re: Design for pilot test

Postby Michiel Bliemer » Wed Dec 05, 2018 12:53 pm

Yes attribute B is coded correctly, except that you need to use B[1,0] instead of B[1.0] in model1.

Further, you should be very careful in setting priors, and the prior for i1 is far too large. You set it to 0.1, and you write setting it to 0.01, and both are too large. The values for A*D vary from 40 to 280, i.e. the contribution to utility varies from 0.4 to 2.8, which is large, especially for an interaction term. Consider setting i1 to 0.001.

Correlations and interactions in the output are not of specific interest, adding the interaction term mainly ensures that you are able to estimate this interaction term using this design.

Michiel
Michiel Bliemer
 
Posts: 1705
Joined: Tue Mar 31, 2009 4:13 pm

Re: Design for pilot test

Postby samantha » Thu Dec 13, 2018 3:21 am

Hi Michiel,

I have started my pilot testing, however I realized I would prefer to analyze my binary attribute using effects coding, not dummy. For the binary attribute the No choice should be preferred by participants and will have a positive parameter possibly 0.9 based on initial data from my pilot study where the attribute was dummy coded. If I switch my B attribute to the following code:

b2.effects[0.9] * B[0,1]

Does this mean that 0=no and 1=yes where 0.9 can be applied to the yes (0) category?

To effects code this for SAS analysis I will then record no=1 and yes=-1 (as the base parameter) based on the Ngene manual page 113?

Thanks,
Samantha
samantha
 
Posts: 7
Joined: Tue Nov 13, 2018 4:53 am

Next

Return to Choice experiments - general

Who is online

Users browsing this forum: No registered users and 4 guests