Page 1 of 1

Level overlaps in design

PostPosted: Wed Oct 11, 2023 10:44 am
by pakhi
Hi all,
I am conducting a DCE to elicit preferences for a neurodevelopmental follow-up care for children in Australia. I am currently doing a pilot study mainly to check the survey functionality and obtain priors for the main DCE study. I have 2 alternatives and an opt-out.
I have the following seven attributes: location (levels 0, 1, 2), modality (levels 0, 1, 2), cost (levels 0, 1, 2, 3, 4), mental health support (levels 0, 1), education (levels 0, 1, 2, 3, 4), managing appointments (levels 0, 1), and waiting time (levels 0, 1, 2). I have dummy coded all the levels.
I have some constraints on the design:
• If location is 0, cost can only be 0 or 1.
• If location is 1 and 2, cost can be 1, 2, 3, or 4.
I was able to run the design in NGENE using the codes pasted below; however, I observed:
• When I used 20 (2 blocks) and 30 rows (3 blocks), there were no level overlaps in any of the attributes.
• When I used 40 (4 blocks) and 50 rows (5 blocks), there level overlaps few times in location and cost attributes (without me imposing any overlap constraint).
I am not sure why this was happening? I ended up selecting the design with 30 rows even though the d-error for 40 and 50 rows was better. But I did not want any overlaps in the design.
Are there any other suggestions to improve my design?

Code: Select all
Design
;alts = optA*, optB*, optout
;rows = 30
;block = 3
;eff = (mnl,d)

;cond:
if (optA.location = 0, optA.cost = [0,1]),
if (optB.location = 0, optB.cost = [0,1]),
if (optA.location = [1,2], optA.cost = [1,2,3,4]),
if (optB.location = [1,2], optB.cost = [1,2,3,4])

;model:
U(optA) = b1.dummy[0.000001 | 0.000002]  * location[0, 1, 2]               
            + b2.dummy[0.000001 | 0.000002]   * modality[0, 1, 2]             
            + b3.dummy[0.000001 | 0.000002 | 0.000003 | 0.000004]   * cost[0, 1, 2, 3, 4]             
            + b4.dummy[0.000001]  * mentalhealth[0, 1]         
            + b5.dummy[0.000001 | 0.000002 | 0.000003 | 0.000004]   * education[0, 1, 2, 3, 4]   
            + b6.dummy[0.000001]  * appointments[0, 1]         
            + b7.dummy[0.000001 | 0.000002]   * waiting[0, 1, 2]                 
            /
U(optB) = b1 * location
           + b2 * modality
           + b3 * cost
           + b4 * mentalhealth
           + b5 * education
           + b6 * appointments
           + b7 * waiting
           $


Thanks very much in advance.
Kind regards,
Pakhi

Re: Level overlaps in design

PostPosted: Wed Oct 11, 2023 11:07 am
by Michiel Bliemer
While level overlap will be minimal in efficient designs, it may appear due to certain constraints. For example, the algorithm used in your script considers both attribute level balance constraints and conditional constraints, and with more rows it is more likely that there is some level overlap although when you run the script long enough it should have virtually no overlap.

I ran it with 60 rows (which allows attribute level balance, which is not possible with 50 rows) and in only 3 choice tasks there was only a single attribute overlapping after about 5 minutes of running.

Yes of course the D-error for more rows is always better because it contains more data, but that does not mean that the information collected per choice task is better.

If you want more variation in your design, you can also generate 60 rows and remove some choice tasks that you do not like.

Michiel

Re: Level overlaps in design

PostPosted: Wed Oct 11, 2023 12:04 pm
by pakhi
Thank you Michiel, that helps.

Re: Level overlaps in design

PostPosted: Tue Oct 31, 2023 4:28 pm
by pakhi
Hi all,

In the context of the pilot study discussed above, I proceeded with 30 rows and conducted a pilot involving 53 respondents. From the Nlogit output (pasted below), it seems like cost is a dominant attribute. Given this circumstance, would it be advisable to overlap cost attribute?

Code: Select all
Nlogit
    ;lhs = choice,cset,alti
    ;choices = A,B,Neither
    ;checkdata
    ;model:
    U(A) = A + LOC0*loc0 + LOC1*loc1 + MOD0*mod0 + MOD1*mod1 + COST0*cost0 + COST1*cost1 + COST2*cost2 + COST3*cost3 + MH0*mh0 + EDUC0*educ0 + EDUC1*educ1 + EDUC2*educ2 + EDUC3*educ3 + APP0*app0 + WAIT0*wait0 + WAIT1*wait1 /
    U(B) = B + LOC0*loc0 + LOC1*loc1 + MOD0*mod0 + MOD1*mod1 + COST0*cost0 + COST1*cost1 + COST2*cost2 + COST3*cost3 + MH0*mh0 + EDUC0*educ0 + EDUC1*educ1 + EDUC2*educ2 + EDUC3*educ3 + APP0*app0 + WAIT0*wait0 + WAIT1*wait1 $
+----------------------------------------------------------+
| Inspecting the data set before estimation.               |
| These errors mark observations which will be skipped.    |
| Row Individual = 1st row then group number of data block |
+----------------------------------------------------------+
No bad observations were found in the sample

Normal exit:   6 iterations. Status=0, F=    410.2792

-----------------------------------------------------------------------------
Discrete choice (multinomial logit) model
Dependent variable               Choice
Log likelihood function      -410.27920
Estimation based on N =    530, K =  18
Inf.Cr.AIC  =    856.6 AIC/N =    1.616
Model estimated: Oct 31, 2023, 11:48:23
R2=1-LogL/LogL* Log-L fncn R-sqrd R2Adj
Constants only must be computed directly
               Use NLOGIT ;...;RHS=ONE$
Chi-squared[16]          =    187.86744
Prob [ chi squared > value ] =   .00000
Response data are given as ind. choices
Number of obs.=   530, skipped    0 obs
--------+--------------------------------------------------------------------
        |                  Standard            Prob.      95% Confidence
  CHOICE|  Coefficient       Error       z    |z|>Z*         Interval
--------+--------------------------------------------------------------------
       A|    -.29748         .82757     -.36  .7192    -1.91948   1.32452
    LOC0|     .18770         .38152      .49  .6227     -.56006    .93546
    LOC1|    -.16442         .17467     -.94  .3466     -.50676    .17793
    MOD0|     .29992         .21674     1.38  .1664     -.12489    .72473
    MOD1|     .02539         .19356      .13  .8956     -.35397    .40476
   COST0|    2.10931***      .51951     4.06  .0000     1.09109   3.12753
   COST1|    2.00203***      .32469     6.17  .0000     1.36564   2.63841
   COST2|    1.25454***      .34950     3.59  .0003      .56953   1.93956
   COST3|     .36966         .34588     1.07  .2852     -.30825   1.04757
     MH0|     .07059         .15446      .46  .6476     -.23215    .37333
   EDUC0|    -.14253         .32817     -.43  .6641     -.78574    .50068
   EDUC1|     .47341         .31555     1.50  .1336     -.14506   1.09188
   EDUC2|    -.01836         .31774     -.06  .9539     -.64113    .60440
   EDUC3|     .16538         .30797      .54  .5913     -.43823    .76899
    APP0|    -.08701         .14980     -.58  .5613     -.38061    .20659
   WAIT0|     .91185***      .23671     3.85  .0001      .44790   1.37580
   WAIT1|     .31578         .22628     1.40  .1628     -.12771    .75928
       B|    -.26638         .81606     -.33  .7441    -1.86583   1.33306
--------+--------------------------------------------------------------------
Note: ***, **, * ==>  Significance at 1%, 5%, 10% level.
-----------------------------------------------------------------------------


Thank you.

Kind regards,
Pakhi

Re: Level overlaps in design

PostPosted: Fri Nov 03, 2023 7:51 pm
by Michiel Bliemer
If you would use the parameter estimates from the pilot as priors in your efficient design, Ngene would automatically create some overlap on the cost attribute and only allow small differences in cost across the two alternatives.

You could alternatively consider a smaller range for cost or use a larger number of levels for cost such that cost differences across alternatives can be smaller.

Michiel