Opt-out experimental design

[ASAL2433]

Hi,

Thank you for your help on my previous questions! I have found this page very helpful!!

I am conducting a DCE, with the following options: Screening Approach A, Screening Approach B or no screening. I will implement the opt-out using a two-staged approach i.e. (1) all three options presented, and (2) if no screening is chosen they will be presented with a forced choice (Approach A or B).

I will be using a D-efficient design, which I have currently written as:

Design

;alts = AppA, AppB
;rows = 60
;block = 5
; cond:
If (AppA.CONDITIONS=0, AppA.WTIME= [2,6]),
If (AppB.CONDITIONS=0, AppB.WTIME= [2,6])
;eff = (mnl,d)
;model:

U(AppA) = Cond.dummy[0] * CONDITIONS [0,1]
+ fp* FPOSITIVE [0.1, 5, 55]
+ fn * FNEGATIVE [0.1, 5, 10]
+ inc * INCONCLUSIVE [0, 3, 5]
+ sex.dummy[0] * SEXR [0,1]
+ number * NTESTS [1,2]
+ wait * WTIME [2, 6, 10]
+ cost * SCOST [0, 500, 1000] /

U(AppB) = Cond * CONDITIONS
+ fp* FPOSITIVE
+ fn * FNEGATIVE
+ inc * INCONCLUSIVE
+ sex.dummy * SEXR
+ number * NTESTS
+ wait * WTIME
+ cost * SCOST

$

Do I need to include the opt-out within the experimental design? My thought process was that I don't as we will may present participants with a forced choice, and if there are too many opt-outs then I want to be able to analyse this data set, thus I want the experimental design efficiency to be maximised for these two options. Is this correct?

Thanks,
Amber

[johnr]

Hi Amber

There is a theoretically correct answer and a practical answer to this question. I will start with the practical answer. In general, most people optimize for one set of choices only and use that design irrespective - typically one would use the first choice - so include the no choice. This seems to work in practice, and I doubt anyone would question if you do this. Theoretically however, such choices represent a conditional set of rankings type data. By that I mean, if you select either A or B, then you only have information on the 1st ranked alternative in terms of the preferences for three alternatives shown. If they choose none however, you have full preference rankings, such that you know neither is preferred to A and B, and subsequently, you know if A is preferred to B or B is preferred to A. When you construct the data (not design), you create what are known as pseudo observations (just the language used) to represent the second choice (if a second choice was made). Ngene currently doesn't allow you to do this (that I know of).

The closest would be model averaging, but this is not the exact case you describe. This is because, model averaging (the code for which I show below) assumes that each model specification has some likelihood of occurring (defined by the user weights defined). That is, you are saying the model specification is correct for model 1 (weighted by some amount) and specification is correct for model 2 (weighted by some amount). This is not the same as saying some people may switch between model 1 and model 2 specifications depending on how they answer (in my case model 2).

So my answer to this (and happy for others to chime in) is that theoretically, you should account for both, but practically nobody does (and it may not be strictly possible to do so anyway).

John

Design
;alts(M1) = AppA, AppB
;alts(M2) = AppA, AppB, No
;rows = 60
;block = 5
; cond:
If (AppA.CONDITIONS=0, AppA.WTIME= [2,6]),
If (AppB.CONDITIONS=0, AppB.WTIME= [2,6])

;eff = M1(mnl,d) + M2(mnl,d)

;model(M1):
U(AppA) = Cond.dummy[0] * CONDITIONS [0,1]
+ fp* FPOSITIVE [0.1, 5, 55]
+ fn * FNEGATIVE [0.1, 5, 10]
+ inc * INCONCLUSIVE [0, 3, 5]
+ sex.dummy[0] * SEXR [0,1]
+ number * NTESTS [1,2]
+ wait * WTIME [2, 6, 10]
+ cost * SCOST [0, 500, 1000] /

U(AppB) = Cond * CONDITIONS
+ fp* FPOSITIVE
+ fn * FNEGATIVE
+ inc * INCONCLUSIVE
+ sex.dummy * SEXR
+ number * NTESTS
+ wait * WTIME
+ cost * SCOST

;model(M2):
U(AppA) = Cond.dummy[0] * CONDITIONS [0,1]
+ fp* FPOSITIVE [0.1, 5, 55]
+ fn * FNEGATIVE [0.1, 5, 10]
+ inc * INCONCLUSIVE [0, 3, 5]
+ sex.dummy[0] * SEXR [0,1]
+ number * NTESTS [1,2]
+ wait * WTIME [2, 6, 10]
+ cost * SCOST [0, 500, 1000] /

U(AppB) = Cond * CONDITIONS
+ fp* FPOSITIVE
+ fn * FNEGATIVE
+ inc * INCONCLUSIVE
+ sex.dummy * SEXR
+ number * NTESTS
+ wait * WTIME
+ cost * SCOST /

U(No) = 0 $