Establishing priors

[arista]

Dear all

I am a PhD student and I am about to conduct a DCE in digital health. I am looking to find out how to present digital health products in the best possible way to increase their uptake and I would very much appreciate your input on my work.

I have 5 attributes, 2 with 4 levels and 3 with 3 levels. I am going to look at the main effect only, planning an MNL. There will be two unlabelled alternatives and an opt-out option ('neither'). I plan to generate 48 choice sets blocked into 4 versions, 12 choice-set in each survey.

My research is novel and there are no priors available, which is why I aim to pilot the DCE first. I have four questions.

1. Using the Ngene manual and reading research I figured out the code for the pilot using Dz efficiency design - with priors set to zero. Is my code correct?

Code: Select all

Design
;alts = alt1*, alt2*, alt3
;rows = 48
;block = 4
;eff = (mnl,d)
;model:
U(alt1) = b1[0] + b2[0] * cost [1,2,3,4] + b3[0] * developer [1,2,3] + b4[0] * ratings [1,2,3,4] + b5[0] * description [1,2,3] + b6[0] * images [1,2,3] /
U(alt2) = b2 * cost [1,2,3,4] + b3 * developer [1,2,3] + b4 * ratings [1,2,3,4] + b5 * description [1,2,3] + b6 * images [1,2,3]
$


2. I ran the code in Ngene and it gave me the following details: D error = 0.023; A error = 0.029; B estimate = 100 and S estimate = 0. Is it possible to get 100 as B estimate and 0 for S estimate?

3. Do I need to give the direction of the priors that are now set to zero (e.g. [0.0001] or [-0.0001])? If yes I think this would be difficult as to date, there is no data about this kind of research.

4. This may be a silly question. Once I have the data from the pilot study, what data will is use and how should I use that to find out the priors?

Many thanks in advance.

Best wishes,
Arista

[Michiel Bliemer]

The syntax looks mostly fine, answers to your questions are below.

1.

First, in generating efficient designs, you need to use the levels that you will actually use in model estimation. So if your four cost levels are $15, $20, $25, and $30 for example, you would typically use cost[15,20,25,30] in order to have a direct interpretation of the coefficients and being able to directly compute willingness-to-pay values using the coefficients.

Secondly, some attributes may be qualitative, for example "description" sounds like a variable that you may want to dummy code. In that case, you need to use:

... + b5.dummy[0|0] * description[1,2,0] + ...
where 0 is the base level and coefficients 0 and 0 belong to levels 1 and 2 relative to the base level

Thirdly, it is only possible to remove dominant alternatives if you provide the sign of attributes, for example cost likely has a negative sign and ratings sounds like it will have a positive sign. In that case, you could use b2[-0.00001] * cost[...] + b4[0.00001] * ratings[...]. Similar for qualitative variables, if you know that for "description" level 2 is likely better than level 1, and level 1 is likely better than level 0, you could use b5.dummy[0.00002|0.00001] * description[1,2,0]. For attributes for which you do not know the sign or ordering you can leave the prior to zero. If you only know the sign or ordering of 2 or 3 attributes, then there is no need to check for dominance and you can leave all priors equal to 0.

2. B-estimates are 100% because you are using zero priors, this means that each alternative gets a probability of exactly 33% and hence is completely probability balanced. You can ignore the B-estimate. S-estimates cannot be computed when putting in zero priors, they should be ignored if priors are not known. S-estimates can only be interpreted if you use appropriate priors that come from a pilot study.

3.
See my answer in 1 above.

4.
In order to estimate the choice model, you will need:
(a) the attribute levels in the design for all your attributes, e.g. cost levels developer levels, etc. You may need to dummy code some of these variables because you put them into estimation software.
(b) choice observations, e.g. alt1 was chosen in choice task 1, neither was chosen in choice task 2, alt2 was chosen in choice task 3, etc.
Estimation software will use data from (a) and (b) to find the most likely values for b1, b2, ..., b6, and if you dummy code some variables you get obtain more, e.g. b5(d1) and b5(d2) if you dummy code description.
Model estimation will give you parameter estimates B and standard errors S. Then you can for your final design use Bayesian priors like b[(n,B,S)].

Michiel

[arista]

Hi Michiel,

thank you very much for your response. It was very useful.
I read more around the dummy coding in the manual and updated the code. I also realised that I might be able to use the direction of priors based on my professional experience.

May I kindly ask you to have another look at the updated code, please? I want to make sure I got the directions of the priors right.

Code: Select all

Design
;alts = app1*, app2*, neither
;rows = 48
;block = 4
;eff = (mnl,d)
;model:

U(app1) = app[0]
+ cost[-0.00001] * COST [0,2.99,5.99,8.99]

+ developer.dummy[0,00002|0,0001] * DEVELOPER [1,2,0]
? 0 = does not show; 1 Mhealth Essentials =; 2 = NHS digital;
? my professional opinion is that 2 is preferred over the other two levels, and 1 which is a hypothetical company is preferred over 0

+ ratings.dummy[0.00003|0.00002|0.00001] * RATINGS [1,2,3,0]
? 0 = does not show, 1 = 3 stars, 2 = 3.8 stars, 3 = 4 stars;
? I need to use dummy as one level of this attribute is qualitative
? we know that the more ratings an app has the more likely people are going to select it and that people want to see the ratings, so 3 preferred over 2, 2 preferred over 1, and all preferred over 0

+ description.dummy[0.00002|0.0001] * DESCRIPTION [1,2,0]
? 0 = brief, 1 = long, 2 = short
? we know that people prefer short over the other two, and long over the brief, so 2 preferred over 1, and 1 preferred over 0

+ images[0.00002|0.0001] * IMAGES [1,2,0]
? 0 = logo, 1 = screenshot, 2 = logo and screenshot
? we know that people prefer 2 over 1, and 1 over 0
/
U(app2) = app +
+ cost * COST
+ developer * DEVELOPER
+ ratings * RATINGS
+ description * DESCRIPTION
+ images * IMAGES
$



All the best,
Arista

[Michiel Bliemer]

I made several changes in the syntax below, namely:

* I changed the priors to indicate the correct ordering; note that the first prior relates to the first attribute level (1), and the second prior relates to the second attribute level (2), both relative to the base (0)

* I changed a comma into a period in two places, e.g. 0,0002 should be 0.0002

* Given that it is difficult to find a design without any dominant alternatives using the default swapping algorithm, I changed it to the modified Federov algorithm

* Since the modified Federov algorithm does not guarantee attribute level balance, I ensured a high degree of attribute level balance for cost by imposing attribute level constraints, requiring each level to appear 10-14 times across the 48 choice tasks. Note that imposing constraints is not necessary for dummy coded variables because the D-error computations automatically make sure that a high degree of level balance results (for example, if a certain level does not appear much, then the dummy coded coefficient for that level could not be easily estimated and the D-error would be very high).

Please run this syntax for some time, the modified Federov algorithm is not very fast.

Code: Select all

Design
;alts = app1*, app2*, neither
;rows = 48
;block = 4
;alg = mfederov
;eff = (mnl,d)
;model:

U(app1) = app[0]
        + cost[-0.00001] * COST[0,2.99,5.99,8.99](10-14,10-14,10-14,10-14)
        + developer.dummy[0.00001|0.00002] * DEVELOPER[1,2,0]
          ? 0 = does not show; 1 Mhealth Essentials =; 2 = NHS digital;
          ? my professional opinion is that 2 is preferred over the other two levels, and 1 which is a hypothetical company is preferred over 0
        + ratings.dummy[0.00001|0.00002|0.00003] * RATINGS[1,2,3,0]
          ? 0 = does not show, 1 = 3 stars, 2 = 3.8 stars, 3 = 4 stars;
          ? I need to use dummy as one level of this attribute is qualitative
          ? we know that the more ratings an app has the more likely people are going to select it and that people want to see the ratings, so 3 preferred over 2, 2 preferred over 1, and all preferred over 0
        + description.dummy[0.00001|0.00002] * DESCRIPTION[1,2,0]
          ? 0 = brief, 1 = long, 2 = short
          ? we know that people prefer short over the other two, and long over the brief, so 2 preferred over 1, and 1 preferred over 0
        + images.dummy[0.00001|0.00002] * IMAGES[1,2,0]
          ? 0 = logo, 1 = screenshot, 2 = logo and screenshot
          ? we know that people prefer 2 over 1, and 1 over 0
        /
U(app2) = app
        + cost * COST
        + developer * DEVELOPER
        + ratings * RATINGS
        + description * DESCRIPTION
        + images * IMAGES
$

Michiel

[arista]

Thanks very much. I managed to conduct a pilot study with 49 participants that yielded 468 observations.

Best wishes,
Arista

[arista]

Dear Michiel,

I fitted MNL to get the priors from the pilot study, for a Bayesian D efficient design, and this is the syntax I wish to run in Ngene. For the pilot (see in one of our messages above), I used app[0] for alt1 and app for alt2. For this syntax, I took the constants for alt 1 and 2 out, but specified the opt-out option's constant. To my understanding, this is still fine, isn't it?
I would appreciate if you could take a look at this syntax, please. Is it correct? Also, is gauss 3 OK, or should I use gauss 2?

Thank you very much in advance.

Code: Select all

Design
;alts = app1*, app2*, neither
;rows = 48
;block = 4
;alg = mfederov
;eff = (mnl,d, mean)
;bdraws = gauss(3)
;model:
U(app1) = cost[(n,-0.446,0.03)] * COST[0,2.99,5.99,8.99](10-14,10-14,10-14,10-14)
        + developer.dummy[(n,0.172,0.229)|(n,-0.544,0.228)] * DEVELOPER[1,2,0]
          ? 0 = NHS digital; 1 Mhealth Essentials =; 2 = does not show;         
        + ratings.dummy[(n,-0.948,0.27)|(n,-0.213,0.249)|(n,-0.638,0.256)] * RATINGS[1,2,3,0]
          ? 0 = 4.8 stars, 1 = 3.2 stars, 2 = 4 stars, 3 = does not show         
        + description.dummy[(n,-0.081,0.244)|(n,-0.538,0.244)] * DESCRIPTION[1,2,0]
          ? 0 = short, 1 = long, 2 = brief         
        + images.dummy[(n,-0.436,0.232)|(n,-0.107,0.224)] * IMAGES[1,2,0]
          ? 0 = logo and screenshot, 1 = screenshot, 2 = logo
        /
U(app2) = cost * COST
        + developer * DEVELOPER
        + ratings * RATINGS
        + description * DESCRIPTION
        + images * IMAGES
/
U(neither) = neither [0]
$


[Michiel Bliemer]

1. You can put the constant either in alt1 and alt2, or in neither. So the model specification is fine.

2. You need to set the prior for the constant, it cannot be zero. You need to estimate the constant using your pilot study. Setting it to zero will lead to wrong choice probabilities for your design optimisation. Namely, the utilities of alt1 and alt2 will be much smaller than zero, meaning that in your choice probabilities people mostly prefer neither. Perhaps this is true, but it is best to estimate the constant and not just guess that it is zero.

3. Having such a large number of Bayesian priors means that doing 3 Gaussian draws for each of them results in 3^10 = 59,049 draws per design evaluation. That means that finding an efficient design will take a very long time. I suggest you chose 3 levels for attributes that are very important (i.e., the largest contribution to utility, which are cost and ratings in your case), and use 2 levels for all other attributes, which results in 5,184 draws per design evaluation, which is slow but acceptable. You can achieve this by using the following:

;bdraws = (3,2,2,3,3,3,2,2,2,2)

4. The modified Fevderov algorithm is very slow and since you are not using any constraints, it is not necessary. I suggest removing ;alg = mfederov in order to use the default swapping algorithm. This means you can remove the attribute level constraints for cost. And then run the syntax for a long time (at least a day or so).

The resulting syntax is below.

Code: Select all

Design
;alts = app1*, app2*, neither
;rows = 48
;block = 4
;alg = mfederov
;eff = (mnl,d, mean)
;bdraws = gauss(3,2,2,3,3,3,2,2,2,2)
;model:
U(app1) = cost[(n,-0.446,0.03)] * COST[0,2.99,5.99,8.99]
        + developer.dummy[(n,0.172,0.229)|(n,-0.544,0.228)] * DEVELOPER[1,2,0]
          ? 0 = NHS digital; 1 Mhealth Essentials =; 2 = does not show;         
        + ratings.dummy[(n,-0.948,0.27)|(n,-0.213,0.249)|(n,-0.638,0.256)] * RATINGS[1,2,3,0]
          ? 0 = 4.8 stars, 1 = 3.2 stars, 2 = 4 stars, 3 = does not show         
        + description.dummy[(n,-0.081,0.244)|(n,-0.538,0.244)] * DESCRIPTION[1,2,0]
          ? 0 = short, 1 = long, 2 = brief         
        + images.dummy[(n,-0.436,0.232)|(n,-0.107,0.224)] * IMAGES[1,2,0]
          ? 0 = logo and screenshot, 1 = screenshot, 2 = logo
        /
U(app2) = cost * COST
        + developer * DEVELOPER
        + ratings * RATINGS
        + description * DESCRIPTION
        + images * IMAGES
/
U(neither) = neither [0]  ? IMPORTANT: estimate the prior for this constant
$

Michiel

[arista]

Thank you very much.
While gathering parameter estimates was clear to me, I don't understand how can I estimate prior for a constant, either for alt 1 or 2, or the neither option?

Best wishes,
Arista

[Michiel Bliemer]

You estimate a constant in the same way as you estimate other coefficients when you estimate a model, so I am not sure i understand your question?
Whether you use Biogeme, Nlogit, Apollo, or other estimation software, you need to specify the utility functions including all coefficients (that may contain constants).

Depending on your estimation software, you may need to create a column with ones in your data for the constants, i.e. V = b0 * 1 + b1 * cost + ...

I refer to the manual of your estimation software.

Michiel

[arista]

Thank you. Unfortunately, I am not familiar with any of that software I will just start learning to use them. I gathered priors for attributes using SPSS which is unable to give estimates for the alternatives. However, I used now Stata and I was able to estimate coefficients of alt 1 and 2 (the opt-out option was the baseline). Is the code correct, please?

Code: Select all

Design
;alts = app1*, app2*, neither
;rows = 48
;block = 4
;eff = (mnl,d, mean)
;bdraws = gauss(3,2,2,3,3,3,2,2,2,2)
;model:
U(app1) = app [(n,1.152,0.14)]
        + cost[(n,-0.446,0.03)] * COST[0,2.99,5.99,8.99]
        + developer.dummy[(n,0.172,0.229)|(n,-0.544,0.228)] * DEVELOPER[1,2,0]
          ? 0 = NHS digital; 1 Mhealth Essentials =; 2 = does not show;         
        + ratings.dummy[(n,-0.948,0.27)|(n,-0.213,0.249)|(n,-0.638,0.256)] * RATINGS[1,2,3,0]
          ? 0 = 4.8 stars, 1 = 3.2 stars, 2 = 4 stars, 3 = does not show         
        + description.dummy[(n,-0.081,0.244)|(n,-0.538,0.244)] * DESCRIPTION[1,2,0]
          ? 0 = short, 1 = long, 2 = brief         
        + images.dummy[(n,-0.436,0.232)|(n,-0.107,0.224)] * IMAGES[1,2,0]
          ? 0 = logo and screenshot, 1 = screenshot, 2 = logo
        /
U(app2) = app [(n,1.09,0.14)]
        + cost * COST
        + developer * DEVELOPER
        + ratings * RATINGS
        + description * DESCRIPTION
        + images * IMAGES

$