Partial Profile Designs vs Two-step Design Approach

[rich_imr]

Hi All,

What is the best way to handle large numbers of attributes in discrete choice experiments?

Is it better to do a partial profile design or to do some two-step approach such as conducting an "initial study" using a partial profile design, then conduct a final study using the most important attributes, which were derived from the initial study (implicit in the second method would be to synthesize the learnings from both studies to get some ranking of all the attributes)?

I've done some searching, but haven't found any "defining" papers on the subject.

Any advice and/or direction is greatly appreciated!

Regards,

Richard

[Michiel Bliemer]

I believe that the most common way is to hold focus groups to determine which attributes are most relevant. Essentially all relevant attributes typically enter the choice experiment, although it depends on the research question whether some of the attributes can be assumed as given/fixed in the scenario and not vary across the alternatives.

If a large number of attributes is relevant for the study, then a partial profile design would be useful also in the final study in order to decrease choice task complexity while still being able to investigate preferences towards all relevant attributes. But if in a pilot study you find that certain attributes actually do not seem relevant (although it is debatable whether they are not relevant or simply cannot be estimated with a small pilot study sample size), then one may consider removing these attributes from the choice experiment. This will make the choice experiment simpler and allows more information to be captured on the remaining attributes. This is particularly useful for small sample sizes. If your sample size is large, then why not including all potentially relevant alternatives?

Michiel

[johnr]

I'm not sure if I am a fan or not, but some people use best worst experiments to cull large lists of attributes - they use a two step approach to the study - study one uses BW, then from that, study 2 use a DCE with attributes from the BW study. That is, the attributes are the alternatives in the best worst experiment (not the levels) and people choose the most important/least important attributes out of the sets. This gives you a relative ranking of attribute importance and they then use the top X attributes in the follow up DCE. I've seen this done on multiple occasions successfully. I'm a little sceptical for a few reasons however. Firstly, the data generation process between the BW and DCE tasks is probably very different. One you are getting importance ratings between attributes, and the other you are trading off levels of attributes. The problem might be that price isn't important if the product costs 20 cents, but hugely important if it costs $1000. Second, I guess how the BW task is framed is important to the outcome also. I saw this done in a health context once, where the BW task ranked the attributes, but it wasn't clear to me (or the researcher when I discussed this with her), whether you should include only the top X attributes, or a mix of top and bottom attributes. In her case, the ordering wasn't attribute importance, but some sort of ranking of attribute preference - and the lower ones were all the negative attributes whilst the top attributes were the positive attributes. It made no sense to me at the time that you would only include positively viewed attributes in the DCE, as the negative ones could also impact on choice in the DCE.

Its an intriguing idea, and as I said, where I have seen it done, it appears to have worked. Would I advocate doing it in practice however I don't know, but it is not too dissimilar to the use of qualitative research I guess, so there might be something there.