Blocking best-worst design theoretically sound?

[rich_imr]

Greetings,

I am not sure if this question is fair game on this forum (please kindly let me know if it is not), but since Ngene is an important part of Applied Choice Analysis 2nd ed., I will assume I can ask it.

After reading pg 290 in Applied Choice Analysis 2nd ed (and chapter 6), I am unsure as to how they can justify theoretically generating a best-worst design with 136 choice sets and 34 blocks. I am likely missing something, but from what I read from other sources if a large number of items were to be assessed, a hybrid best-worst design would need to be used. For example, if 20 (or some large number) items were to be studied, then each respondent should go through an exercise where they choose some smaller top set of items (e.g. under 10 items with the aim to decrease the number of choice sets each respondent would need to complete to a reasonable number) from the 20 items and they perform the best-worst exercises on all items in their top set.

Could anyone direct me to any material or literature that addresses my question? Thank you very much for any answers.

Regards,

R

[johnr]

Hi R

You can ask any question you like, provided it is polite, we will try and answer it where possible.

Before I begin, please note that I did not write the part of the book to which you refer, and the design used in the paper referred to in the book to which I was a co-author was slightly different (there were actually three designs in the paper). That is neither here nor there however.

Having worked with both Jordan and Terry in the past, I have an idea about what is being done, although I have never heard of a hybrid BW design before. Let me summarise what I understand is happening.

1. The overall objective of the study called for a DCE (or BW case 3). They had too many attributes for the DCE (or BW3).
2. In order to cull the number of attributes, they used a BW case 1 where the objects where the extended list of attributes. From this, they obtained an attribute importance ordering.
3. They took the top X number of attributes (in one case, the top X and bottom Y) from the BW1 task, and put these into the DCE.

The hint here lies in 1 above – the objective of the study required a DCE. However, not all studies require a DCE, and sometimes, as with the paper referred to in the book, the objective can be achieved using just the BW1 (or 2 in this case) task – here the objective was to obtain attitudes, not cull a set of attributes for a further purpose. So unfortunately, I do not agree with you on this point,

Further, having done steps 1 to 3 myself once, I must admit that I do not necessarily agree that this is a good thing to do. Why? If you agree that BW1 (or 2) are object orientated, and DCEs or BW3s assume trade-offs between attribute levels across alternatives, then why is the very first assumption we make that the data generation processes (DGP) are the same for both tasks? Surely, that would be the last assumption one should make, but hey, I’m the nutter who brings his dog to work everyday, so I would be the last person to listen to on any topic really.
I will leave it there before I go on a major rant about how BW is being miss used to begin with. I’m trying to limit myself to one rant a day.

John

[rich_imr]

Thank you very much for your reply!