Size of pilot study

[rheinber]

Hi NGENERs
I am writing to learn from your experience. What is the size of a decent pilot study that allows to estimate sufficiently precise priors to construct a Bayesian d-efficient design for the main study? I saw that, in Bliemer & Rose 2011 Transp Res A, you used a small sample of 36 respondents who gave you a total of 216 choice observations. Is that a sample large enough to inform an experiment with say 5 attributes (3x2 levels, 2x4 levels)? Or is it just the best that you could do at the time?
Thanks for sharing your opinion on this question,
Chris

[johnr]

Hi Chris

As with all things, more is better. In the paper you are refering to however, we chose 36 for a very specific reason. In that paper, we used an orthogonal design during the pilot. The smallest orthogonal design had 108 choice tasks, so we blocked it into 18 blocks of size 6. So as to preserve orthogonality, we gave each block to exactly 2 respondents during the pilot - hence 2 * 18 = 36. Whilst I cannot speak for others, we typically use about 20 respondents as a pilot although I have been involved in some projects (in health economics) where the pilot was 180 respondents (about what we would use during the main field phase in other discipline areas).

John