choice-metrics.com

Hi, I am conducting a mode choice study in a suburban region of Australia.
I have determined my attributes, their levels and have conducted a small pilot study.
unfortunately my responses dribbled in, however that gave me the opportunity to run my model several times as they did so.
some of my priors seem to converge and some don't (my sample is still very small) however each iteration I ran through Ngene.
I noticed that across runs my sample sizes were fluctuating wildly! from about 350 participants needed right down to 25

is this normal?
what does this mean?
what is the right answer? (25 seems very low for a survey to get the opinions of a whole region)
Can you please tell me roughly how the minimum sample size is calculated?
for the values that haven't converged yet can I use the values from the literature or should I use all the priors from the same place? (the most recent run?)

Kind regards

The calculation for minimum sample size in Ngene is based on the standard hypothesis of testing for 95% significant levels, which case we want t-ratios of 1.96 or higher.

It holds that:
t-ratio = beta / se(beta)

This t-ratio depends on the priors for beta that you put in, and se(beta) depends on the number of respondents and the design as well as the priors for beta. Therefore, the sample size calculation is only correct if the priors are very close to the true beta values. If your sample size estimates fluctuate a lot, that means that your beta estimates (and priors) fluctuate a lot. Note that Ngene reports sample size S-estimates per parameter, and these will stabilise as the number of respondents and estimations you do grows. However, for the parameters that do not yet converge and fluctuate a lot, those S-estimates will be poor predictions and I would not trust them much. Therefore, it is best to look at the S-estimates that stabilise and ignore the S-estimates for the other parameters unless you are quite confident about their priors.

You could possibly transfer priors from the literature, but note that they may have a different scale, which means that you need to resize all parameters with the same scaling factor. You could take one of the parameters you trust in your model and scale all other priors from the literature towards that one. In other words, you can transfer ratios of parameters (like willingness to pay), but you should not transfer absolute values due to possible scale differences in the data collection methods.

Michiel