A way of checking the redesign after pilot studies

[Keiko Aoki]

Dear Members,

Now I am trying to make a better design with Ngene to conduct the surveys for a few groups, A, B, C.
After that, I am going to conduct pilot studies in each group.
Each group has several different characteristics.
So, I would like to compare among groups in the study.

At first, I will explain my situations .

Situations:
1. I make a first design Dz. It assume that priors are near 0 like b[0.00001] because of unlabeled design. I employ the D-efficient design.
2. I conduct pilot studies with Dz for each group A, B, C (about 100 persons).

By estimating the results of pilot studies, I can make two draft redesigns based on estimation results in the cases of pooled data and of each data.

So, I am wondering which design I should employ as the final one?

If the case of pooled data is better, I can estimate the results with final design and compare among groups. However, I am wondering that Dz prefer if D-error in the Dz is less than D-error in this case.

On the other hand, if the case of each data is better, I am wondering that I can not compare because redesigns in each group are different scenario. Also, I think that there is a problem which is different among D-errors in each group compared to it in Dz.

I was wondering if you could comments in my questions.

Thank you for your time in advance.

Best regards,
Keiko Aoki

[Michiel Bliemer]

I do not entirely understand you questions, but:

1. You can compare groups using a single design or using different designs per group. However, the statistical power in comparison would be lower if you use different designs, so you need a larger sample size. If the main purpose is to compare different groups, then using a single design is best. See also my response to your previous post.

2. You cannot compare D-errors of of a design with different priors. After estimating your parameters, you generate a Bayesian efficient design and use that one for your main study. No need to compare with your pilot design given that the priors are different. Note that the value of a D-error has no meaning at all, we only know that the lower the better, but you should not worry about the value itself and also not compare across different studies or when using zero priors.

Michiel

[Keiko Aoki]

Dear Professor Bliemer,

Thank you for your answer quickly.

In your answer 1, I am sorry not to be able to explain my questions enough.
I would explain to analyze between design, not within design.
I read your answers again.
So, I just find that using the same Ngene program can be compare even though a design is either between or within one.
That means that I should employ pooled data in all groups in between design.

Is that correct ?

In your answer 2, you explain to generate a Bayesian (d efficient) design after the pilot study.
Until now, I have understood that one should use the same design in the main design also (in my case, d efficient design).

Would you teach me the reason on changing the design ?

Thank you for your time.

Best regards,
Keiko Aoki

[Michiel Bliemer]

I do not really understand what you are trying to do, but you can estimate models with any design (even random designs) and you can compare designs and population groups with any designs. If you want to compare across multiple groups of people, or across multiple scenarios, then it is best to use the same design, if that is not the case, then you can use different designs. In most cases, you will pool the data in model estimation.

A Bayesian efficient design informed by priors obtained from a pilot study will increase the efficiency of you data collection. You can also use the same design you used in your pilot study, it will just not be as efficient as using informative priors (versus uninformative zero priors). Many people keep the same design for the main study and pilot study, so there in no issue in doing that.

Michiel

[Keiko Aoki]

Dear Professor Bliemer,

Thank you for the quick replay again.

I though a CE design with between and within designs simultaneously.
Although I should have explained it from the begging, I could not do that because I am not familiar to generating a design with Ngene.

Because of your answers, I will conduct a pilot study based on 0 priors in the D efficient design and then generate a Bayesian one by pooled pilot date.

At last, co-author and I are sure that it is really important to refine a CE design.
So, we start using Ngene because we believe that we can do our purpose with Ngene.
I am getting questions more as I study a way of using Ngene with the forum.
Especially, I had lot of effort to understand a setting priors.

But, we could receive a lot of tips with The forum.

We really appreciate you.
So, we write down you in the acknowledgement in our manuscript.

Thank you for your time.

Best regards,
Keiko Aoki