IMPORTANT MESSAGE ON BEST-WORST DESIGNS IN NGENE

[Michiel Bliemer]

Since the appearance of the book Hensher et al. (2015) "Applied Choice Analysis", there is confusion about the ability of Ngene to generate efficient designs optimised for best-worst choices. Below are some questions and answers that you may be interested in.

Q: Can Ngene 1.1.2 optimise designs specifically for best-worst choices?

A: No, the latest release version of Ngene only optimises for single best choices. This does not mean that the designs cannot be used for best-worst situations, but simply means that they are optimised for the best choice, and not for the worst choice. We have not mentioned or promoted best-worst designs in our manual or on our website.

Q: I see Ngene syntax in the Hensher et al. (2015) book “Applied Choice Analysis” for best-worst choices. Will this syntax work?

A: No, this syntax will not work, Ngene will generate an error message.

Q: How did this Ngene syntax end up in Hensher et al. (2015)?

A: Programmers of Ngene work together with a small number of academics to develop new methods and features, and to test them. These academics have had access to a test and research version of Ngene containing prreliminary code for optimising best-worst designs. Somehow by mistake syntax for this test and research version ended up in the book instead of syntax of the release version. This was outside the control of ChoiceMetrics and should not have happened.

Q: Can I have the test and research version of Ngene?

A: No, this version is strictly for testing and research only, it is not a stable version and may contain errors. Once certain features have been thoroughly tested, we aim to implement them into the release version, which is an entirely separate implementation.

Q: When will best-worst designs be included in the release version?

A: At this stage we cannot give a release date yet, as it has significant implications to the software and all the model types included in Ngene. We are continuously working on adding new features to Ngene and over the past years we have always provided updates for free to all Ngene users. Best-worst is one of these features we are working on, together with several others.

Q: I specifically purchased Ngene for generating best-worst designs. What can I do?

A: Although ChoiceMetrics has never claimed that Ngene is capable of best-worst designs and we have never promoted this, we understand that there may be some confusion because of the Hensher et al. (2015) book. Out of courtesy we are willing to refund (which also means deactivate) your Ngene licence on request if you purchased Ngene after June 2015 (when the book with these syntax files first appeared). Please email sales@choice-metrics.com before 4 December 2015.

Q: Are best-worst designs not the current state-of-the-art?

A: While there is a growing number of people applying best-worst choices in their surveys, there is some debate regarding this type of data collection. Some have argued that best-best is perhaps a better way of asking questions, more consistent with actual decision making. Others have warned about possible issues in model estimation when combining best and worst choices. Several empirical studies have showed great differences in estimation when including worst choices. The vast majority of surveys uses best-choice only, which is a generally accepted methodology. We would like to emphasize again that designs optimised for best choices can also be used for best-worst choice tasks. This comes at the cost of some design efficiency, although your standard errors may actually improve because more information is obtained per choice task.