Wizard of Oz

In this type of test, a service is offered to customers, which they perceive as being automated, even though it is manually executed. This helps to quickly test the interest in and design of the service before making large technological investments.

The Wizard of Oz test essentially tries to fool potential customers into believing they are using a finished, automated offering, while it is still being run manually. It dates back to the use of the Mechanical Turk in the 17th century, where chess players were fooled into playing against a chess robot. In reality, a person under the table was moving the chess pieces with magnets. For testing purposes, this is a great format to try out your front-end offering as well as your whole business model without the need to build the back end. By letting potential customers use your offering, you can understand how they interact with it and how the infrastructure needs to be built to provide the most efficient service and the best experience. For it to work properly, you will need to invest time and resources to build the visual appearance of a finished offering. Thus, it is a test you should use later in the development process of your new offering.

Helpful Tips

Make it pretty: The key to convincing your customers that your offering is ready is to make it look as good as if it was finished. As you don’t invest in functionality, spend something extra in making it look exciting and ready to use.

Accept the time investment: Someone will have to deliver on the promise you are giving to customers. Be aware that you will spend a lot of time conducting this type of test and plan ahead.

Protect yourself: Things might go wrong. Maybe you are not able to deliver what you have promised. While this is fair game in a test environment, where the user knows it is a test, this does not work in this test format. Check the legal framework and adjust your claims and liability accordingly.

Use a fake name: Especially if you have imagined the perfect name for your offering, don’t use it for this test. Things tend to go wrong in tests and you do not want your offering to be associated with bad service or lengthy delivery.

Aardvark, a social search service later acquired by Google, finds answers to questions by exploring social media. It tested its service by searching for the answers manually, thus not needing to invest in the development of the necessary algorithm before understanding whether customers were interested.

How to Guide

Develop a blueprint of your offering without the underlying technology. Plan the steps needed to provide the service manually, allocate the necessary resources for the duration of the test, and create feedback options and success metrics.