Behind the Curtain

Behind the Curtain - Paper-prototyping and Wizard of Oz Experimentation Site
Unfortunately, “if you build it they will come” only worked in the movie Field of Dreams.

This is a fast-paced workshop we hold regularly. The goal is for you to be as excited about seeing what other people do with your idea as you are building it in the first place. The workshop takes place over two days (and possibly one very long night). We ask people to build something, but don’t worry: we’re not going to break out the resistors and hammers. We thought it would be good to start with some materials that everyone can deal with (cardboard, markers, and maybe scissors – if you don’t run).

By building things to look like they would, and ‘work’ like they should, we can see how the design actually impacts people before we waste all of our solder and bits. We feed you, get materials, and bring some users. All you bring is your imagination.

Additional Details

Make-Believe and Day-Dreaming are necessary cognitive tools for the development of ideas. It may be hard to beleive, but designers also employ make-beleive and day-dreaming in more practical and applied ways. “Cognitive Walkthroughs” and their embodied relative the “Wizard of Oz” have been used by reserarchers, designers, filmmakers and artists to test their ideas before fully implementing. When working on large, complex and expensive projects it becomes clear that having a preview of the final product will help find bugs and user issues as well as serve as documentation of the design process.

Example 1: Expensive, polished and somewhat unrealistic… but cool!

In the late 90’s Apple Inc’s (then Apple Computer) CEO John Scully championed a project known as “Knowledge Navigator” which later spawned the fabled Apple Newton project.

Before development started on any real product Apple produced one of the most polished Wizard of Oz videos ever created. It received quite a bit of criticism from contemporary computer scientists who cited the fact that a few of the demonstrated technologies were unlikely to exist in the near future.

Despite these criticisms the “Knowledge Navigator” video was a valuable piece of “pre-visualization” that was used not only to flesh out a design, but to also inspire those who would later work on the Newton project.

Example 2: Quick, dirty and effective

It has been noted that the creators of South Park used paper cut-outs to create the early episodes of the show because it allowed Trey Parker and Matt Stone to quickly, cheaply and effectively realize their ideas. Despite the obvious lack of quality, the idea was successfully realized and this early prototype became the model of a popular TV show.

WoZ Experiments

The examples provided above are simply non-interactive videos - they are useless if one wanted to actaully do a user-study on their system with real experimental participants. It is possible to run experiments on a WoZ setup, but it requires a slightly different set of skills. If you have ever been involved in a school play or some other live dramatic endeavour yo already have experience with WoZ for experiments. Essentially those who are running the experiment do the computation/visualization in-place of the fictional system being tested.

In the early days of computer based speech recognition many WoZ implementations were devised to test scenarios. You can imagine how simple it would be to instantly implement a robust Natural Language Processor when using a human as the computer! In fact this is exactly how researchers at CMU started work on their NLP systems.

Audio interfaces are simple because it just requires someone to speak. What if you wanted to create a physical device like a phone? Well, it can be almost as simple, but it may require a bit more imagination on the part of your participants.

For example: You could make a mock-up of your phone with cardboard and markers. The screen could be made by drawing on stickers or post-it-notes. When a participant interacts with the phone those who are running the experiment must watch closely at what the participant is doing. The experimentors can then swap out appropriate interface stickers based on the participants actions. eg. if participant presses “Cancel” the previous screen’s sticker is placed on the phone. If the participant clicks “OK” then the appropriate response screen-sticker is placed on the mock-up.